What method is used to calculate the 'quality' of a solar/planetary image? What sort of algorithm is used to calculate the relative quality of an image, such as is performed by PIPP, Registax, or Avistack? 

Such applications take planetary or solar images and prior to stacking and sharpening, they can optionally select the best images. When looking at images to see which have been selected by the algorithms, I sometimes see images that I wouldn't have selected. This has piqued my curiosity as to what methodology is employed in ranking or grading the images. With a bit of a primer to understand the concepts, I would like to put some thought into how such algorithms might be improved...
A little further research, and I found out that the basic principle, from which several variations derive, is to take the sum of the squares of the difference between adjacent pixels, to get a score. 

The principle is that a higher quality picture has a higher probability that there will be significant differences in adjacent pixel values, i.e. that there can be a significant variation from pixel to pixel, whereas a lower quality picture will have similar valued adjacent pixels, as they have "blurred" together. Therefore, when taking the square of the difference of adjacent pixels and summing them together, higher quality pictures will show a significantly higher score, or summation.

I'm going to demonstrate this with a simplified example below, but I'm interpreting the meaning of "adjacent pixels" in a particular way, and I am hoping that if I am incorrect, someone will tell me and I can modify this answer appropriately.

Imagine we are imaging an object that exactly fills the frame of our small 16-pixel camera. In the middle of this object is a very dark square, that coincidentally happens to exactly fill the central 4 pixels of our camera. If the seeing for this object is perfect, then the camera will be exposed with a value of 255 for each of the light pixels, and 0 for each of the dark pixels.


This is how the differences between adjacent pixels would be calculated (I'm assuming only horizontal and vertical differences, not diagonal):


However, movement of the camera or poor seeing resulted in the light being spread more between the pixels, the calculations of the differences between adjacent pixels would be different:


The score, that is the sum of the squares of the differences, for the first (perfect) image is 520200.

The score for the blurred image is 304200. Therefore the sum of the squares of the differences between adjacent pixels suggests that the first picture is the better one.

This strategy isn't perfect, obviously. The presumption is that better images will exhibit a better score due to larger differences between adjacent pixels, but this is an assumption. 

The squaring of the difference is intended to provide greater weight to larger differences, but I came across a comment on the PIPP support website that says that PIPP has modified the basic algorithm to give still greater weight to sharper contrast changes over lower contrast changes - so maybe squaring the differences isn't enough sometimes. 

Apparently this quality measurement strategy doesn't work when the object is small in size on the sensor, and may be bright or even over-exposed. The above methods may select blurred and smeared frames over frames that are actually better. In this case, a strategy that looks at the histogram of values for the frame and selects the one with the higher peak, may produce a better selection of appropriate 'best quality' frames. It assumes that the histogram will have a higher count of frames sharing a pixel value across a smaller range, rather than spreading the light out across a wider range of pixel values.

An alternative quality measurement strategy simply adds up the values of all the pixels in an image. A higher score will indicate more light got through to the sensor, which implies that less cloud (for instance) got in the way.