Display panels or monitors continue to increase in resolution, such as 1080p, 4k×2k, etc. Many image sources have a lower resolution than the displays. When the panel displays these sources in a display with higher resolution, traditional scaling techniques do not produce high fidelity results with rich details and sharp edges. Examples of these traditional techniques include one-dimensional poly phase scaling (1D scaler), bilinear interpolation, bi-cubic interpolation, edge-guided (2D) scaling, etc. Super resolution techniques offer a solution to bridge the gap between advanced, higher resolution displays and lower resolution video sources.
Multi-image super resolution techniques construct a high resolution image from multiple low resolution images by fusing information among them. These algorithms can bring some details with smaller ratios of magnification in zooming. When larger ratios of magnification are needed, they cannot give enough details. Further, even at smaller ratios, if the motion is not in the right range, the ability of these techniques to reconstruct high frequency details suffers. These techniques also require many hardware resources and complex logic.
Different types of single frame super resolution (SFSR) techniques exist. Typically, these processes may involve using comparisons between the original low resolution image and various scaled versions of the low resolution image to generate the high resolution image at the desired magnification, or higher resolution. However, they generally do not recover enough details and can even suppress details that look like noise. In fact several of the algorithms use techniques similar to non-local means noise reduction. These approaches keep the sharpness of edges at the expense of more random details. Because the techniques prefer edges over details, the edges have abrupt transitions, making the resulting image look like an oil painting with hard edges between colors.
The original low resolution image provides the best source for details and edges which can be taken as examples for the resulting higher resolution image, if a match can be found Using the original image provides details and edges that are more natural when compared to other intermediate layers in most self similarity super resolution (SSSR) processes. Using the original low resolution image, often referred to as layer 1×, has benefits. However, a problem arises because in most SSSR processes more than 90% of the high-resolution examples come from the closest low resolution layer (25-33% increase in magnification) and not the 1× layer. Typically, this results from a comparison of the data in the closest low resolution layer and the original layer using the sum of absolute differences (SAD) or a weighted sum of square distance (SSD) process. The closest lower resolution layer to the desired high resolution layer typically has the smallest differences. This results in the data from the closest lower resolution layer being used, rather than the data from the original low resolution layer.
USPTO Backgrounds
USPTO Backgrounds