When an imaging device such as a camera takes pictures under one or more sources of light, the image will have a color bias depending on the color and temperature of the specific source(s) of light. For example, under light generated from a tungsten source, un-modified pictures will have an overall yellowish-orange cast. Under natural lighting during twilight however, images will often have a very bluish cast. In order to mitigate the potentially heavy color biasing that occurs under varying light conditions, adjustments are typically performed either internally within the device or during the processing phase to balance the light so that the resulting images appear relatively normalized to the human eye.
According to contemporary photographic techniques, each pixel in a scene or image can be represented as a vector with one dimension for each of a multitude of color channels. For example, in a three color image, each pixel can be represented as a three dimensional vector (e.g., typically the vector [R,G,B]). This vector can be projected down to a lower dimensional space, such as by transforming it to a luminance/chrominance color space such as YUV coordinates. The YUV pixel can then be represented by just its color terms as a two dimensional vector [u,v]. In a two dimensional space, the color of common illuminants will have a distribution that falls mostly along a curve in color space. Plankian illuminants (ideal blackbody radiators) have a color of light that vary with one dimension, the temperature color. The temperature colors form a smooth curve in color space. Many common lamps radiate colors that are similar to Plankian illuminants, so they tend to fall along this curve. The curve ranges from blue (high temperatures) to red (low temperatures).
There exist several approaches to automatic white balancing. In several conventional approaches, characteristics of an image (e.g., attributes of the pixels comprising the image) are used to estimate the color of the illumination. This illumination, represented as a value, is subsequently factored out of the pixel colors. A popular method is known as the “Gray World” approach. According to the Gray World method, the color values corresponding to pixels of an image are averaged and the average color of the image is used as the estimated color of the illuminant (and thus, removed). Factors of scale on each color channel are chosen so that the average color, after scaling is performed, results in a neutral color value.
Unfortunately, the estimated illuminant color derived from the average of the pixel values is sub-optimal for the purposes of normalization. In certain circumstances, the estimated illuminant color can be a highly unlikely color for an illuminant and factoring the illuminant color out of the image will result in images with distorted coloring. For example, in scenes with mostly green foliage, the average color value will be a value that approximates some shade of green. According to the Gray World model, the illuminant will be estimated as a green light and will be subsequently factored out of the image, thus resulting in foliage that appears neutral, i.e., gray, adversely affecting the appearance of the image.
Another problem with the Gray World method is that large colored surfaces can bias the estimate for the entire scene or image. For example, in scenes that are comprised by large portions of blue sky, the gray world technique will over bias the illuminant color as blue. After the blue illumination is factored out, the sky will turn gray (neutral) and the other image areas will look yellow. Previous attempts to solve this problem have included removing pixels that were too similar to adjacent pixels, so that a large colored area would be reduced to a smaller, representative patch. Unfortunately, this method is inaccurate and ineffective. With fewer samples, the estimate becomes less stable and less reliable. Also, in many cases, a color can fill a large part of a scene, but the color is not continuous, and therefore would not be affected. Also, there are often pixels that are colored by random sensor noise. These pixels can be awarded two much weight during the illuminant determination and bias the estimate after the other, more common colors have been reduced.
USPTO Backgrounds
USPTO Backgrounds