Generally, a digital photographing device such as a digital camera or digital camcorder uses a charge coupled device (CCD) that uses various information for each pixel in order to obtain a full-color image. At least three types of color data are required in order to display an image viewable by human eye. A full-color image can be rendered based on pixel values for three independent colors, R, G, and B.
The CCD is a photographing device for converting an optical signal into an electric signal. CCD may be a single chip type or a multi-chip type. In a multi-chip type, each pixel receives three colors by using three chips having sensors reactive to three colors (R, G, and B) respectively. In the single chip type, each pixel receives only one color, and a color filter array (CFA) having sensors reactive to each color. The most general pattern of the CFA is a Bayer pattern.
In case of the multi-chip type, each color for constituting one screen has information for the entire screen. Accordingly, it is possible to re-construct the entire screen by using the colors. However, in case of the single-chip type shown in FIG. 1, each pixel is provided with a different sensor for receiving a different color, even if sensors for receiving three colors are positioned on one chip. Accordingly, each pixel has only one color information among three colors.
For example, in case that a B pixel or a R pixel has no green detecting sensor to restore a green color value (G value), green is rendered by using information received by a green detecting sensor of an adjacent pixel. That is, a color interpolation algorithm for rendering the green color is used.
The related art color interpolation method in the single-chip type Bayer pattern color filter comprises a bi-linear interpolation method, a color shift interpolation method, and an adaptive interpolation method using a gradient of a brightness. Each interpolation method is explained below.
FIG. 1A is a view showing a 5×5 Bayer pattern in a bi-linear interpolation method. As shown, in the bi-linear interpolation method, color information of the most adjacent four pixels (G3, G7, G9, and G13) is used in order to restore a G value (for example G8), and an average of the adjacent four pixels (G3, G7, G9 and G13) is used as shown in the following formula 1.
                              G          8                =                                            G              3                        +                          G              7                        +                          G              9                        +                          G              13                                4                                    [                  Formula          ⁢                                          ⁢          1                ]            
FIG. 1B is a view showing a 5×5 Bayer pattern in a color shift interpolation method. As shown, the color shift interpolation method is for obtaining all the adjacent G values by using the bi-linear method and then obtaining an R8 by using a previously-obtained G8 based on an R-value known by a sensor.
It is assumed that a ratio between an R value and a G value adjacent to a pixel B8 to be restored is constant, and then an average for the ratio between the R value and the G value is calculated among the adjacent four pixels G2, G4, G12, and G14. As shown in the following formula 2, the calculated average value and the G value (G8) of the pixel (B8) are multiplied to each other thereby to restore the R value (R8).
                              R          8                =                              G            8                    ⁢                                                                      R                  2                                                  G                  2                                            +                                                R                  4                                                  G                  4                                            +                                                R                  12                                                  G                  12                                            +                                                R                  14                                                  G                  14                                                      4                                              [                  Formula          ⁢                                          ⁢          2                ]            
FIGS. 2A and 2B are views showing a Bayer pattern in an adaptive interpolation method using a gradient of a brightness. In FIG. 2A, α denotes vertical edge information, and β denotes horizontal edge information. The following formula 3 is used to obtain the vertical edge information α and the horizontal edge information β.α=abs[(B42+B46)/2−B44]β=abs[(B24+B64)/2−B44]  [Formula 3]
Once the vertical edge information a and the horizontal edge information β are obtained, it can be determined if a color shift is less in a horizontal axis direction or in a vertical axis direction. If α is less than β, the color shift in the horizontal axis direction is less than the color shift in the vertical axis direction. On the other hand, if α is greater than β, an average value between G34 and G54 is determined as G44. Also, if the α is equal to the β, an average value among the adjacent values, G34, G43, G45, and G54 is determined as G44 (Refer to formula 4).
                              G          44                =                  {                                                                                                                (                                                                        G                          43                                                +                                                  G                          45                                                                    )                                        /                    2                                    ,                                                                                                  if                    ⁢                                                                                  ⁢                    α                                    <                  β                                                                                                                                                (                                                                        G                          34                                                +                                                  G                          54                                                                    )                                        /                    2                                    ,                                                                                                  if                    ⁢                                                                                  ⁢                    α                                    >                  β                                                                                                                                                (                                                                        G                          43                                                +                                                  G                          45                                                +                                                  G                          34                                                +                                                  G                          54                                                                    )                                        /                    4                                    ,                                                                                                  if                    ⁢                                                                                  ⁢                    α                                    =                  β                                                                                        [                  Formula          ⁢                                          ⁢          4                ]            
When a G value (for example G44) has been restored, as shown in FIG. 2B, an R value and a B value are obtained in a condition that a ratio between R and G values (R:G) and a ratio between B and G values (B:G) are constant (Refer to formula 5).
                                                                        R                34                            =                                                                                          (                                                                        R                          33                                                -                                                  G                          33                                                                    )                                        +                                          (                                                                        R                          35                                                -                                                  G                          35                                                                    )                                                        2                                +                                  G                  34                                                                                                                        B                34                            =                                                                                          (                                                                        B                          33                                                -                                                  G                          33                                                                    )                                        +                                          (                                                                        B                          35                                                -                                                  G                          35                                                                    )                                                        2                                +                                  G                  34                                                                                        [                  Formula          ⁢                                          ⁢          5                ]            
FIG. 3 is a view showing a Bayer pattern in an adaptive interpolation method using a gradient of a brightness. Gdiff—ver of FIG. 3 denotes a difference between right and left values of a pixel to be restored (for example G44), and Gdiff—hor denotes a difference between upper and lower values of the pixel to be restored (for example G44).
The following formula 6 is used to obtain a vertical value and a horizontal value (Gdiff—ver, Gdiff—hor).Gdiff—hor=abs[G54−G34]Gdiff—ver=abs[G43−G45  ][Formula 6]
The obtained horizontal and vertical values (Gdiff—ver, Gdiff—hor) are compared with arbitrarily set thresholds to determine a G value by using the following formulas.(Gdiff—hor>threshold)AND(Gdiff—ver>threshold)→G44=(G34+G45+G54+G43)/4  1.(Gdiff—hor<=threshold)AND(Gdiff—ver<=threshold)→G44=(G34+G45+G54+G43)/4  2.(Gdiff—hor<=threshold)AND(Gdiff—ver>threshold)→G44=(G43+G45)/2  3.(Gdiff—hor>threshold)AND(Gdiff—ver<=threshold)→G44=(G34+G54)/2  4.
When the G value for every pixel is restored by the above formulas, an R value and a B value are restored in the same manner as the interpolation method by using a gradient of a brightness. The thresholds are differently set according to each image sensor thereby to optimize each image sensor.
The interpolation method can be largely divided into a bi-linear interpolation method, a color correction interpolation method, and an interpolation method using a spatial correlation. The bi-linear interpolation method requires less calculation and is simple to implement. However, the method causes a zipper effect and a blurring phenomenon.
The color correction method comprises a color shift interpolation method, and an implementation method using a gradient of a brightness. The color correction method maintains a soft color by using a color difference and a color ratio. The color correction implementation method is provided at a camera due to a simple implementation and a constant color.
The interpolation method using a spatial correlation produces the best quality image by using a color difference and a shift ratio. However, the method's implementation is complicated and requires heavy calculations. Also, since the method uses only upper/lower components and right/left components, a blurring phenomenon may result when a biased line or an edge exists in an actual image.
In the related art interpolation methods, when calculation is simple, picture quality is reduced. And when calculation is complicated, picture quality is improved but calculation overhead is increased. Also, in case of a diagonal image or a biased image, picture quality is reduced since only the upper, lower, right, and left components are considered without consideration of the diagonal components.
A solution to the above problems is needed.
USPTO Backgrounds
USPTO Backgrounds