1. Field of the Invention
The present invention relates to apparatus for and method of performing edge enhancement in a widely used popular digital camera, such as a portable digital camera, having a low number of image pixels, and more particularly, to edge enhancement apparatus and method to simultaneously perform both an edge enhancement operation and an interpolation operation without any additional circuit or block for edge enhancement.
2. Description of the Related Art
In a portable digital camera or a widely used popular digital camera, a sensed image is not vivid since an image sensor and a lens are small in size, and a function of an image processing IC has been too much simplified for a cost reduction. In particular, a boundary of the sensed image becomes blurred. In order to prevent the sensed image from being blurred, an edge enhancement method has been employed in the digital camera to enhance the boundary of the sensed image to obtain a vivid sensed image.
An edge of the sensed image contains information about the sensed image itself. The edge of the sensed image represents the boundary at which a position, a shape, and a size of an object of the sensed image are changed. The edge of the sensed image is disposed on a point at which the brightness of the sensed image is changed from a high brightness level to a low brightness level or from the low brightness level to the high brightness level.
As described above, the edge enhancement method is used to obtain the vivid sensed image in the widely used popular camera having a number of image pixels less than a standard. Among a number of edge enhancement methods, a method of detecting an edge of an original image using various methods and adding the detected edge to the original image to enhance the edge of the original image has been widely used.
The edge enhancement method is classified into one of various methods according to a kind or type of an input image, and a fastest and simplest edge enhancement method is to calculate pixel data of a center pixel and adjacent pixels of the center pixel to determine a maximum value. This method is one of a homogeneity operator method and a difference operator method.
FIGS. 1A and 1B show views explaining conventional homogeneity operator and difference operator methods, respectively. As shown in FIG. 1A, the homogeneity operator method includes forming a window having 3×3 pixels, subtracting pixel values of adjacent pixels disposed around of a center pixel of the window from a pixel value of the center pixel to obtain differences, and outputting a maximum absolute value among absolute values of the differences as an output value. Since the homogeneity operator method performs obtaining the differences between the pixel value of the center pixel and the pixel values of the adjacent pixels of the window, 8 subtracting operations are required with respect to the center pixel of the window.
As shown in FIG. 1B, the difference operator method includes obtaining a first difference between pixel values of an upper left pixel and a lower right pixel of a center pixel, obtaining a second difference between pixel values between an upper center pixel and a lower center of the center pixel, obtaining a third difference between pixel values of an upper right pixel and a lower left pixel of the center pixel, and obtaining a fourth difference between pixel values of a middle left pixel and a middle right pixel of the center pixel. Since the difference operator method requires 4 subtracting operations with respect to the center pixel, the difference operator method is faster than the homogeneity operator method in performing the subtracting operation.
The above-described homogeneity and difference operator methods are performed with respect to a black/white image having a brightness component.
Various edge detecting methods are used for a color image which is not the black/white image. Generally, in the color image having an RGB (R, G, and B components) space, operations are performed with respect to the pixel values of each color components of R, G, or B, and a gray scale edge map is produced from operation results using the following formula 1.G(x,y)=√{square root over (G2red+G2green+G2blue)}  <Formula 1>
Here, Gred represents a pixel value of a red component, Ggreen represents the edge map of a green component, and Gblue represents the edge map of a blue component.
However, in an HSV (hue, saturation, and value (brightness or luminance) components) space, the edge of the color image is detected from the value (V) component among the hue (H), saturation (S), and value (V) components. Therefore, it is possible to detect the edge of the color image by using the value (V) component rather than all color components, such as the H, S, and V components,
FIG. 2 shows a block diagram of a conventional edge detector detecting the edge of the color image using a luminance component. Referring to FIG. 2, the sensed image sensed by an image sensor 201 is stored in a memory 202 as a Bayer pattern. Since 3 or more data components are required to realize the color image, pixel values of three independent color (R, G, and B) components can be used as the data components. The image sensor 201 sensing the color image requires a color filter away (CFA) in which color sensors are arranged. In each color sensor of the image sensor 201, the pixel value of one of the color components is extracted with respect to each pixel, other color components of the pixel, which is not extracted with respect to the pixel, can be calculated through the CFA using information of adjacent pixels of the pixel. This method is a generally known Bayer pattern method corresponding to a CFA method. Restoring the color image through the image sensor 201 having the CFA is called interpolation and demosaicing.
The sensed image (image information) stored in the memory 202 is processed to a RGB pattern image in a 3×3 line interpolation unit 203.
Assuming that the interpolation unit 203 performs a 3×3 line interpolation method with respect to the image information stored as the Bayer pattern in a unit of a 3×3 window having the number of 3×3 pixels, since the Bayer pattern is one of four patterns, the 3×3 window is one of the following four patterns.
00: rgrgrg . . .                gbgbgb . . .        
01: bgbgbg . . .                grgrgr . . .        
10: gbgbgb . . .                rgrgrg . . .        
11: grgrgr . . .                bgbgbg . . .        
When a 3×3 line Bayer pattern which is shown in FIG. 3A is provided to the interpolation unit 203, RGB values of the RGB pattern are as follows.                R=(R1+R2+R3+R4)/4        G=(G1+G2+G3+G4)/4        B=B1        
When the 3×3 line Bayer pattern which is shown in FIG. 3B is provided to the interpolation unit 203, the RGB values of the RGB pattern are as follows.                R=(R1+R2)/2        G=G3        B=(B1+B2)/2        
When the 3×3 line Bayer pattern which is shown in FIG. 3C is provided to the interpolation unit 203, the RGB values of the RGB pattern are as follows.                R=(R1+R2)/2        G=G3        B=(B1+B2)/2        
When the 3×3 line Bayer pattern which is shown in FIG. 3D is provided to the interpolation unit 203, the RGB values of the RGB pattern are as follows.                R=R1        G=(G1+G2+G3+G4)/4        B=(B1+B2+B3+B4)/4        
The 3×3 line Bayer pattern interpolation method may include a method of obtaining a mean of pixel values of the adjacent pixels excluding the pixels having maximum and minimum pixel values as well as the above-described method. In order to perform the 3×3 line interpolation method, 3 line data should be simultaneously transmitted to an image processor.
The RGB pattern image interpolated in the interpolation unit 203 is processed in an edge detector 206 through YCrCb(YUV) pattern converter 204 and another memory 205.
However, when the interpolation and the edge enhancement are processed on the sensed image using the above conventional methods to realize an image processed block, a time delay occurs between an input image and an output image since the interpolation and the edge enhancement are performed in a unit of a frame, and also a frame buffer and an additional memory are required to temporarily store the input image and the edge map, respectively, since the interpolation and the edge enhancement are performed in a unit of a frame.
The sensed image is not vivid since functions of an image processor IC is too much simplified to sense a clear image as the input (sensed) image due to cost reduction, and the image sensor and a lens of the digital camera having a low number of pixels are small in size. Particularly, the boundary (edge) of an object in the sensed image becomes blurred. Although the edge of the object of the sensed image can be enhanced using the conventional edge enhancement method to obtain a more vivid image, an edge enhancement apparatus employing the conventional edge enhancement method requires an additional function unit (block) in addition to an interpolation processing unit (block) as shown in FIG. 2.
Japanese patent laid-open No. hei7-107268 discloses a conventional image processing apparatus for obtaining high resolution information for a sharp edge by calculating the high resolution information adjacent resolution information in a linear interpolation unit and by adding edge information of an edge information unit to linear interpolated high resolution information.
However, since the above conventional image processing apparatus is an apparatus for processing an edge processing operation after the interpolation has been performed, the additional function block is required in addition to the interpolation block. Moreover, the above conventional image processing apparatus cannot be properly employed in the digital camera or the widely known popular digital camera.
A new method of simultaneously performing the edge enhancement and the interpolation without any additional function block needs to be easily employed in the digital camera or the widely known popular digital camera regardless of the size or the cost of the image processing IC.
USPTO Backgrounds
USPTO Backgrounds