1. Field of the Invention
The present invention relates to an image processing apparatus and an image processing method for capturing, recording, and playing back still and moving images and, in particular, to a correction technology of a captured image.
2. Description of the Related Art
Recently, image processing apparatuses such as digital cameras have become available on the market. The image processing apparatuses record and play back a still image and a moving image captured by a solid-state imaging device, such as CCD (Charge-Coupled Device) and CMOS (Complementary Metal-Oxide Semiconductor) devices, using a recording medium, such as a memory card including a solid-state memory device.
When capturing an image using a solid-state imaging device, such as CCD and CMOS devices, dark noise can be corrected through a computing process using two types of data: dark image data, which is acquired from the imaging device after charge accumulation under the same conditions as actual shooting without exposure, and actually shot image data with exposure. This dark noise correction process can compensate for image quality degradation due to dark current noise generated by the imaging device and missing pixels due to a small defect specific to the imaging device. Thus, the captured image data can be corrected so as to provide a high-quality image.
Additionally, by arithmetic interpolation using image data of pixels adjacent to a defective pixel, a point defect can be corrected to further reduce the image quality degradation.
In a known correction method of a defective pixel, an output of a sensor after a standard charge accumulation time under predetermined conditions is evaluated to detect a defective pixel when the sensor is shipped from a factory. Then, the type of the defective pixel (e.g., a black spot or a white spot), the address (e.g., positional data (x, y)) of the defective pixel, and level data of the defect are acquired to correct the defective pixel using these data (refer to, for example, Japanese Patent Laid-Open No. 2000-23051).
It is known that a defective pixel, in particular, a white-spot defect, significantly varies its level depending on a charge accumulation time during shooting. Thus, even a defective pixel which does not cause a problem in an ordinary several-second shooting period causes image quality degradation in a long shooting period, since the long shooting period significantly increases the level of the defective pixel. In particular, without the above-described dark noise interpolation correction, a small defect is not corrected. Therefore, image quality degradation due to the missing pixel noticeably appears during a long shooting period.
However, if a defective pixel is detected in accordance with a level of the defective pixel during a long shooting period and if the correction process is carried out for all the pixels, the number of corrections for the defective pixels significantly increases.
In known interpolation methods for a defective pixel, the level of the defective pixel is generally replaced with a value computed from the outputs of pixels adjacent to the defective pixel and having the same color as the defective pixel.
If the arithmetic interpolation is carried out by firmware using image data of pixels adjacent to the defective pixel stored in a memory after capturing, the following processes are required: a readout process of the outputs of adjacent pixels having the same color, a computing process, and a writing process of the corrected value to the defective pixel. Therefore, as the number of defective pixels increases, a considerable amount of time is required to access the memory. This long operation time creates a bottleneck, such as a release time lag and a delay between shots.
On the other hand, to decrease such a long operation time, the process carried out by hardware is proposed. For example, in Japanese Patent Laid-Open No. 2000-23051, the address of a defective pixel is recorded in advance. If the readout address is identical to the recorded address, the output of the defective pixel is replaced with the output of a pixel adjacent to the defective pixel. This hardware process can provide a sufficient correction for a single defective pixel. However, if adjacent multiple defects, in which a plurality of adjacent pixels have a defect, exist and if a high-quality correction is expected in which the result of defect correction is invisible, pixels in the vicinity of the defective pixel should be used for the interpolation. For example, if the output of the defective pixel is replaced with the output of the previous pixel having the same color by hardware and if the consecutive pixels having the same color in the horizontal direction have defects, all the outputs are replaced with the output of the previous pixel. Accordingly, for, in particular, an image having a high spatial frequency, the result of the correction becomes visible, which is a problem.
The firmware process, as described above, can make an appropriate determination at a readout time of pixels in the vicinity of the defective pixel. Accordingly, an optimum correction method can be applied to the consecutive defective pixels. For example, even consecutive defective pixels can be corrected by first correcting the outermost defective pixel among the consecutive defective pixels using outputs of non-defective pixels in the vicinity of the defective pixel, and then sequentially correcting the inner defective pixels. However, the hardware that carries out the real-time correction using the outputs in the vicinity of the defective pixel during a readout operation requires, at a time, a large amount of processing memory for storing outputs of pixels in the vicinity of the defective pixel. Thus, the hardware configuration becomes significantly large and complicated, and therefore, the hardware method is not practical.
USPTO Backgrounds
USPTO Backgrounds