Many popular consumer electronics applications such as cameras and mobile handsets as well as professional electronic applications such as laser-triangulation, video surveillance, monitoring employ complimentary metal oxide semi-conductor (CMOS) image sensors. Various applications pose different requirements on the spatial and temporal resolution. In general as sensor resolution increases, the operation of the sensors in desired frame rates involves higher data reading-out rates.
FIG. 1 illustrates a typical CMOS image sensor. The CMOS image sensor includes a pixel area 100 which comprises a matrix of N×M elements called pixels, N being the number of columns and M being the number of rows. Each pixel comprises a photosensitive region for accumulating incoming light energy in the form of electric charge and some transistors employed for controlling the pixel operation and for downloading the information from the pixel to the memory element typically placed at the bottom of a column. The photosensitive region may be for instance a photodiode.
The CMOS image sensor of FIG. 1 further comprises a row selection circuit 101 for driving the control signals to the pixels in the sensor area using horizontal control lines 102. The control signals may be applied to all pixels at the same time or to all pixels in a single row of the area. Accordingly, the signals may be global control signals or local control signals respectively. The local control signals are applied to one pixel row and then the same control signals are applied to another (for instance subsequent) pixel row, etc. Moreover, solutions are possible in which multiple rows are selected by the local control signals. For instance, high speed modern CMOS image sensors usually work using a “global shutter” operation. In this operation mode, the image sensing is performed in all pixels at the same time. Accordingly, the row selection circuit 101 applies the sequence of control signals activating image sensing to all pixel rows at the same time. In the past and in case of less speed demanding applications, the download of the pixels from the area is performed sequentially in a row by row basis or sequentially block by block, wherein a block includes multiple rows.
Furthermore, the CMOS image sensor includes typically a read-out channel (read-out circuit) 105. The read-out channel sequentially (row by row) receives the information contained in the pixels of each sensor row via the data column lines 103 and processes the data read. Modern CMOS image sensors typically include within the processing performed by the read-out channel also the operations of amplification and digitalization of the pixel information. After the processing, pixel information is transmitted from the sensor via output ports.
In compliance with the above description, the download of the pixel area is performed row by row driven by the row selection circuit. In a more sophisticated sensor, the row selection circuit may by capable to start in a row different from the first one and/or may skip some rows. This enables defining and downloading multiple regions of interest (ROI) separately and separated from several rows without wasting time due to downloading undesired rows. Nevertheless, since the control signals are propagated horizontally (row-wise) it is only possible to download a horizontal region of interest consisting of one or multiple full rows (complete rows).
FIG. 2 illustrates an example of an image sensor including two regions of interest and an effective read out process capable of skipping empty rows. In particular, FIG. 2 shows a pixel area 200 with a first region of interest 201 and the second region of interest 202. As can be seen, the first and the second regions of interest 201, 202 are both formed by multiple of entire rows. This is appropriate for the second region of interest in which all pixels 212 carry information which is needed to be downloaded. However, in the case of the first region of interest 201, only pixels 211 include the information which is needed to be downloaded. Accordingly, the remaining pixels of the first region of interest 201 are downloaded even if they are not needed since they don't carry any relevant information. This is illustrated in the bottom part of FIG. 2 showing the row by row download process of the pixel area 200 based on the two regions of interest 201 and 202. At first, the three rows corresponding to region of interest 202 are downloaded. Then, the 13 rows corresponding to the first region of interest 201 are now molded. Accordingly, assuming that the time Trow is the time necessary for downloading one row, the total download time for the two regions of interest 201 and 202 will be 3×Trow+13×Trow=16×Trow.
Indeed, in the present example since the pixel area 200 includes 46 rows, downloading only 16 of them speeds up the application considerably. However, there are applications in which the read out speed is essential and in which the regions of interest do not have rectangular shapes. In such cases, even with this approach, many unnecessary pixels are downloaded.
Moreover, there are many applications in which the ROI changes from image to image and which, at the same time, require a very high speed, which is habitually limited by the time required to read the information out of the sensor.
One typical example for this type of application is laser triangulation. Laser triangulation application is illustrated in FIG. 3. Laser triangulation includes monitoring a reflected structured laser light projected over an object which is located or moving in front of the camera.
Usually, the structured laser light is shaped as a single line. The laser triangulation is used to determine a distance to an object and/or structure of an object. For instance, the laser triangulation may be used for three dimensional scanning of the object. FIG. 3 shows a schematic principle of laser triangulation 310 in which an object 311 is illuminated by a laser source 312 of a scanning device 315. The laser produces a beam 313 which is reflected from the object 311 and the reflected beam 314 is detected by a sensor 316.
Laser scanning of a three dimensional object 320 is also illustrated on the bottom part of FIG. 3. The possible results of the scanning are shown in images 330 and 340 respectively. As can be seen from the exemplary reflection images 330 and 340, the images are almost empty except for a thin white line corresponding to the reflected laser line of which the shape is distributed along the pixel area. For instance, in the images 330 and 340 the minimum rectangular region of interest necessary to download the laser line has a vertical size almost as tall as the entire pixel area whereas the percentage of pixels with the relevant information (the line) is very small.
In the example above, the image 330 is almost empty (dark) except for the thin white (bright) line corresponding to the reflected laser line whose shape is distributed along the pixel array. In an image like this, the minimum rectangular ROI necessary to download the laser line is almost as tall as the entire pixel array whereas the percentage of the pixels with relevant information (pixels of the reflected line) is very small.
USPTO Backgrounds
USPTO Backgrounds