The invention relates to a pixel circuit for an image sensor. More specifically, it relates to a pixel circuit and an operating method thereof, wherein an exposure measurement circuit is configured for measuring the light exposure intensity from a photoreceptor signal derived from a light exposure of a single photoreceptor, upon detection by a transient detector circuit of a change in said photoreceptor signal.
Conventional image sensors acquire the visual information time-quantized at a predetermined frame rate. Each frame carries the information from all pixels, regardless of whether or not this information has changed since the last frame has been acquired. This approach obviously results, depending on the dynamic contents of the scene, in a more or less high degree of redundancy in the recorded image data. The problem worsens as modern image sensors advance to ever higher spatial and temporal resolution. The hardware required for post-processing of the data increases in complexity and cost, demand on transmission bandwidth and data storage capacity surges and the power consumption rises, leading to severe limitations in all kinds of vision applications, from demanding high-speed industrial vision systems to mobile, battery-powered consumer devices.
One approach to dealing with temporal redundancy in video data is frame difference encoding. This simplest form of video compression includes transmitting only pixel values that exceed a defined intensity change threshold from frame to frame after an initial key-frame. Known frame differencing imagers rely on acquisition and processing of full frames of image data and are not able to self-consistently suppress temporal redundancy and provide real-time compressed video output. Furthermore, even when the processing and difference quantization is done at the pixel-level, the temporal resolution of the acquisition of the scene dynamics, as in all frame-based imaging devices, is still limited to the achievable frame rate and is time-quantized to this frame rate.
The adverse effects of data redundancy are most effectively avoided by not recording the redundant data in the first place and directly reducing data volume at the sensor output level. The immediate benefits are reductions in bandwidth, memory and computing power requirements for data transmission and post-processing, hence decreasing system power, complexity and cost. In addition, the frame-based, clocked principle of operation of conventional CMOS or CCD image sensors leads to limitations in temporal resolution as scene dynamics are quantized to the frame rate at which the pixel field of view is read out, and poor dynamic range.
The problem to be solved by the present invention is to provide a method and an apparatus for the continuous acquisition of the full visual information of an observed dynamic scene with high temporal and intensity resolution, over a wide dynamic range (of recordable and processable light intensity) and thereby generating the minimum necessary amount of data volume. Thus, the generated data are not constituted by a succession of frames containing the image information of all pixels, but an (asynchronous) stream of change and intensity (i.e. grey level) information of individual pixels, which are recorded and transmitted only if an actual change in light intensity in the field of view of the individual pixel has been detected by the pixel itself.
This method leads to a substantial reduction of generated data through complete suppression of the temporal redundancy in the picture information that is typical for conventional image sensors, though with the data containing the same, or even higher, information content. The picture element for an image sensor that implements the aforementioned method as well as the required asynchronous data readout mechanism can be realized on basis of analogue electronic circuits. An image sensor with a multiplicity of such picture elements is typically realized and fabricated as an integrated system-on-chip e.g. in CMOS technology.
Implementing such a sensor and thus avoiding the above mentioned drawbacks of conventional image data acquisition would be beneficial for a wide range of artificial vision applications including industrial high-speed vision (e.g. fast object recognition, motion detection and analysis, object tracking, etc.), automotive (e.g. real-time 3D stereo vision for collision warning and avoidance, intelligent rear-view mirrors, etc.), surveillance and security (scene surveillance) or robotics (autonomous navigation, SLAM) as well as biomedical and scientific imaging applications. As the sensor operation is inspired by working principles of the human retina, one advantageous example application is the treatment of a degenerated retina of a blind patient with an implantable prosthesis device based on the data delivered by such a sensor.
A solution for achieving the aforementioned complete temporal redundancy suppression is based on pixel-individual pre-processing and acquiring of the image information, event-controlled (i.e. independently of external timing control such as clock, shutter or reset signals) and conditionally (i.e. only when changes in the scene have been detected). As explained below, the control of the image data acquisition is transferred to the pixel-level and can be done at very high temporal resolution (e.g. fully asynchronously).
In the case of the optical transient sensor, or dynamic vision sensor (DVS), changes in lighting intensity received by the individual, autonomously operating pixels are detected by an electronic circuit, “a transient detector”, described in U.S. Pat. No. 7,728,269.
U.S. patent application US 2010/0182468 A1 discloses combining transient detector circuits, i.e. light exposure intensity changes detector circuits, and conditional exposure measurement circuits. A transient detector circuit individually and asynchronously initiates the measurement of a new exposure measure only if—and immediately after—a brightness change of a certain magnitude has been detected in the field-of-view of a pixel. Such a pixel does not rely on external timing signals and independently requests access to an (asynchronous and arbitrated) output channel only when it has a new grayscale value to communicate. Consequently, a pixel that is not stimulated visually does not produce output. In addition, the asynchronous operation avoids the time quantization of frame-based acquisition and scanning readout.
For each pixel, the transient detector circuit monitors a photoreceptor voltage derived from a first photodiode for detecting relative voltage changes that exceed a threshold. Upon such detection, the transient detector circuit outputs a command for the exposure measurement circuit of the same pixel to start an absolute intensity measurement, i.e. an absolute grey level measurement. The exposure measurement circuit uses a second photodiode of the pixel, placed adjacent to the first photodiode, and derives its measure from the time duration for discharging the photodiode junction capacitance with the instantaneous photocurrent.
However, the pixel circuit disclosed in US 2010/0182468 A1 is not optimal since it consumes a large area for a pixel element and thus cannot achieve high resolution. Furthermore, time-based exposure measurement through direct photocurrent integration often leads to a prohibitively long measurement time of a new exposure value, especially at low pixel illuminance levels, due to the corresponding small photocurrents. Finally using two separate photodiodes for change detection and exposure measurement leads to spatial divergence and motion direction dependency of the image data acquisition process, resulting in a reduction in imaging quality.
USPTO Backgrounds
USPTO Backgrounds