2.1 Image Processing Techniques for Traffic Analysis
FIG. 1 shows the overview of the operation of a video-based traffic monitoring system. A camera mounted on a structure, such as the streetlight pole, looking over the traffic scene serves as the sensor device for the capturing of traffic images. The captured analogue video images are then transmitted to a processor which converts the analogue video into digital form. The digitized images will then be processed and analyzed for the extraction of traffic information using image processing techniques. The extracted information can then be transmitted to an external user, such as a traffic control center, for traffic monitoring/control.
Generally, application of image processing techniques for video-based traffic monitoring system can be divided into four stages:    1. Image acquisition    2. Digitization    3. Vehicle detection    4. Traffic parameter extraction
Stages 1 and 2 are basically the same for most of the existing video based traffic monitoring systems. The fundamental differences between individual systems are in states 3 and 4.
During the vehicle detection process, the input video image is processed whereby the presence of vehicle in the Region of Interest (ROI) is determined. The ROI can be a single pixel, a line of pixels or a cluster of pixels. During the traffic parameter extraction stage, traffic parameters are obtained by comparing the vehicle detection status of the ROI at difference frames (time interval).
2.2 Vehicle Detection
The fundamental requirement of a video-based traffic monitoring system is the capability to detect the presence of vehicle in the ROI. Most video-based traffic monitoring systems employed the background-differencing approach for vehicle detection. This is a process that detects vehicles by subtracting an input image from a background image created in advance. The background image is that one, where only the road section depicted but no vehicle appears, and is served as a reference.
2.2.1 Problem
2.2.1.1 Dynamic update of background scene
The basic requirement for using this method is the need of a background reference image to be generated. The background image must also be constantly updated so as to reflect the dynamic changes in ambient lighting condition of the road section, such as during the transition from day to night and vice-versa. Such variation of light intensity could cause the system to “false trigger” the presence of vehicle. However, the main problem when using the background-differencing approach is the difficulty in obtaining an updated background image if the road section is packed with heavy traffic or the lighting condition changes rapidly. The changes in lighting condition could be due to passing cloud or shadow of the nearby building structure cause by the changes in altitude of the sun.
2.2.1.2 Moving Shadow
Another problem of using the background differencing approach is that during a bright sunny day, vehicle can cast a “moving” shadow onto the next lane, as shown in FIG. 2. This shadow may cause false detection on the affected lane.
2.2.1.3 Night Detection (Headlight Reflection)
One other factor contributing to false detection, when using the background differencing approach, is the headlight of the vehicles at night, as shown in FIG. 3.
2.2.1.4 Detection at Chevron
Detection of a vehicle is generally performed on a roadway where the vehicle is travelling. However, there are circumstances where detection of vehicles at locations, other than the roadway, is required. For example, detection of a stopped vehicle at a shoulder or chevron (region consists of white stripes which occurs mainly at the joining point between entrance/exits and the expressway as shown in FIG. 4). Detection of a vehicle at a shoulder can usually be performed using a similar technique as the detection of a vehicle on the roadway. The detection of a vehicle on the chevron, however, becomes problematic when using the conventional background differencing approach.
The difficulty in detection of a vehicle on the chevron area, as compared to a normal roadway region, is that the background is not homogeneous. When using the conventional background differencing technique, the input image is compared with a background image pixel-by-pixel within the ROI. The comparison output will be high if a vehicle is present. However, when the ROI is within the chevron area, which consists of black and white stripes, a slight movement of the camera will result in a high output even when no vehicle is actually present. When using the edge density information for the detection of vehicle within the chevron region, the detection becomes insensitive. This is because the background edge density of the ROI is relatively high due to the black/white stripes, hence, it becomes difficult to distinguish the vehicle from the background based on the edge density.
2.2.2 Known Solution to Problem
2.2.2.1 Dynamic Update of Background Scene
One solution to update the background image is by looking at different frames in the image sequence. In any one frame, parts of the road are covered by cars. As time goes on, the cars will move and reveal the covered road. If the sequence is long enough, a clear picture of the car-free road can be found. The background image is generated pixel by pixel. The intensity of each point is observed in several initialization frames. The intensity value that occurred most often can be chosen to be the background value at that point. Another approach is by using the interpolation (over several frames) method, in a way it is by taking the average value of the pixel at different frames.
The shortcoming of using these two approaches, however, is that the process of selecting the most often occurred intensity value for each pixel (or the average value) over a sequence of frame can be intensive in computation if the sequence is long. If the sequence is short, it may be difficult to get enough background pixel intensity values in a congested traffic condition. Such dynamic update of the background scene is also not effective if the change of light intensity is too abrupt such as the shadow cast by a moving cloud.
2.2.2.2 Night Detection
When using the background differencing approach for the detection of vehicle in the night, false detection could arise due to problems such as headlight reflection. To overcome such problem, a technique that has been adopted is using the headlight as the indication of the presence of vehicle. The direct approach of using this method is that the vehicle's headlight is detected if a group of pixels' intensity values are greater than its surrounding pixels by a threshold value. The problem of using such technique is that it is difficult to establish the threshold value separating the headlight intensity from the surrounding pixels. Since the absolute intensity values of the headlight and the surrounding pixels can vary dynamically depending on the overall intensity of the road section. It is also computationally intensive to perform such two dimensional search in real time.
2.2.2.3 Day-Night-Transition
Since the night detection employs a different process for the detection of vehicle from that of the day detection. Inevitably, there is the requirement of automated switching from one detection process to another during the transition between day and night. The solution lies in the automatic detection of the day/night status of the traffic scene. However, this can be difficult since the transition between day and night, or vice versa, is gradual. Analyzing the overall average intensity value of the image, to distinguish between day and night, does not provide a reliable solution. This is because in a heavy traffic condition, the headlight of vehicles could significantly increase the overall intensity of the image. One way of avoiding the vehicle headlight is to select a detection region lies “outside” the traffic lane. However, since the traffic scene is an uncontrolled outdoor environment, there is no assurance that the condition of the detection region remains unchanged over a long period of time.
2.3 Traffic Parameters Extraction
During the parameter extraction stage, traffic parameters are extracted by comparing the vehicle detection status of the ROI at difference image frames of different time interval. Traffic parameters, generally, can be divided into two types, traffic data and incident. Depending on the method of parameter extraction employed, generally, the basic traffic data includes vehicle count, speed, vehicle length, average occupancy and others. Using the basic traffic data, other data such as gap-way and density can be easily derived. Traffic incident consists of congestion, stopped vehicle (on traffic lane or shoulder), wrong-direction traffic and others.
2.3.1 Known Solution and Problem
Existing method for the extraction of traffic parameters, generally, includes the window technique (or trip-line) and the tracking technique as shown in FIGS. 5 and 6, respectively.
2.3.1.1 Window Technique and Problem
Using the window technique, the ROI is usually defined as isolated sets of window (rectangular box) as illustrated in FIG. 5. The basic function of each window is for the detection of vehicle and hence counting the number of vehicles. In order to measure the vehicle speed two windows are required. By obtaining the time taken for the vehicle to travel from one window to the other, knowing the physical distance between the two, enable the system to determine the vehicle speed. Then, by obtaining the length of time the detected vehicle present on one window and the vehicle speed will yield the vehicle length. The advantage of the window technique is that it is computationally simple.
Error Due to Frame Rate Resolution
The disadvantages of the window technique is that its accuracy, for length and speed measurement, is affected by the resolution of the processing frame rate and the actual speed of the vehicle. In FIG. 7, vehicle A first activated window x at frame f. At frame f+n, of FIG. 8, vehicle A activates window y. To calculate the vehicle speed, it is assumed that the vehicle had travelled a distance of dw in the time period of n frames. dw is the physical distance between the two windows. However, due to the limited frame rate resolution, the actual distance which vehicle A had travelled is dv (compare FIGS. 7 and 8). Therefore the error rate can be as much as (dv-dw)/dw. The boundary of this error increases as the frame rate decreases.
Error Due to Occlusion
When using two windows for speed measurement, the distance between the two windows must be maximized in order to reduce the error due to frame rate resolution. However, increasing the distance between the two windows will increase the possibility of occlusion at the window to the upper part of the image. The occlusion can be illustrated as shown in FIGS. 9 and 10, which show two successive frames of video images. These two figures also show the typical angle of camera view for traffic data extraction. Due to the error of perspective, vehicle B appeared to be “joined” to vehicle A at frame f of FIG. 9, hence, window x will not be able to detect the time when vehicle B presents (at window x). At frame f+n (FIG. 10), however, window y can successfully detect vehicle B since the error of perspective is minimal at the lower extreme of the image. When window y is used as the counting “sensor”, its counting error due to occlusion will be minimized. However, the accuracy of the vehicle speed measurement (and hence vehicle length) will be affected by the occlusion problem at window x. The occlusion will be even more apparent in the event of congestion.
2.3.1.2 Tracking Technique and Problem
When using the tracking technique, a search is first performed along a “tracking zone” of ROI as shown in FIG. 6. When a vehicle is detected, its location is determined. This vehicle will then be tracked, along the tracking zone, in subsequent frames. By tracking the vehicle in each frame, with its location, the vehicle speed is measured. The vehicle length can be measured directly by detecting the front and end of the vehicle.
The advantage of using the tracking method is that it is theoretically more accurate than the window technique in terms of speed measurement. Since the exact location of the tracked vehicle is determined at each frame, accuracy of its speed measurement is, therefore, not affected by the frame rate resolution. The disadvantage of the tracking method, as compare to the window technique, is that it is more intensive in computation. However, with the advance of computer processing power, this shortcoming is becoming less significant.
Error Due to Occlusion
For direct length measurement using the tracking technique, that is by detecting the vehicle's front and end, the vehicle must be isolated from both preceding and succeeding vehicles for at least one frame. However, due to the angle of perspective, it may be difficult to isolate the vehicle from succeeding vehicle such as that shown in FIG. 11 (vehicles A and B). In FIG. 12, thought vehicle A can be isolated from B, its front however is out of the camera field of view, hence, unable to determine its length.
USPTO Backgrounds
USPTO Backgrounds