As the popularity of intelligent surveillance system gradually rises, some technologies focus the development on using the image analysis technique in the back-end system to capture meaningful event information. A stationary camera has limited coverage range and has blind spots. When an event occurs in a larger region, the fixed camera cannot obtain the surveillance screens of the entire event easily. Some technologies explores the possibility to use flying carrier, such as, hot air balloon, unmanned airplane, to fly with the camera to take the bird's eye view of the ground event and analyze the detected image, as an aid to the ground surveillance system for large region detection to eliminate the blind spots.
Among the tracking technologies of ground objects via computer vision on flying carrier, the moving object detection technology, such as, uses affine warping technology to perform mutual registration of the consecutive images of a moving object, and then computes the normal flow of the two consecutive stabilized images to detect the moving object. Then, a four-connectivity connected component labeling technology is used to label the objects. For labeled objects in each image, the attributes, such as, center of mass, axis orientation, length, and so on, are used to compute the affinity of the objects in the neighboring images and association is assigned to enable the moving object tracking.
There are three major strategies for moving object tracking. The first is to use KLT tracker to associate the objects in neighboring images. The second is to compute the appearance and movement feature of the object, and uses a threshold to determine the association of the moving objects in neighboring images to uses the features of the majority of the moving objects to compute the optimal match probability. The third is to use filer, such as particle filter, for moving object tracking.
Vision-based tracking of region of interest (ROI) can be based on image template matching or based on feature tracking. The former tracking technology is based on the image feature of the ROI, and searches for the maximum affinity response region in the next image, for example, the mean shift scheme uses the gradient information of the feature space computed via the mean shift scheme to rapidly find the tracking target region. The latter is to detect feature points in the ROI, and uses KLT tracker to track the correspondence between the features of two consecutive images. The correspondence relationship is the basis for tracking the ROI. For example, random sample consensus (RANSAC) is based on the law of large numbers, and selects a plurality of feature points randomly to estimate the homography transform of ROI between the two consecutive images, and uses recursion to find the homography transform that matches the majority of all the feature points best. When the number of the correct or suitable inliers is too few, the RANSAC method requires a plurality of recursion. That is, a large amount of computing resource must be consumed to obtain reliable tracking result.
The vision-based tracking of ROI patents, such as, U.S. Pat. No. 6,757,434 disclosed a tracking technology for ROI of video images, applicable to image compression. As shown in FIG. 1, the technology, aiming at ROI 110 of the (k−1)-th image, uses boundary projection to predict the boundary 120 of ROI in the k-th image, and reversely finds matching point 130 in (k−1)-th image. U.S. Publication No. US2010/0045800 disclosed a technology to divide the ROI into inner circle and outer circle and computes the color histogram of the inner and outer circles as features separately to act as a basis for tracking.
Image-based tracking of ROI papers, for example, “Region-of-interest Tracking based on Keypoint Trajectories on a Group of Pictures”, International Workshop on Content-based Multimedia Indexing, 2007, disclosed a technology to use M-estimator to estimate the affine transform of the ROI in two consecutive images, and use an optimization algorithm to solve the M-estimator problem. This technology uses statistics significance to remove outliers. The optimization process will consume a large amount of computing resources.
The current flying carrier object tracking technology usually needs a large amount of computing resources. Basically, a PC-level or higher computing device is needed for real-time computing. However, the flying carrier has limited load weight capacity; therefore, a light embedded system is more appropriate. Hence, the object tracking algorithm needs fast and efficient computation.
USPTO Backgrounds
USPTO Backgrounds