The invention pertains to machine vision and, more particularly, three-dimensional (3D) machine vision. The invention has application in manufacturing, quality control, and robotics, to name but a few fields.
Machine vision refers to the automated analysis of images to determine characteristics of objects represented in them. It is often employed in automated manufacturing lines, where images of components are analyzed to facilitate part-picking, as well as to determine part placement and alignment for assembly. When robots are the means for automated assembly and automated analysis of images is used to facilitate part picking, placement, and alignment, the system is referred to as vision-guided robotics. Machine vision is also used for robot navigation, e.g., to insure the recognition of scenes as robots travel through environments.
Though three-dimensional (3D) analysis has long been discussed in the literature, most present-day machine vision systems rely on two-dimensional (2D) image analysis. This typically necessitates that objects under inspection be “presented to” the vision system in constrained orientations and locations. A conveyor belt is commonly used for this purpose. Objects being assembled or inspected are typically placed at a particular known, stable, 3D configuration on the belt, but at an unknown position and orientation and moved to within the vision system's field of view. Based on an object's 2D pose (i.e., position and orientation) in the field of view, and taking into account that it is disposed on the conveyor (thereby, rendering certain its “lie” and its distance from the vision system camera), the system applies 2D geometry to determine the object's exact 3D pose and/or conformance with expected appearance.
Examples using such 2D vision analysis are provided in prior works of the assignee hereof, including U.S. Pat. No. 6,748,104, entitled “Methods and apparatus for machine vision inspection using single and multiple templates or patterns”, U.S. Pat. No. 6,639,624, entitled “Machine vision methods for inspection of leaded components”, U.S. Pat. No. 6,301,396, entitled “Non-feedback-based machine vision methods for determining a calibration relationship between a camera and a moveable object”, U.S. Pat. No. 6,137,893, entitled “Machine vision calibration targets and methods of determining their location and orientation in an image”, U.S. Pat. No. 5,978,521, entitled “Machine vision methods using feedback to determine calibration locations of multiple cameras that image a common object”, U.S. Pat. No. 5,978,080, entitled “Machine vision methods using feedback to determine an orientation, pixel width and pixel height of a field of view”, U.S. Pat. No. 5,960,125, entitled “Nonfeedback-based machine vision method for determining a calibration relationship between a camera and a moveable object,” U.S. Pat. No. 6,856,698, entitled “Fast high-accuracy multi-dimensional pattern localization”, U.S. Pat. No. 6,850,646, entitled “Fast high-accuracy multi-dimensional pattern inspection”, and U.S. Pat. No. 6,658,145, entitled “Fast high-accuracy multi-dimensional pattern inspection,” to name a few.
With the increased reliance on robotics, everywhere from the factory floor to the home, the need for practical 3D vision systems has come to the fore. This is because, in many of these environments, objects subject to inspection are not necessarily constrained in overall position and lie, e.g., as might otherwise be the case with objects presented on a conveyor belt. That is, the precise 3D configuration of the object may be unknown.
To accommodate the additional degrees of freedom of pose and position in a 3D scene, 3D vision tools are helpful, if not necessary. Examples of these include U.S. Pat. No. 6,771,808, entitled, “System and method for registering patterns transformed in six degrees of freedom using machine vision”, and U.S. Pat. No. 6,728,582, entitled, “System and method for determining the position of an object in three dimensions using a machine vision system with two cameras.”
Other machine vision techniques have been suggested in the art. Some require too much processor power to be practical for real-time application. Others require that objects subject to inspection go though complex registration procedures and/or that, during runtime, many of the objects' features be simultaneously visible in the vision system field-of-view.
Outside the machine vision realm, the art also provides contact-based methods of determining 3D poses—such as using an x,y,z measuring machine with a touch sensor. However, this requires contact, is relatively slow and can require manual intervention. Electromagnetic wave-based methods for determining 3D poses have also been offered. These do not require physical contact, but suffer their own drawbacks, such as requiring the oft impractical step of affixing transmitters to objects that are subject to inspection.
An object of this invention is to provide improved methods and apparatus for machine vision and, more particularly, for three-dimensional machine vision.
A related object of this invention is to provide such methods and apparatus as have a range of practical applications including, but not limited to, manufacturing, quality control, and robotics.
A further related object of the invention is to provide such methods and apparatus as permit determination of, for example, position and pose in three-dimensional space.
A still further related object of the invention is to provide such methods and apparatus as impose reduced constraints, e.g., as to overall position and lie, of objects under inspection.
Yet still a further related object of the invention is to provide such methods and apparatus as minimize requirements for registration of objects subject to inspection.
Still yet a further object of the invention is to provide such methods and apparatus as can be implemented in present day and future machine vision platforms.
USPTO Backgrounds
USPTO Backgrounds