Several years ago the present inventors pioneered the concept of using digitally captured images to identify objects within the images, and then using such identifications to retrieve information from various databases. Examples include:                Using a local device (cell phone, digital camera, PDA or other device) to capture an image of an object in an art museum, identifying the object from the image data, and then providing the user with information regarding the object (i.e., about or relating to the object);        Using a local device (cell phone, digital camera, PDA or other device) to capture an image of an automobile as it drives along a road, identifying the make and model from the image data, and then providing a user with a link to a website relating to that particular make and model;        Using a local device (cell phone, digital camera, PDA or other device) to capture an image of a bar code, logo, or other indicia in a magazine, using information contained in the indicia to identify a product, and providing a telephone number or other contact information relating to that product;        Using a local device (cell phone, digital camera, PDA or other device) to photograph a billboard of a restaurant, identifying the restaurant from a barcode, special target, written language, or other information contained in the photograph, and using that information to access a database to provide the user with restaurant's location, menu, or telephone number; and        Using a local device (cell phone, digital camera, PDA or other device) to capture an image of a sign at a sports stadium, using information extracted from the image to automatically purchase an entry ticket for the user, and providing the user with an entry code that can be used to bypass the long lines of ordinary ticket purchasers.        
In such embodiments it was specifically contemplated that analysis of the images could be performed locally (i.e. on the cell phone, PDA or other device capturing the image), distally at a server, or more preferably using some combination of the two. It was also contemplated that any available database could be accessed to provide the returned information, including publicly accessible databases on the Internet. It was not appreciated, however, that one could integrate these concepts with the searching capabilities of standard Search Engines.
In the 1990s Yahoo!™ introduced the idea of indexing web pages accessible on Internet, and providing a Search Engine that to access the index. Since that time dozens of other searching systems have been developed, which use all manner of various search methods, algorithms, hardware and/or software. All such systems and methods that accept user inputs of Key Information, and then utilize such Key Information to provide the user with information of interest, are referred to herein as Search Engines. The user, of course, can be a natural person, as well as a device (computing or otherwise), algorithm, system, organization, or any other entity. In searching for information, a Search Engine can utilize any suitable search domain, including for example:                A database (including for example a relational database, an object database, or an XML database).        A network of resources including for example web pages accessible within the Internet; and        A public or private collection of documents or information (e.g., documents, information, and/or messages of a company or other organization(s)) such as that maintained by LEXIS™.        
In a typical search, Key Information is provided to the Search Engine in the form of key words comprising text, numbers, strings, or other machine-readable information types. The Search Engine then searches its indices of web pages for matches, and returns to the user a hyperlinked listing of Internet Uniform Resource Locators (“URLs”), as well as some brief display of context in which the key word(s) are used. The information of interest can sometimes be found in the hyperlinked listing, but is more frequently found by linking directly to the listed web pages.
Providing Key Information to Search Engines in the form of text strings has inherent difficulties. It involves strategy in the selection of the text to be entered, and even with respect to the format of the keywords (for example using wildcards). Another difficulty is that small computing and/or telephony devices (e.g. telephones, both mobile and non-mobile), have small and/or limited keyboards, thus making text entry difficult.
USPTO Backgrounds
USPTO Backgrounds