The present invention relates generally to automatic eye tracking wherein visual feedback information is presented to the user during operation of the eye tracker. More particularly the invention relates to a system according to the preamble of claim 1 and a method according to the preamble of claim 13. The invention also relates to a computer program according to claim 25 and a computer readable medium according to claim 26.
The concept of eye tracking is well known in the art, and a number of different techniques have been developed for accomplishing automatic eye and gaze tracking. In the area of remote, non-obtrusive eye tracking, the most commonly used designs are based on so-called pupil center corneal reflection (PCCR) methods, which render it possible to determine the eyes' positions as well as their gaze direction. Thus, it is possible to estimate a point of regard, for instance on a computer display, such that based on such estimations a user may enter commands to the computer.
However, the known eye-controlled solutions often fail to accomplish a fully satisfying user-interaction with the computer. One common problem is that the eye-controllable input interface imposes comparatively heavy motor tasks on the user's eyes, which are truly perceptive sensory organs. This may lead to fatigue symptoms and discomfort experienced by the user. Another problem is that the user, who perhaps is a handicapped person, may have problems to control his/her gaze with sufficiently high precision to control the computer as desired. Of course, this problem is particularly accentuated if the eye-controllable screen objects are small. Additionally, even if the user is capable of controlling his/her gaze with very high precision, imperfections in the eye-tracking equipment may introduce measurement errors in respect of an estimated point of regard on the display. Aiming to improve the man-machine interface the U.S. Pat. No. 6,152,563 proposes that so-called collapsing symbols be presented over a selected window object in order to visually confirm the selection of the object during an interval prior to any manipulation. Thus, the user can abort an erroneous selection before the manipulation in respect of the object is effected. The article also proposes the use of zoom windows to enhance the precision of the user's gaze fixations on certain on-screen objects.
However, it is not only difficult to adequately estimate the user's point of regard. Another dilemma in eye tracking is that it may be problematic for the user to accomplish a selection of an on-screen object, or by other means initiate a computer-controlled action. Expressing the user's intent to the computer via an eye-controlled interface may be difficult for many reasons. One major issue is the fact that the eyes are designed for perception, not for motor tasks, and therefore the gaze moves over the display also when the user registers information thereon, however not is interested in generating any control commands. A dwell time, i.e. the time during which the user fixates his/her gaze on an object on the screen, may be used to express the user's intention to manipulate a certain object. A disadvantage by this approach is that the interaction process becomes relatively slow, and risk annoying the user. Confirmation commands in the form of blinks may speed up the process substantially. However, human blinks are often unintentional, and hence the user might produce many undesired control commands. Of course, a physical button, or a key, provides a much more reliable confirmation means. Nevertheless this option may not be open to the user, who perhaps is incapable of manipulating such buttons/keys. Confirmation via a voice recognition interface constitutes yet another alternative, which is less reliable, and perhaps still not a viable option for the user due to physical disablement, or factors in the environment in which the system is used, such as noise.
The human gaze pattern includes so-called saccades, which represent rapid movements of the gaze from one point to another. These movements are almost never involuntary, excluding nystagmus patients. However, using saccades to enter data or commands via an eye-controllable input interface is problematic for other reasons. Namely, if a saccade activates a function in respect of a particular on-screen object, typically the saccade itself results in that the user no longer observes this object, at least not directly. Consequently, further eye-controlled manipulation of the object becomes very difficult, if at all possible. Again, this is a problem related to the combined motor and perception tasks placed on the eyes by the eye-controllable input interface.
The cursor symbol is generally an important object to control when interacting with a computer. However, this is not straight-forward either. Namely, the most high-resolution part of the retina, the macula (or the yellow spot), measures a diameter equivalent to a field of view of one degree. Therefore, a human being never needs to direct his/her gaze with a higher precision than what is given by this measure, i.e. within one degree. Hence, regardless of how high precision the eye-tracker has, a perfect match between the user's gaze and an on-screen cursor symbol still cannot be accomplished. Instead, any cursor symbol being directly controlled in response to an estimated point of regard is bound to introduce artifacts, which will be perceived as errors by a human user. Nevertheless, U.S. Pat. No. 6,637,883 describes an eye-tracking system for displaying a video screen pointer at a point of regard of a user's gaze. For enhanced precision, the camera which registers the eyes' movements is here positioned in an eyeglass frame worn by the user. Naturally, this imposes separate problems, i.a. related to fitting and comfort.
Instead of controlling the cursor symbol in direct response to the estimated point of regard, relative cursor controls are normally preferable, i.e. solutions wherein the cursor is caused to move indirectly by manipulation of a particular control means on the screen for each direction up/down and left/right. However, when placing the point of regard on such a control means, the user cannot see the cursor symbol to be controlled any longer.
U.S. Pat. No. 6,204,828 discloses an integrated gaze/manual cursor positioning system, which aids an operator to position a cursor by integrating an eye-gaze signal and a manual input. When a mechanical activation of an operator device is detected the cursor is placed at an initial position which is predetermined with respect to the operator's current gaze area. Thus, a user-friendly cursor function is accomplished, however no cursor-based commands are input to the computer based on the eye-gaze signal.
The published U.S. patent application No. 2005/0047629 describes a solution wherein, upon activation of a mechanical input device, a region of the screen to which a user's gaze is directed is expanded, so that selection of objects within this region is facilitated. Components located immediately outside of a radius from the gaze point may be contracted and/or translated in connection with this expansion. The mechanical input device, such as a conventional mouse, is then used to control a graphical pointer to said region to effect the selection command. Nonetheless, also this strategy is problematic because it requires a mechanical input device, and it only provides one possible position for the eye controllable interface and thus offers no options for having more than one type of eye control signal. Hence, the degree of flexibility is relatively low.
USPTO Backgrounds
USPTO Backgrounds