Apple patents eye-tracking gaze controls for iOS, Mac devices

Source: USPTO

As granted by the U.S. Patent and Trademark Office, Apple's U.S. Patent No. 8,937,591 for "Systems and methods for counteracting a perceptual fading of a movable indicator" deals with Troxler's fading, a problem unique to eye-tracking and gaze-based GUI solutions.

According to Apple, Troxler's fading, otherwise known as the Troxler Effect, is a phenomenon that affects visual perception of unmoving stimuli or objects in one's peripheral vision. As explained, one result of the phenomenon is a perceived fading of a visual stimulus when its location on the retina becomes fixed, making it a stabilized retinal image.

It is thought that the brain uses the mechanism to deal with blind spots on the retina, but in persistent gaze tracking systems, Troxler's fading can be detrimental to user experience. For example, when a user fixes their vision on a particular point, such as a computer cursor, and that object moves along with their gaze, it can quickly disappear.

Not accounting for the Troxler Effect in particularly effective and accurate gaze-based control systems can result in the perceived fade of a movable indicator. It should be noted that in order to invoke a stabilized retinal image, such a system would need to be accurate enough to detect slight shifts in a user's gaze and reflect those changes onscreen almost instantaneously.

Assuming Apple creates an adequately precise gaze-tracking GUI, today's patent offers a blend of hardware and software solutions to counter the Troxler Effect and any resulting perceived fluctuation in visual acuity.

In one embodiment, the invention employs eye-tracking hardware to follow a user's gaze and relay that information to an onscreen GUI. A movable indicator is associated with a user's point of gaze, thereby facilitating input through detection of eye movement.

To thwart perceived cursor fade, Apple suggests a system that measures possible perceptual fading by detecting time elapsed between eye movements, blinking or a combination of both. By measuring the time between eye movements, as per any number of predefined thresholds, the system is able to counteract fade before it starts.

Another method uses infrared sensors to detect when a user is blinking and moves the cursor during that short time span. The system may compare a sequence of images taken of a user's face or detect a user's pupil — or lack thereof — to determine an eye blink.

Finally, the patent describes a method by which the user's distance from the display or viewing angle is measured and tracked.

All three methods end with a repositioning of the onscreen movable indicator if the system determines a user is in danger of experiencing perceived fade, which in most cases can be established through lack of eye, body or facial movement. To preemptively counter perceived fade, a countermeasure generator may vary cursor positioning, apply animations to the indicator or otherwise provide visual modifications to prevent against a distracting user experience.

Apple notes a number of use case scenarios, including standard Mac laptops and multitouch iOS devices, but a gaze-tracking system could be an ideal input option for the company's long-awaited Apple TV refresh. If an eye-tracking feature is included, perhaps with voice control functionality for making selections and calling up menus, Apple would be ushering in a new breed of "hands-off" device control.

In April of 2014, rumors claimed motion controls would play a "key role" in next-generation Apple TV hardware, but that specific implementation is thought to incorporate technology acquired through Apple's then-recent purchase of 3D scanning firm PrimeSense.

While most recent Apple patents have focused on physical touch-based controls, such as last week's Touch ID "joystick," today's patent proves the tech giant is still experimenting with alternative forms of input.

Apple's gaze-based GUI with perceived fade countermeasures was first filed for in April 2012 and credits David P. Julian as its inventor.