Apple awarded pair of 3D user interface patents related to computer vision

Apple was awarded two new patents this week detailing 3D input with advanced camera systems. They were discovered on Tuesday by AppleInsider.

The first patent, No. 9,417,706, covers a hand motion observed by a camera, forcing a user interface to transition from one state to another. The second, patent No. 9,417,763, addresses a mobile device's determination of its location in space, using all sensors to determine relative motion and identifying a frame of reference, such as a user's face.

No. 9,417,706 requires only a camera connected to a computing device. The patent references a situation where a specific motion must be done by a user to enable an action. While a desktop computer with attached webcam is shown in the patent documentation, the technology granted is not limited to desktop computing.

An example given in the patent document is of a user sitting, and raising an arm over a shoulder, and then back down. In the patent document's example, a "push" or a "wave" at the computer could initiate a monitoring action in the operating system, which could to tell a device to start watching for future gestures. Another example illustrated by the patent is a user controlling items on the computer screen in 3D.

A specific application mentioned in the first patent is an unlock gesture, similar to the iOS "swipe to unlock," but performed in the air rather than on a touch-sensitive surface.

The second patent, number 9,417,763 has augmented reality implications. Should the patent ever be implemented, it suggests a device that combines yaw, pitch, and roll information with user facial monitoring. A device with technology suggested by the patent can not only keep track of a device's location relative to the user, but also one that can monitor the user's eye position, which the filing says will allow for "more realistic virtual 3D depictions of the objects on the device's display."

A practical use of patent No. 9,417,763 is actuation of high-powered GPU routines, which would be disabled to preserve the battery when the user is either not looking at the device or cannot see the screen of the device.

Patent No. 9,417,706 was applied for in May 2014, and awarded in May 2015. Patent No. 9,417,763 was filed in December of 2014 and awarded in April 2015. Both were published on Tuesday.

Neither patent appears to be currently in use by any Apple product. Apple and suspected shell companies file for hundreds of patents per year, and many never see use.