An Apple patent application published Thursday details a method of unlocking a mobile device using facial image and depth data gleaned from specialized sensors and cameras, lending credence to rumors that the company is developing facial recognition systems to augment, or replace, Touch ID fingerprint scanning.
Source: USPTO
As published by the U.S. Patent and Trademark Office on Thursday, Apple's patent application for "Embedded authentication systems in an electronic device," filed for in 2013, details advanced facial recognition techniques for authenticating owners using portable devices.
In particular, the claims -- and not the following summary or detailed description -- explain methods of capturing 2D image and corresponding depth data of a user's face by combining information gathered by a visible light image sensor, an infrared sensor and a three-dimensional capture system. Apple's invention also allows for the inclusion of an ultra-violet sensor, a scanning laser, an ultrasound sensor and similar equipment.
In practice, authentication software contains instructions for detecting a user's face opposite one or more of the included imaging sensors. In response to detecting a face, the visible light, infrared and 3D capture devices gather image data for local processing.
Apple notes the 3D capture device might collect data relating to the contour of the user's eye sockets, chin, nose or a combination of those facial features. Texture analysis may also be employed to detect unique lines, patterns or spots on a user's skin. Further, infrared light emitted by the system might be reflected by a user's eyes and captured by one or more sensors.
Facial attributes detected in captured image data are compared against a library of authorized users for authentication. Importantly, the patent application notes the techniques described can be used to unlock a portable device, like an iPhone.
Perhaps not coincidentally, a nearly identical list of hardware is predicted for inclusion in Apple's upcoming "iPhone 8."
KGI analyst Ming-Chi Kuo in an investor note earlier this year said "iPhone 8," or "iPhone Pro," would integrate a "revolutionary" front-facing camera system capable of 3D sensing and modeling.
Like today's published invention, the rumored iPhone solution incorporates an actively aligned IR transmitter, specifically a VCSEL laser, and IR receiver alongside the usual RGB FaceTime camera. In this arrangement, the VCSEL laser and receiver subsystem acts as a depth sensor by calculating the distance to a target using light pulses and time of flight (TOF) measurements.
Earlier this month, a Bloomberg report claimed Apple is working to replace fingerprint scanning with facial recognition technology, though the solution might not debut this year. The supposed system is fast, it takes only milliseconds to recognize and authenticate a user, and is said to be more secure than current Touch ID technology found on iPhone, iPad and Mac.
While competitors in the smartphone space have rolled out their own face-based authentication systems, like Samsung's new Galaxy S8, Apple has been reluctant to adopt similar technology. Security concerns are thought to play a major role in the company's decision not to include facial recognition in iPhone.
For example, at the Galaxy S8 launch event in March, a YouTuber was able to defeat the handset's optical facial recognition feature using a digital photograph displayed -- ironically -- on another S8.
Samsung was quick to point out that the function is not meant for high-level security. Facial recognition on the Galaxy S8 is limited to opening a locked handset, while more secure biometric assets like the onboard iris scanner and fingerprint reader are used to access Samsung Pay and sensitive information stored in the phone's Secure Folder.
Thanks to its depth mapping capabilities, Apple's facial recognition solution is much more secure than those relying solely on optical data. A nefarious user would not be able to spoof an authenticated face using a 2D digital photo, for example. Still, it is unclear whether Apple intends to deploy the facial recognition feature as a Touch ID replacement or an augmentation to the fingerprint scanning tech.
Whatever the case, "iPhone 8" might arrive with 3D sensing disabled. A report this week claims engineers are facing issues with software designed to govern advanced handset functions like the specialized front-facing camera and wireless charging. If the problems are not rectified in time, Apple might release the handset without those features enabled, then activate them at a later date via a software update.
Today's patent application is the descendent of a string of filings dating back to 2007. Interestingly, compared to the most recently published document, a 2013 patent grant for U.S. Patent No. 9,519,771, Apple only updated the claims portion of the filing, leaving the summary and detailed description sections largely untouched. Notably, the Thursday publication contains only a few references to the facial recognition technology outlined in its claims, with a bulk of the text focusing on general screen unlocking methods, biometric authentication using fingerprint data, vein patterns, voice recognition and other related techniques.
Apple's 3D facial recognition patent application was first filed for in December 2013 and credits Tony Fadell, Andrew Hodge, Stephan Schell, Ruben Caballero, Jesse Lee Dorogusker, Stephen Zadesky and Emery Sanford as its inventors. Fadell, known as the "father of the iPod," exited Apple in 2008, while Zadesky, another high-ranking executive rumored to be in charge of Apple's self-driving car project, left the company last year.