Apple's Craig Federighi details Face ID in new interview [u]

In a brief interview on Friday, Apple SVP of Software Engineering Craig Federighi answered a few questions about the Face ID facial recognition feature that will ship out to customers in November, dispelling some of the more contentious rumors, providing background on its development and touting the system's benefits.

Federighi spoke with TechCrunch to clear up some questions and concerns about Apple's biometric authentication solution that have cropped up in the days following Face ID's debut and first public demonstration on Tuesday.

Starting with Face ID's data backbone, Federighi said Apple collected a "quite exhaustive" library of face scans from consenting subjects that was subsequently used to train the system. To maximize accuracy, Apple captured high quality depth maps of each person's face from various angles.

"Phil mentioned that we'd gathered a billion images and that we'd done data gathering around the globe to make sure that we had broad geographic and ethnic data sets. Both for testing and validation for great recognition rates," Federighi said. "That wasn't just something you could go pull of [sic] the internet."

He goes on to say that Apple is holding on to the high fidelity depth map data for further algorithm training, noting the dataset is needed for system optimization. As can be expected of a valuable, and potentially sensitive, asset, Apple's facial scan library is protected.

The same can be said about user data. Media outlets, pundits and potential buyers have raised concerns over what Apple does with data generated from a Face ID scan. Some have suggested data is sent to the cloud for processing, while others pondered whether Apple keeps a repository of scans to train its neural network.

"We do not gather customer data when you enroll in Face ID, it stays on your device, we do not send it to the cloud for training data," Federighi said.

Just like Touch ID, Apple does not retain Face ID data, that information lives in the iPhone X Secure Enclave. Also like Touch ID, raw Face ID image data is processed and stored as a mathematical model that cannot be reverse engineered back into "model of a face."

Developers are also not privy to facial scan data through ARKit. Instead, apps leveraging the API are granted access to a depth map, not raw sensor data, that can be applied to photographic effects and other software features, Federighi said.

Further, TrueDepth's dot projector and infrared receiver are designed to work only at short distances, a hardware limitation noted by industry analysts in the weeks leading up to iPhone X's unveiling. Extrapolating this information out into the real world, it can be assumed that Apple's phone is not capable of scanning faces of passersby.

The executive notes iPhone X boasts a redesigned Secure Enclave capable of performing the re-training procedures that power Face ID's adaptive features. Mentioned during Tuesday's keynote, Face ID is able to adapt to changes in a user's face. For example, the system will work when a user grows a beard, wears a hat or puts on glasses. As Federighi noted in an email to a curious customer, Face ID even works with most sunglasses.

There are limitations to the system's capabilities. Face ID will be unable to recognize a user whose face is obscured by a mask or niqab, the report says. In such cases, users can opt to enter a passcode instead. A user passcode is also required after five failed attempts or if a user has not unlocked their phone with Face ID in 48 hours. There are also idle time limits, meaning passcode unlock is required if an iPhone X has not been unlocked in 6.5 days and Face ID was not used to unlock the device within 4 hours.

Federighi earlier this week noted users can manually disable Face ID by gripping iPhone X's side buttons, a feature that could come in handy when a user is forced to hand over the device unwillingly. The executive confirmed to TechCrunch that the feature works by depressing either volume button and the sleep/wake button. The squeeze to passcode feature is also available on iPhone 8.

"So, if you were in a case where the thief was asking to hand over your phone — you can just reach into your pocket, squeeze it, and it will disable Face ID," he said. "It will do the same thing on iPhone 8 to disable Touch ID."

From Federighi's description, Face ID appears to be extremely flexible insomuch as users need not worry about holding iPhone X at exact distance on a specific angle.

"It's quite similar to the ranges you'd be at if you put your phone in front facing camera mode [to take a picture]," he said. "Once your space from eyes to mouth come into view that would be the matching range - it can work at fairly extreme angles — if it's down low because your phone is in your lap it can unlock it as long as it can see those features. Basically, If you're using your phone across a natural series of angles it can unlock it."

Users can also turn off attention detection, one of the safeguards built into Face ID to protect against errant unlocking. When activated, attention detection requires users to look at iPhone X in order to trigger the authentication procedure. This might not be possible for people with certain conditions, for example blind users.

Alternatively, Face ID can be disabled altogether, leaving users to unlock their device with a passcode.

As for usability, Federighi says Face ID was developed using face data collected from a broad spectrum of geographic locations, ages and ethnic backgrounds. In other words, the system should function normally, no matter your cultural background.

A day after Face ID was unveiled, U.S. Senator Al Franken penned a letter to CEO Tim Cook requesting additional information on the system, including questions touching on data acquisition and retention. Aside from answers already available in Apple's PR and website, Federighi's answers today address many of Franken's concerns.

Finally, Federighi said Apple intends to release a white paper on Face ID for security researchers and other interested in learning more about the cutting-edge facial recognition technology.

Update: Apple has provided AppleInsider with additional commentary on Face ID:

Our teams have been developing the technologies behind Face ID for several years, and our users' privacy has been a priority since the very beginning.

Face ID provides intuitive and secure authentication enabled by the TrueDepth camera system and the A11 Bionic chip, which uses advanced technologies to accurately map and match the geometry of a user's face. Face ID data never leaves the device, is encrypted and protected by the Secure Enclave.

We've tested Face ID on people from many countries, cultures, races and ethnicities, using over one billion images to train our neural networks and defend against spoofing.

We're confident that our customers will love using the feature and find it an easy and natural way to unlock their iPhone X. We will offer more details on Face ID as we near the product's availability.