Mikey Campbell
In a patent filing with the U.S. Patent and Trademark Office on Thursday, Apple outlined a system that analyzes the characteristics of an image's subject and uses the data to create a "faceprint," which can then be matched with other photos to determine a person's identity.
Curiously, most of the patent language in Apple's application for "Auto-recognition for noteworthy objects" focuses on "famous people and/or iconic images," and not a user's friends or family.
The invention calls for a digital image management application to analyze a subject in a given photo, creating a "faceprint" of the person by using facial recognition to identify certain unique characteristics. Here the filing notes that a faceprint is "a subset of feature vectors that may be used for object recognition," and can therefore be used for non-facial objects like structures.
From the filing's background:
In order to automatically recognize a person's face that is detected in a digital image, facial detection/recognition software generates a set of features or a feature vector (referred to as a "faceprint") that indicate characteristics of the person's face. The generated faceprint is then compared to other faceprints to determine whether the generated faceprint matches (or is similar enough to) one or more of the other faceprints. If so, then the facial detection/recognition software determines that the person corresponding to the generated faceprint is likely to be the same person that corresponds to the "matched" faceprint(s).
After a photo is analyzed, the management application compares the faceprint with other generated faceprints stored locally or remotely. If a match is discovered, the software tags the person with an identity, such as Tom Hanks. This stage relies on face recognition technology that assigns a reliability score to the faceprint, thus allowing for more accurate matches. If a score falls below a certain threshold, the system rejects the match.
User feedback can also be included during this phase to aid in enhancing the attached metadata by pressing either a green check or red "X" button.
As for the database, the patent allows for a device like the iPhone to create and store faceprints locally or pull from an off-site cache. This remotely-generated faceprint store has the ability to operate in the cloud, and can be sent to or shipped with the device. Images can be pre-processed and tagged with metadata for quicker facial recognition, but the step is not necessary for the system to function as the remote system is integrated in the cloud.
Illustration of remote faceprint database with metadata.Additional features include the ability to group together multiple faceprints using metadata along with reliability scores. The patent gives the example of pictures of Paul McCartney taken over a number of decades. While his face looks markedly different than it did forty years ago, vetted metadata assigned to a pool of images can help parse out photos of the legendary artist to give a decade-by-decade retrospective.
Going further, the metadata can be used to offer information about a subject like their Facebook page, Twitter feed or an iTunes Store link to their music.
While the exact purpose of the invention is unclear, the solution could extend beyond famous people and places and be used to intelligently identify subjects in a user's photo library for quick and easy tagging or file management.
-m.jpg)
Christine McKee
Andrew Orr
Wesley Hilliard
Brian Patterson
Andrew O'Hara
Malcolm Owen
9 Comments
So would this face print thing work in the context of photos taken and stored on a device? Or would it also ultimately be used to unlock a device? ie if the different sensors notice that the device is being moved from the pocket to being used, would it then unlock after face recognition?
And the hits just keep on coming. Keep an eye on the copy cats.
Actually, after reading and possibly failing to understand the claims in the patent application, I rather doubt that this will survive a challenge. Face recognition is widely established in a number of contexts .... access control systems, security scanners ... why even facebook does something similar and gets itself into trouble with privacy law.
I would be surprised to find there is no relevant prior art.
What good is this patent, when the USPTO will just invalidate it in a couple years.
[quote name="ombra2105" url="/t/154991/apple-invention-uses-faceprints-to-identify-people-objects#post_2245915"]So would this face print thing work in the context of photos taken and stored on a device? Or would it also ultimately be used to unlock a device? ie if the different sensors notice that the device is being moved from the pocket to being used, would it then unlock after face recognition?[/quote] It could be an extension of their previous feature in iPhoto that identifies people in photographs, but I doubt it would be used for security purposes. It's just to easy to trick. I would think this would be better for FaceTime for a TV because a user doesn't sit directly in front of the TV like with a Mac or iDevice. You would need an camera that can zoom and crop an image to the appropriate size for a face when in to be useful. Perhaps the home or remote user can use a button on the remote to move from the whole room to face to face without ever actually having to adjust the camera and have the camera focus follow the person if they shift a little.