Future Memoji could be created automatically from user's photos

An iPhone may automatically generate Memoji for a user in the future, by analyzing a photograph of the user's face to determine the closest combination of facial features to produce a digital avatar with a similar likeness, without the user needing to make any major choices.

Apple's Animoji and Memoji features have proven popular, having been used for advertising purposes as well as for social media trends like Animoji Karaoke. In iOS 13, Apple extended Memoji to allow users to create collections of stickers using the user-based character, which can be added to Messages.

Even so, the feature does rely on users going through a process to create their Memoji, which can take a few minutes to accomplish, though to get a perfect likeness it can take quite a while longer. Some users, such as the visually impaired, simply cannot use the feature as intended, and either have to rely on another person to capture their likeness accurately, or resign themselves to using an Animoji.

In a patent granted by the US Patent and Trademark Office on Tuesday, Apple 's "Technique for creating avatar from image data" suggests this need not be the case, as one could be produced from a photograph.

In short, the patent uses an image of the user to determine the facial features of the subject, then selects digital representations and compiles them into an avatar.

The processing method measures the spatial facial attributes of the subject, classifying the expression of the person at the same time, and determining which from a selection of avatar templates is the closest match. The template avatars are deformed based on the expression classification to aid matching.

An illustration of how images of a subject can be broken down into facial features and location measurements

Other attributes of the subject are then identified and the selected avatar template is deformed further, before the process is completed an a user's avatar is available to use.

The classification can include the identifying of spatial locations of hair on the subject, then to determine multiple different templates of hair styles which can be attached to the template, as well as for identifying eyeglasses, facial hair, and other elements. Eye, hair, and skin color can also be used as part of the final deformation process.

The data for the compiled avatar could also be transferred into other templates, such as to provide users with variations on an appearance. This can include costume changes or those of expressions, or even the aforementioned Memoji stickers.

Apple files numerous patent filings on a weekly basis, but while the existence of a patent suggests areas of interest for Apple's research and development efforts, they don't guarantee the described concepts will appear in a future product or service.

In this case, as it is an image processing patent that is entirely based in software, and given Apple's existing work with Memoji, it seems plausible some form of automated Memoji creation feature could be made available in the future.

While the patent is more about a flat 2D image rather than using three-dimensional data, the existence of Face ID and the TrueDepth camera array may help Apple further the concept. By using the available 3D mapping capability, Apple has the potential to take detailed measurements of a user's face for avatar creation, with the relative ease potentially allowing users to generate new Memoji whenever they want.

This is not the only way Apple believes it can enhance Memoji and Animoji for its users. One patent application that surfaced in November 2018 suggested the recording of audio with Animoji, but rather than leave the audio plain, Apple proposed switching out spoken key words for sound effects and custom character animations, such as a dog's bark when the user says "woof" to the iPhone.