Apple AR headset games could use Face ID to properly identify & outfit players

Patents surrounding Apple AR and VR efforts now include one that focuses on how computer-generated reality allows players to change their appearance — and what happens when the hardware owner doesn't want them to.

It wouldn't be much of a fantasy battle game if all the characters were dressed in their users' casual clothes. But Apple argues that changing your appearance in a game could present some unwanted issues, and it's been granted a patent on how to stop it.

Specifically, "Identity-based inclusion/exclusion in a computer generated reality experience," is a newly-granted patent that will prevent a player changing their appearance. And if necessary, will simply exclude them from the entire CGR game.

"While a device is presenting a CGR experience to a user in an environment, representations of people within the physical environment may be altered," says Apple. "For example, in a multiplayer gaming experience, people other than the user may be presented as monsters or donning battle armor."

"However, in various circumstances, it may be undesirable for all people to be included in the CGR experience," it continues. "For example, it may be undesirable for a user's children or safety personnel to be displayed in an altered manner."

What Apple plans is for a person entering your game — or any other multiuser CGR environment — to be identified. Then depending on their identity, they can either be not allowed to change their appearance, or they can be excluded.

It does depend on whether the CGR is all you see, or whether this is an AR case where you're seeing a mixture of real and virtual objects. You can't erase someone from reality, but you can prevent them changing their appearance.

The patent is mostly concerned with identifying users in the CGR environment. Oddly, it doesn't cover recognizing them via their login credentials. That would not be foolproof, but it would be a basic start.

"[Instead, the] method includes, while presenting a CGR experience, capturing an image of scene; detecting, in the image of the scene, a person; and determining an identity of the person," says Apple. It uses a "a deep learning neural network" similar to how Apple describes how Face ID works.

"[The] first identity of the first person is determined by applying a classifier to the pixels of the pixel set [captured by a user's headset]," it says. "[The] classifier is provided with a positive image of a particular person and generates a feature vector by comparing the positive image to many other negative images of many other people (e.g., thousands of images from a library)."

Detail from the patent. Figures 501 or 502 could be excluded from the CGR environment

So each user who enters the game can be compared and identified in much the same way that Photos handles face recognition. The patent discusses an alternative, though, which is simply that when someone enters your game, you are prompted to identify them as someone who should or should not be allowed to change their appearance.

This does all mean that each user in a multiplayer environment can see different people changing their appearances. The recognition is designed per user, at least as far as this particular patent is concerned.

The patent is credited to seven inventors, including Daniel Ulbricht. His previous related work includes a patent application about how an "Apple Glass" kind of headset could track where a user is looking.