Apple looks to the future of video conferencing with Memoji avatars

Forget video conferencing through Zoom, Apple has been looking at a way of using virtual and augmented reality to stage meetings in 3D space. Each attendee can be at their own table and "see" the others arranged around them. It's not quite like having a "Star Wars" type of hologram of the person, but through screens or glasses, you'd be able to see avatars that move and interact.

The application, "Communication system and method for providing a bionic virtual meeting room," couches the idea in typical highly detailed legalese, but the concepts behind it are fascinating.

"The invention relates to a method for operating a communication system for providing a virtual meeting of at least one first user and at least one second user," starts the patent. "The invention relates to... displaying a virtual meeting space on a display device [and] detecting a biometric characteristic of a first user.

"[It involves] determining a level of abstraction associated with the first user from among a plurality of different settable levels of abstraction," it continues, "and displaying, in the virtual meeting space, an avatar representing the first user having the biometric characteristic of the first user."

"This way, objects, especially virtual objects, like an avatar representing the first user, can be presented to the second user over any distance, which can advantageously be used to provide virtual meetings between different users, especially any arbitrary number of users, in a common virtual meeting room," says the patent.

Think of the way that Animoji and Memoji present an animated face that shows the same mouth and head movements that the user is doing. The patent stresses that "providing a biometric characteristic" of each user is not just for fun, it is for directly improving communication.

"The main area of application of such virtual realities lies in the field of entertainment and games," admits the patent. "But it would be desirable to make use of such virtual realities also in other fields, especially in the field of human interaction."

"Such adaptions by biometric characteristics of the first user are especially important for human interaction," it says. "For example the addition of gaze directions, eye blinking or mimics as biometric characteristics to virtual characters in the virtual meeting space can greatly improve empathy and perception of the person."

"In many situations, especially in international, professional or other intercultural contact it has been determined as useful to modulate and moderate individual behavior and appearance to either tone it down, bias or amplify it to support the purpose of the interaction and communication," continues the patent.

If this feels like just one step below having holograms of people in the same meeting room, it does still require a method of displaying these avatars to each attendee. That means the patent goes to lengths to describe how such avatars could be shown on devices.

These could be "computers, mobile virtual reality or augmented reality glasses, tablets or smart phones" and the specifics of how each avatar is displayed varies between the devices. Consistently, however, the patent is concerned with the fidelity of presenting a human's characteristics through these avatars.

The invention is credited to Eberhard Schmidt, who has previously been listed on patents regarding display apparatus and presenting images on display devices.

This patent does not concern itself with the details of how an avatar may actually be created or chosen by the user, just with how it can react and display their characteristics. A previous Apple patent, however, has described how a future iPhone may be able to generate a user's Memoji automatically from a photograph.