Apple plans to dominate all aspects of AR with 'Apple Glass' and more

The rumored "Apple Glass" headset may be a single hardware product, but the continuing flood of research looks more like Apple is intending to use AR as a sea change in how we use all of our devices.

Whether Apple AR's first product turns out to be "Apple Glass" or something else, it is not going to ship as a hobby device like the Apple TV. It's also not going to be even remotely like AirPods or HomePod, which might slot in to the existing Apple Music service but are otherwise standalone products.

Instead, when Apple AR hits, it is going to ultimately hit big. Perhaps it will launch with one device at first, but that device will bring with it something akin to its own ecosystem — and one that it is at the very least as comprehensively worked out as the iOS one is today.

Apple has not started with the idea of making an AR device and seeing where it goes. The company has spent years investigating every possible aspect, designing new systems and devices. And it has been patenting everything along the way.

New patent application filings this week represent just the latest of countless filings, but they also provide a preview of just how thorough and wide-reaching Apple is being.

One, filed in January 2020 but revealed this week, tackles the broad issues regarding "Head-Mounted Display and Facial Interface Thereof." It's concerned with the core principles of fastening a unit to a wearer's face, and then about displaying graphical content on it.

Detail showing one method of securing a headset to a user's head comfortably

The separation of the unit and the display into related but different concerns, seems to be key to this application. So is making that head-mounted display stay on the user's head securely and comfortably.

"The facial interface is coupled to the display and engages the face of the user to support the display thereon," says the patent. "The facial interface may influence comfort of the user, especially when worn for long periods of time, and stability of the head-mounted display on the head of the user."

It's about fitting a head-mounted device and keeping it secure. "The facial interface includes an upper portion that engages a forehead of the user and side portions that engage temple regions of the user," continues the application. "The facial interface converts forward force applied to the upper portion by the forehead into inward force applied by side portions to the temple regions."

So where regular glasses might need only to be balanced well on the ears, this patent proposes a tighter fit that would keep any attached display steady. While describing it as being adjustable, and say that it is meant to "comfortably engage the facial structures of the user," Apple does also refer to it making "a light seal."

This application is credited to the same three inventors listed on the recent one regarding "Apple Glass" measuring stress. That includes Kristi E. Bauerly, whose previous work includes a parent regarding "an improved form of headphones."

Computer Generated Reality

Similarly, "Computer-Generated Reality Platform," is another broad patent application, this time concerned with almost the opposite issue in AR, the issue of what is shown to the user. It's specifically about creating immersive and high-quality images, both of which have also been addressed in previous Apple patents.

"As the capability of electronic devices increases and their ability to output high-quality visual displays improves, applications are becoming more immersive," says this application. "One such example is the increasing mainstream demand for computer-generated reality (e.g., "mixed reality" and "virtual reality") applications."

Part of this application covers procedural issues to do with the sequence of events when one device responds to "a request to launch a first application in the three-dimensional computer-generated environment."

Detail showing one example of a real-world environment being augmented by AR

Most of the rest of the application is concerned with broad descriptions of what constitutes a Mixed Reality (MR) system, where a user sees the real world plus virtual objects placed in it. However, Apple is also concerned with enabling extra features — and providing privacy.

"Such techniques can improve the user experience, increase privacy and security, and enable computer-generated reality interfaces (e.g., 3D interfaces) with advanced functionality," it says.

This application is credited to six inventors, including Timothy R. Oriol, most recently listed on Apple's plans for a virtual privacy screen.

Apple's big augmented reality concerns

Apple wants to bring AR to market as a fully-worked out and practical proposition, with useful and desirable products. But throughout the very many patents it's filed or had granted in the last several years, it has also continually worked to fit AR in with its existing eco-system. And with its other main desires.

Chief amongst those is privacy. While certain patent applications, such as that privacy screen one, address the issue directly, all of them refer to it. All of them. AppleInsider has not read a single patent that doesn't at the very least include a nod to the existence of privacy issues.

Invariably, they also devote some time to showing how a given AR project addresses the issues or, very often, is itself presenting solutions.

Just occasionally, though, another Apple concern comes through and this one is rather more secret. Apple is forcefully upfront about its privacy policies, and Tim Cook has frankly admitted to pursuing AR, but it never talks about "Apple Car."

It's the project that is so big that Apple is regularly required to file legal documents regarding it, and yet won't even acknowledge its existence. It also files for patents on just about everything from self-driving mechanisms to having the car measure the driver's stress levels.

Augmented Reality, the Apple Car, and "Project Titan"

"Apple Car" and Apple AR cross over many times in this slew of patents, though. The company has even looked at how passengers could wear "Apple Glass," or similar devices, on the road — without getting car sick.

Detail from Apple's latest patent application regarding AR in the Apple Car's windshield

Alongside the patent applications revealed this week for the headset and how the whole AR environment is presented to users, Apple also filed for one regarding windshields. It has done this before, too, with a 2018 filing regarding projecting AR onto a car windscreen while being driven, for instance.

The latest one concerns taking what a driver sees through the windshield and then representing them as virtual AR objects on a car's dashboard.

"When traveling... users may often reference their surroundings when developing plans or requesting information about the world around them," says this patent application. "However, it is often difficult to present requested information to the users of the autonomous machine in a manner that is intuitive to the users, i.e., presenting information of a three-dimensional space on a two-dimensional display."

"Also, oftentimes the indications of the users to reference objects in the world around the machine may be imprecise, especially as the machine is moving through the environment," continues the application. "Thus, it may be difficult to accurately determine what object within the world about which the user of the machine is attempting to receive information."

If the car's dashboard can display a representation of the buildings on the road, for instance, it may be easier for a passenger to say "turn left at the next block" and be understood.

Apple Augmented Reality everywhere

No doubt any firm working on any AR products would like to see them widely adopted. In Apple's case, though, it feels as if AR is going to properly come out of the gate and be instantly integral to just about everything the company does.

Apple can take the time to bring all of these disparate issues together, and Apple has taken the time to work on every conceivable aspect of AR.

Perhaps only Apple could bring an entire ecosystem to the market in one go. But certainly, and soon, we're going to suddenly wonder why we ever bought devices that don't use AR.

Keep up with AppleInsider by downloading the AppleInsider app for iOS, and follow us on YouTube, Twitter @appleinsider and Facebook for live, late-breaking coverage. You can also check out our official Instagram account for exclusive photos.