Apple AR plans to stop 'Apple Glass' wearers walking into traffic

Alongside multiple research projects to do with different elements of creating an immersive Apple AR experience, Apple is investigating how to protect "Apple Glass" wearers from colliding into real-world objects, walls, or cars.

Apple has still not said that is making "Apple Glass," but the previous sea of patents on the subject have now been joined by a further squall of patent applications. Overall, these new ones cover half a dozen different small essentials for such a Head Mounted Display, but one in particular addresses potential problems.

Specifically, if you are wearing "Apple Glass" and so are experiencing some amazing audio visual smorgasbord of augmented or virtual reality creations, you are still standing or walking in the real world. Right alongside virtual objects there are going to be real ones, and it hurts a bit more if you walk into one of those.

"While the user is fully or partially immersed in a VR setting of virtual objects, physical objects in the surrounding physical setting continue to exist," says Apple in "Techniques for Switching Between Immersion Levels," a newly-revealed patent application.

"For example, the user may be fully immersed in VR corresponding to a pre-historic world populated with dinosaurs," dreams Apple. "While that virtual pre-historic world may be deficient in living room furniture, the living room in which the user is located continues to include a coffee table."

The object on the table is virtual, but you'd best know the table is real before your knees hit it

"Moreover, even though the user's dog may be absent from that virtual pre-historic world, the dog may continue to roam about the living room," it continues.

According to Apple, previous solutions to this have boiled down to the user taking off their headset when they suspect they're about to tread on their dog. "However, as experiences become increasingly immersive, the user may be unable to sense such undesirable interactions fast enough to avoid them," it says.

"Moreover, abruptly removing the electronic device during an experience detracts from that experience," continues Apple. "As such, it is desirable to address the concerns related to these undesirable interactions while also minimizing any negative impacts on the experience."

Apple's new proposal boils down to a kind of picture-in-picture effect. Where the "Apple Glass" wearer is strolling through downtown Pleistocene Era, they can be shown a live view of their real surroundings.

The patent application is about when they would see this, when it would be triggered. But the aims are to keep the user in their AR experience until they really need to be alerted to real-world problems.

Synthesized reality

A second of the six new patent applications is concerned with effectively making a great screensaver while you listen to music in Apple AR. "Method and Device for Presenting an Audio and Synthesized Reality Experience," is concerned with making an immersive audio/visual experience.

"While music is typically an audio experience," says the patent application, "the lyrical content, sound dynamics, or other features lend themselves to a supplemental visual experience. Previously available audiovisual experiences, such as music videos and/or algorithmic audio visualizations, are not truly immersive and/or are not tailored to a user environment."

Music can be quite immersive enough by itself, thanks, but Apple wants to keep your eyes busy as you listen. It also, separately, wants to keep the headset stay positioned correctly over your eyes, and allow you to adjust them too.

They're called head-mounted, but they also need to stay on comfortably, and for a long time

Keeping "Apple Glass" on straight

There are myriad details that need to be done right for an Apple AR device to be worth using, and the latest batch of patent applications covers as many of those as it can. "Head-Mounted Device with Adjustment Mechanism," for instance, is concerned with just how the user — or an app — can move lenses to make the perfect experience.

"Typically, two lenses are included to display slightly different images to each of the user's eyes in accordance with stereoscopic vision techniques," explains Apple. "An adjustment mechanism may be included to allow the user to change the distance between the two lenses so that the lenses are approximately aligned with respect to the user's eyes."

None that is much use if the "Apple Glass" device doesn't sit straight on your nose, though. So "Lens Mounting Systems for Electronic Devices" is about the "support structures" that hold the lenses and display.

"Adhesive may be used to attach the lenses to the head-mounted support structures," says Apple, which conjures up Harry Potter-style images of spectacles held together with tape.

Eye-tracking and deliberate taps from "Apple Glass"

With the "Apple Glass" on correctly, the lens aligned, and the dog/dinosaur proximity warnings enabled, Apple has two more concerns in its latest patents. One is the opposite of your bumping into things, and it's the Apple AR device deliberately bumping you.

"Head-Mounted Display With Haptic Output," is concerned with matching taps and pressure to whatever audio/visual experience you're having.

"Aural content includes sounds, such as those occurring in the movie, the virtual environment, or the productivity program," it says. There is also, though, a possibility that the haptic taps could instead be related to important notifications.

"[Apps could control] the haptic output devices to provide the health haptic output," continues Apple. "The health input may be a breathing instruction or an exercise instruction for instructing the user."

Alerts like this, or warnings of impending traffic, are also all related to one last newly-revealed patent application. Apple AR needs to know where a user's concentration is, and that is typically determined by reading where their eyes are looking.

"Eye Tracking Using Low Resolution Images," is a way to do this using minimum overhead on the Apple AR device. "[These] eye odometers may be used as a low-power component to track relative movement of the user's eyes with respect to the device," says Apple.

This patent application is about what you're looking at. The others in this new set cover that, plus how the glasses physically align their lenses, and how you can be both immersed and yet alert to real-world surroundings.

Apple may not be making any official announcements, but it is sweating the details.

Stay on top of the latest Apple news right from your HomePod. Say, "Hey, Siri, play AppleInsider," and you'll get latest AppleInsider Podcast. Or ask your HomePod mini for "AppleInsider Daily" instead and you'll hear a quick update direct from our news team. And, if you're interested in Apple-centric home automation, say "Hey, Siri, play HomeKit Insider," and you'll be listening to our newest specialized podcast in moments.