Modular 'Apple Glass' frame components may offer opportunities for expansion

Rather than augmented reality, the "Apple Glass" may have detachable arms that can be swapped for different functionality, including the ability to control devices in the real world.

The forthcoming "Apple Glass" may not be a single device. Instead, it or some future version, may be made up of components that can be swapped to suit a user's needs.

Apple's application for a "Modular system for head-mounted device" chiefly concerns swapping out the arms that let glasses balance on a user's ears. Those arms can have different functions, but the glass frames themselves may be swappable.

The application's drawings show "Apple Glass," or at least spectacles that resemble what that device is expected to look like, with different arms being inserted or removed.

"Head-mounted devices, such as head-mounted displays, headsets, visors, smartglasses, head-up display, etc., can perform a range of functions that is determined by the components (e.g., sensors, circuitry, and other hardware) included with the wearable device as manufactured," says the application. "However, space, cost, and other considerations may limit the ability to provide every component that might provide a desired function."

"For example, different users may have different preferences regarding the components and functions that are provided by a given head-mounted device," it continues. "Some users may desire certain capabilities, such as high resolution display and long battery life, while other users may desire other capabilities, such as smaller form factor."

Apple suggests that it may be possible to provide a set of arms that offer lesser functionality, but "long battery life when outside the home." Or the user could swap in frames that give "high resolution display at home," at the cost of that battery life.

Separately, Apple continues to investigate what a wearer will actually see through "Apple Glass" — and how they will interact with what's displayed.

When Apple chose to not pursue entire virtual reality systems, it was because Jony Ive didn't like the idea of users being cut off for extended periods. Following his lead, the forthcoming "Apple Glass" will use augmented reality to present virtual objects alongside physical ones in the real world.

Now Apple is also looking at blurring the lines between real and virtual objects, in order to help users control devices around them. A real-life physical object, such as a device, that's in front of the user, can be controlled by them tapping on a virtual copy that's only showing on their glasses.

The "Controlling External Devices Using Reality Interfaces" application is a 15,000-word document that specifies examples — though they read like Apple trying to cover all bases rather than actually explain anything.

For instance, the most often repeated example involves a user being able to turn off a real device. The real device is represented as an augmented reality object, it is a virtual device overlaid on reality, and a user can tap a virtual representation of the device's off switch.

Detail from the patent showing a user tapping a virtual button in order to control a real-world device

"In contrast to a VR environment, which is designed to be based entirely on computer-generated sensory inputs," says the patent, "a mixed reality (MR) environment refers to a simulated environment that is designed to incorporate sensory inputs from the physical environment... [An] MR environment may track location and/or orientation with respect to the physical environment to enable virtual objects to interact with real objects."

So as you turn to look at a television, your "Apple Glass" could overlay that device with a virtual one. As your hand reaches up to tap the virtual device, the AR system could tell the real-life one to react. "In accordance with determining that the hand gesture satisfies one or more predetermined criteria associated with the function,"says the application, "[the] user device... causing external device to perform the power on/off function."

If the augmented reality object is precisely mapped to a real one, this doesn't sound particularly useful. Instead of registering that your hand has reached up to tap the virtual device, you could just as easily press the off switch of the real one.

It could be interesting, though, if two people using "Apple Glass" were able to control a device separately. Sticking with Apple's television example, one user might turn up their volume, or the other might change channels, with both solely effecting what they personally see or hear.

Apple's patent application does also talk about when virtual objects are not mapped on top of real ones, though. In descriptions that are reminiscent of the way Apple has previously used ARKit to show representations of Macs on desks, the application talks about being able to walk around an object.

"For example, a system may account for movements so that a virtual tree appears stationary with respect to the physical ground," it says.

This inventors are credited on the patent application about controlling external devices, Justin D. Stoyles, and Michael Kuhn. Stoyles is credited on one patent for generating a video of an emoji based on a user's facial expressions. Kuhn's very many previous patents include one for representing virtual information in a view of a real environment.

The patent application for swappable arms on "Apple Glass" is also credited to two inventors, in this case Paul X. Wang and Dinesh C. Mathew. Wang's many patents include one regarding using motion to control Macs, while Mathew is also credited on ones involving a smart ring.