A newly-granted patent suggests Apple Vision Pro could be used to show energy and objects not visible to the naked eye like electrical current, radio signals from Wi-Fi, or HVAC air flow to help engineers and troubleshooters.
It's like the way a museum floor may look empty until a spy wearing just the right headset can see the criss-crossing laser lines of a security system. But with Vision Pro, it won't be that the headset has some kind of infra-red filter.
Instead, according to the newly-granted patent "Visualization Of Non-Visible Phenomena," Apple Vision Pro could visually represent sensor results that it or connected devices detect.
"For example, a device may detect, and/or receive information regarding, one or more non-visible features within a direct or pass-through field of view of a physical environment," says that patent, "and display a visualization of those detected non-visible features at the correct location of those features in the physical environment."
"The non-visible features may correspond to, for example, electromagnetic signals such as Wi-Fi signals, airflow from an HVAC system, temperatures of physical objects, fluids or gasses," it continues, "[or] an audible fence created for a pet (e.g., using ultrasonic pitches), sounds generated by a musical instrument."
Presenting such information visually first requires detection of the signals. Apple lists a need for "cameras, image sensors, touch sensors, microphones, inertial measurement units (IU), heart rate sensors, temperature sensors, Lidar sensors, radar sensors, sonar sensors, GPS sensors, Wi-Fi sensors," and more.
These sensors don't have to all be present in the headset, though. They can instead be in a connected mobile device, such as an iPhone.
Apple's patent is concerned less about the use of the proposal and more about the details of how information can be received and acted on. It's also written in the same way that all patents try to cover every conceivable future use of the idea.
So while the majority of the patent is concerned with a headset, Apple does also mention that this visualization of non-visible items could instead be present included in an Apple Car windshield.
That's because "non-visible" need not mean something human eyes can't see, like electromagnetic fields. It can equally mean something that we are blocked from seeing, such as 'hidden physical objects such as objects with known locations that are obscured from view by other physical objects."
This patent is credited to six inventors. They include Arun Srivatsan Rangaprasad, who previously worked on systems for Apple Vision Pro to identify and annotate items of interest.
6 Comments
I wonder if it would be possible to have a binoculars feature? Uses could be spotting and pinpointing forest fires from a firetower, navigating a boat, bird watching and identifying, etc.
I wonder if Apple will sue Teledyne FLIR for violating this patent? After all, FLIR's smartphone thermal camera accessory allows the user to see non-visible heat, overlaid on an image of the scene (combining the data from the external thermal sensor with the phone's internal camera. I am not an expert on patents, but it seems to me that Apple has made their patent claims broad enough to encompass some prior existing products.
Perhaps FLIR sues Apple for using FLIR patents? Didn't they (FLIR) have their patents for many years before Apple started this project?
It's not the idea, it's the actual method that's patented.
Definite parallels to the VISOR worn by Geordi La Forge on Star Trek: TNG. The Vision Pro wouldn't necessarily need external sensors itself for some applications if the observed device were output data from internal sensors. For example, if a machine had sensors collecting temperature, electrical, noise, or motor speed data, that could be appear as an overlay when visualizing. Some information could be extrapolated from other data points and that could appear as well.