Augmented reality could offer some benefits in medical applications, Apple believes, suggesting a system were a real-world image is warped so a user of AR glasses can see everything in front of them, even if part of their vision is obscured through partial blindness.
The majority of applications for augmented reality that most people will be familiar with is how it can be used to insert virtual items in a realtime view of the world. While this is useful and entertaining, Apple has looked into ways to take advantage of the technology for more serious matters, such as aiding the visually impaired.
A person's vision can be affected by many different events and conditions, such as the general weakening of sight due to age. Health issues including diabetes, problems with the retina, eye "floaters," and blind spots can form to obscure vision, as well as other ailments where the eye is damaged from a traumatic event.
In these cases, either the user has part of their vision obscured, blurred, or requires some other aid to see the entire picture. A patent granted to Apple by the US Patent and Trademark Office on Tuesday titled "Augmented Reality Device to Warp Images" seeks to aid with just such a problem.
In Apple's solution, a camera or imaging sensor captures the environmental view directly in front of the user, typically from the same perspective as their eye. A portion of the field of view is then selected for further processing to warp the image for the user, then the composite view is then placed in front of the user's eyes.
An example of how a real-world scene can be warped so the user can see behind their ocular obstruction.
The warping functions applied vary depending on what information is provided to the system. In cases where part of the user's vision is completely destroyed, the warp function could compress the obscured area to a side or multiple sides around the obstruction, so the user can still see what is in the area they cannot normally view.
For those with vision intact, areas could go through similar measures to magnify the vision, making it easier to see elements. This option may even be beneficial to users who have normal vision, if only to see a zoomed-in area of a location.
As users move their eyes, there is also a need to use eye tracking technology. Once the gaze of the user is determined, the warp area can be changed to match where the user is specifically looking, so as to not lose the benefits of the technology.
The filing largely relates to the use of a mobile device for processing and showing the distorted view, such as with an iPhone or iPad, but it is plausible the same technology could be employed in augmented reality glasses.
An image from the patent filing of a visually-impaired user viewing a scene via a mobile device
Apple files numerous patent applications with the USPTO on a weekly basis, but while the existence of a patent application doesn't guarantee a future product or service will use the submitted ideas, it does show areas of interest for the iPhone maker.
The current level of augmented reality technology offered by Apple's devices and software already provides image processing capabilities, such as how ARKit can be used on TrueDepth-enabled iPhones to apply filters to a user's face, or to control Animoji.
Closer to the warping offered in the patent, a more recent example is iOS 13's FaceTime Attention Correction feature, which changes the view of the user from staring down to staring directly at the other call participant, by slightly warping the eyes.
Apple has also filed numerous patents and applications in the fields of augmented reality glasses and stereoscopic vision, and is tipped to be bringing out some sort of headset by as early as 2020.