Computer simulations, and the means to visualize them, could be in the palm of your hand in new and potentially revolutionary ways later this year, if rumors about Apple's forthcoming "iPhone 8" bear fruit.
Apple CEO Tim Cook has made an assortment of remarks, heralding augmented reality and virtual reality as potential cornerstones of the company's future. Combined with rumors of a new dual-lens forward facing camera system and advanced facial recognition software in the "iPhone 8," it's possible that Apple's true interest in the space could be revealed later this year.
While virtual reality promises users a more immersive experience, by sinking a user in the environment with video and audio cues, augmented reality instead offers a hybrid and more social approach, merging a virtual world with the one around us.
Augmented reality is a clear interest for Apple. Rumors about the hardware in the "iPhone 8" coupled with Apple CEO Tim Cook's remarks about the technology suggest that the company is going to dive deep into augmented reality very soon.
For about three decades, the term "virtual reality" has been used as a catch-all for any sort of real-world simulation or modification. Initial implementations were used by the entertainment industry for relatively compact rides in amusement parks, with additional user by the military for flight simulator training.
Early implementations employed conventional displays in conjunction with mechanical haptics to provide full feedback of the environment to the user.
Because VR is an immersive and largely solitary experience that requires a headset, it's an unlikely fit for Apple. Instead, the company is likely to embrace an offshoot of VR, augmented reality, in a way that can allow users to interact with the world — Â and others — Â around them, via the iPhone.
AR is less hardware- and software-intensive than VR, using the combination of a camera and display such as those found in the iPhone to manipulate and generate overlays over the surrounding real-life environment. Augmented reality can be used to guide a user on a street, or highlight businesses as a user travels through a town.
Another possible use case is capture of virtual creatures with the flick of a finger, after spotting them in the park on a daily walk such as in Niantic's "Pokemon Go."
Done properly, augmented reality could be readily adopted by the iPhone crowd — assuming it can be made "friction-free." As it turns out, that's an area where Apple historically has excelled.
Tim Cook on augmented and virtual reality
"The smartphone is for everyone, we don't have to think the iPhone is about a certain demographic, or country or vertical market: it's for everyone. I think AR is that big, it's huge," Cook said in an interview in February. "I get excited because of the things that could be done that could improve a lot of lives. And be entertaining."
"I view AR like I view the silicon here in my iPhone, it's not a product per se, it's a core technology. But there are things to discover before that technology is good enough for the mainstream," Cook concluded. "I do think there can be a lot of things that really help people out in daily life, real-life things, that's why I get so excited about it."
"I think AR is that big, it's huge" - Tim Cook
Apple's tangible moves in the AR and VR sector
Apple has made a number of strategic hires and acquisitions over the last five years to further its AR ambitions.
German AR firm Metaio and Flyby Media also go hand-in-hand with in-house development of transparent displays, iPhone-powered VR rigs, AR maps and other related technologies described in recent patent filings.
In January, Apple was reassigned IP from Metaio for an AR device with advanced point of interest labelling. Specifically, a pair of patents detail a mobile AR system — or smartphone — capable of detecting its surroundings and displaying generated virtual information to users in real time.
The "iPhone 8" and augmented reality
While the "iPhone 8" may not be sold to customers on AR prospects alone, it appears to be setting the table for wider adoption of the technology.
According to recent rumors, the sensor expected to be used in this year's "iPhone 8" is intended for a practical face scanning utility, and not solely or specifically for augmented reality application.
However, the technology described for the sensor appears to be a micro-miniaturized version of a LIDAR mapper or rangefinder.
LIDAR is a proven technology, with resolution and accuracy dependent on a combination of factors, not the least of which is the integration of the laser and the sensor — which Apple's acquisition of Primesense in 2013 appears intended for.
To properly overly the environment requires accuracy — which a rear-mounted LIDAR installation supplementing existing range finding equipment associated with the iPhone camera would give. It also requires heavy-duty processing power, which the existing A10 Fusion processor also has — and any successor would as well.
According to analyst Ming-Chi Kuo of KGI Securities, the new camera system expected to arrive in Apple's "iPhone 8" will enable 3D sensing and modeling in applications, which could be used for a range of capabilities including replacing the head of a character in a 3D game, or taking a 3D selfie.
Beyond proprietary camera software, Apple has other AR- and VR-related edges over its competition as well. For example, the iPhone only has a few resolutions, where Android devices have literally hundreds.
The "iPhone 8" is expected to be thinner and lighter than even the iPhone 7. This could allow for less user fatigue, whether installed in a headset, or held by the user for an AR environment overlay.
Apple as iterator, not innovator
First off the starting blocks doesn't guarantee victory — just ask Eiger Labs or Creative Media how their corporate fortunes are doing in the wake of the iPod, how Microsoft's tablet initiative fared after a 2002 debut, or how Samsung's smart watches are selling.
Apple wasn't the first to the computer mouse, the digital media player, an internet-connected phone, or the tablet form factor. It waited until the technology and time was right, perfected each form factor — making it truly "friction free" to adopt.
And, like most of the rest of the developments its known for, Apple isn't going to be first to AR or VR. Google Glass rose and died in less than two years, and is beyond a niche in the company's Mountain View headquarters, if it isn't completely dead — all because the product didn't work well.
Virtual reality is tied up in litigation between Zenimax and Facebook, after the latter made a multi-billion dollar bet on the technology.
Apple stands at a point with the "iPhone 8," controlling the current specs and future development of a near-ubiquitous device sold in the millions per quarter, where it can call the shots in AR for now, and the next decade at least.