Video: A sneak peak at iOS 11 Augmented Reality at the Apple Park Visitor Center

At WWDC17, Apple introduced ARKit, iOS 11's new dev tools for Augmented Reality. It demonstrated interactive 3D games spilling off a table on stage and offered a hands-on look at a simple app that placed virtual objects on a table its iPads could recognize as a 3D plane. This week, it unveiled a practical new real-world app for exploring the Apple Park project.

Something old, something new, something that's $108M just for you

The day before Apple's big event in the Steve Jobs Theater, AppleInsider took some photos of the soon to be opened, new Visitor Center created for Apple Park, which sits just across from the main pedestrian security checkpoint entrance leading to the walking paths into the sprawling campus. The new Augmented Reality experience created for Apple Park shows off the potential of ARKit in iOS 11

What wasn't visible until Apple opened up its Visitor Center as a sneak peak reception after the event was the new Augmented Reality experience created for Apple Park, showing off the potential of ARKit in iOS 11.

When it opens to the public, the Visitor Center will primarily serve as an Apple Store, paired with a cafe. It also houses a vast, 11,000 pound model of the Apple Park site (below), which serves as an aluminum foil for the real star of the show: a virtual tour presented in AR using provided iPads. This new feature distinguishes the new site from the company's other retail locations.

Flagship af

The new Apple Park Visitor Center store also shares a lot in common with Apple's other big budget flagship locations. Despite being located in the sleepy suburbs of Cupertino, California (it straddles the city limits of Santa Clara; just beyond its rear wall there are quiet, tree-lined culs-de-sac of single family homes) the new location shares the same kind of spectacular architecture as some the company's most glamorous pinnacles of merchandising.

The new store features wonderfully crafted stone stairways (below) that convey the same sort of timeless craftsmanship felt in the stores integrated into historic gems such as New York City's Grand Central Terminal (above top), the former bank at Paris Opera (above middle) or London's spectacular Covent Garden site (above bottom).

The new Apple Park store also feels incredibly modern, with double-high walls of optically-pure glass that--like the breathtaking entrance to the Steve Jobs Theater--act as structural components, not just windows.

Unlike the cool urban sophistication of glass and steel seen in Hong Kong's Hysan Place and IFC tower (above top), San Francisco's Union Square (above middle) or The Oculus at NYC's World Trade Center (above bottom), the new Apple Park store blends into its orchard-like surroundings by making use of earthy warm wood slats in its ceilings and across its rooftop deck, which offers a view of the vast Spaceship hiding behind a forest of mature trees just across the street.

Al;AR: augmented aluminum with an iPad app

The aluminum site model is impressive in its heft but only offers a simplistic rendering of the buildings and landscape of Apple Park. However, when you hold up one of the AR iPads that workers hand you, a new AR exploring app lets you see detailed, photo-realistically rendered images of the actual buildings, complete with streets that feature animated vehicles driving down the road as if in SimCity.

You can flick on the rooftops of buildings to see inside them, where you can see the arrangements of internal walls, desks and even animated people walking around. Because the app is depicting the scene using AR, you can look up, across, inside and around the vast model to explore its mini-world of site details.

There's also a time-of-day slider that renders the AR graphics with lighting and shadows from early morning to high noon and into the evening, when the lights are switched on to get a feel of what it would look like to fly over the site at night. If you look at the grand cafeteria in the early morning, you can see an animated depiction of its massive window-wall-doors opening up to expose the vast indoor space to the outside

At dusk there's a red sky, and if you look at the grand cafeteria in the early morning, you can see an animated depiction of its massive window-wall-doors opening up to expose the vast indoor space to the outside, taking advantage of the Bay Area's mild climate (similar to the huge doors of the company's recently opened new San Francisco Union Square store).

In addition to looking under the roof to see the inner workings of the Spaceship, you can also lift the lid on the Fitness Center (which has contrasting stone walls, rather than glass like the rest of the buildings on the site).

You can even lift up the historic barn that's sat on the site for decades--long before Hewlett Packard paved the original orchards to put up a sea of parking lots for its executive briefing center. When you lift the barn, all that's left behind are some bales of hay and what looks like it could be Clarus the dogcow.

You can also lift the lid of the Steve Jobs Theater, which exposes the grand round display area that leads into the theater pit below. Phase 2 buildings are included in the model, but are not yet interactive to look inside (or perhaps that's just to keep the magical mysteries of the new R&D buildings under wraps).

Because the AR experience is dynamically created in software, Apple will be able to continuously update its AR exploration iPads to add new details and quirky Easter eggs to find-- perhaps they'll add a timeline depicting the site's seven years of construction, or a seasonal depiction of Apple Park as its trees lose their fall leaves or sprout out new ones in the spring.

Beyond looking at the structures, you can also use the AR iPad app to see how the site's solar panels and energy distribution work. In energy mode, the vast solar arrays on the Spaceship, the main parking structures and the Phase 2 parking and data center building glow in a sunny gold, and energy distribution is depicted by animated, sparkling tubes that connect the various buildings.

A third airflow mode lets you look at how ventilation systems help to passively cool the buildings. As you walk around the model, you can sense how cool blue air currents blowing in from the Pacific Ocean to the west flow over the site, picking up heat from human activity and the server farms inside, turning into red currents as they pass through.

Apple's un-goggled vision for AR is already here

The AR experience isn't just an interesting way to explore details of the campus that won't open to the public; it also serves as a depiction of how third party developers can use ARKit to create explorable, augmented worlds that are not just fun to fly over but can also serve as interactive, configurable presentations of information that isn't obvious or visible.

Beyond playing games that turn your iOS device into a window looking into a fantasy world, ARKit can also enable users to peel back layers of structure and overlay depictions of invisible forces such as electrical energy, magnetic fields and heat onto real world objects.

ARKit can be used to create apps that interactively visualize the activity inside of a volcano on the horizon, see elevators rising inside an office tower, observe the migration patterns of birds around a globe, track the real world movement of subway trains below your feet in a city, or visualize what you'd look like wearing a new sweater or a specific cut of jeans.

Apple's use of iPads to debut its visitor center AR tour also shows how useful it is to have an immediately intuitive multitouch interface for configuring your augmented view: you can simply touch and swipe to select what layers you want to explore, what invisible ideas you want to visualize and what time of day you want to see depicted in your AR environment.

This comes in contrast to the popular narrative that claims "real" AR or "mixed reality" inherently (or ultimately) demands the kind of goggles that Google and Microsoft have shown off, paired with gloves or handheld controllers to interact with elements in the virtual world.

The hardware expense of Glass, Hololens or Oculus, paired with a way to interact with what you see, combines to create the current barrier of expense that has blunted their mainstream adoption.As soon as iOS 11 ships on September 19, the ARKit platform will represent by far the largest installed base of AR devices

Additionally, goggle-based VR experiences are notorious for triggering nausea in many users because it divorces your body position and head movements from your visual perception of the world, something that Apple's screen-based, handheld AR doesn't.

While immersive VR goggles have entertaining and practical applications, the new AR worlds Apple has made available in iOS 11 allow the user to contrast regular and augmented views in the same way you might hold up a magnifying glass, seeing a before and after view of the difference in reality and the digital augmentation.

ARKit will also serve a much broader market with an installed base nearing 1 billion devices, enabling today's iOS users to experience AR without spending lots of money on an entirely new platform and new hardware up front. As soon as iOS 11 ships on September 19, the ARKit platform will represent by far the largest installed base of AR devices.

Apple hasn't yet detailed when it will open its new Apple Park visitor center to the public, but it's sure to become a popular destination for millions of people to visit, as its other spectacularly crafted retail locations have already become.