Apple's AR ambitions could expand into Personal Shopping and tech support sessions

Using hardware like Apple Vision Pro, future Apple Store Personal Shoppers could demonstrate devices to users, and show the products interactively within the customer's home.

Apple already uses AR to preview new devices, such as letting a prospective Mac Pro user see just how big that machine will be on their desk. Even Apple doesn't do it for every device, though, and new research about personal shopping suggests that it intends to.

A recently-granted patent called "Guided Retail Experience" shows Apple merging AR with its current online chat system. Instead of a text box, this patent proposes showing users see live video of an Apple Store Personal Shopper.

"Conventional electronic consumer experiences allow a user to browse and purchase products online with an electronic device, such as a laptop or desktop computer, tablet computer, or smartphone," says Apple.

"[However, online] shopping can lack the instantaneous feedback, answers to questions, suggestions, demonstrations of products, and human connection of an in-person shopping experience," they say. "Additionally, the user is unable to fully interact with the online representation of the product and, therefore, cannot truly experience a full demonstration of the product while in a remote environment."

While "Guided Retail Experience" is now a granted patent, it was applied for 2021 alongside a very similar "Guided Consumer Experience" patent application which has now been granted too. It concerns the same issues, just with minutely different emphasis.

"Guided Retail Experience," for example, is concerned with how a Personal Shopper and products can be shown to the user, perhaps through an entire virtual Apple Store.

Detail from the patent showing an Apple Store Personal Shopper demonstrating in your home

On the other hand, the "Guided Consumer Experience" patent concentrates on what a user will see about a specific product in such an environment.

Parts of that granted patent are also similar to an Apple Vision Pro one where comparison shopping is improved by the device displaying key information next to the products.

A virtual Apple Store

The patent and the patent application are two sides of the same coin, and ultimately Apple is looking for all-round shopping improvements. It believes it can do this via augmented reality, and that you have enough money left over after buying Vision Pro to carry on shopping.

"This allows a user to remain in a remote environment while having an interactive shopping experience with the salesperson," says Apple, "who can provide relevant products, make suggestions based on the interaction, and facilitate full demonstrations of various products and services in the CGR environment."

"The salesperson can cause a product, or a collection of products, to be displayed in the CGR environment," continues Apple, "and the user can interact with the products to perform a demonstration of the product with or without input from the salesperson."

This all provides "the user with a retail experience that is in a remote environment such as the user's house, but with the benefits and capabilities of an in-person shopping experience."

Detail from the patent application showing the Apple Store app with an in-video Personal Shopper

Detail from the patent showing the Apple Store app with an in-video Personal Shopper

Across the two related patents, Apple proposes a range of interactions. The salesperson could be in a chatbox-like window, just with video instead of text. Or the salesperson could even be shown, through AR, as being in the user's room.

The consumer-focused patent is credited to Jonathan S. Reiling and Alexander B. Oser. The latter is the sole inventor on the patent regarding retail guideance, and his previous related work includes the use of AR to make TV sports more interactive.

Latest Exclusives

Latest comparisons

7 Comments

Francules 6 Years · 122 comments

About 3 years ago

I’ve been waiting my whole life for the right pair of spectacles. Then i realized, i had them on the whole time.

mayfly 1 Year · 385 comments

About 1 year ago

I hope Apple isn't chasing rainbows and unicorns in the AR/VR sector. From what I've seen, especially the failures of Meta and Google's offerings, the entire tech industry may have overestimated the demand for these products and services.

genovelle 16 Years · 1481 comments

mayfly said:

I hope Apple isn't chasing rainbows and unicorns in the AR/VR sector. From what I've seen, especially the failures of Meta and Google's offerings, the entire tech industry may have overestimated the demand for these products and services.

I believe it’s the end experience that will make the difference. If they figure out how to stream a real Apple employee and have a 3d image of them standing in your living room using spacial vision they put on the new iPhone, they will certainly succeed.

Marvin 18 Years · 15355 comments

mayfly said:

I hope Apple isn't chasing rainbows and unicorns in the AR/VR sector. From what I've seen, especially the failures of Meta and Google's offerings, the entire tech industry may have overestimated the demand for these products and services.

From the unit sales, it seems there's interest in the concept of AR/VR:

https://www.uploadvr.com/meta-sold-20-million-quests/

Most people would love to have a Matrix-like or Holodeck-like experience but so far, nothing is delivering this. People who bought the hardware aren't using it much. A lot of the time they are bought as novelty Christmas gifts as it's a gadget that people don't already have but there's not much software for them and they are difficult for average users to setup.

A lot of things right now are tech demos. They showed off photorealistic avatars recently - digital Zuckerberg still looks as unnatural as the real one:

https://www.youtube.com/watch?v=MVYrJJNdrEg

There's a new technology for capturing and rendering 3D environments quickly called Gaussian Splatting:

https://www.youtube.com/watch?v=mD0oBE9LJTQ

Combined with AI voice and text generators, this could be used to recreate digital versions of people that can 'live' forever. Some people never get over a long-term partner, this would let them exist virtually at different ages.

The hardware design, price and lack of software is holding it back but spatial computing is the next computing paradigm that will eventually be used frequently in some form. It can't replace traditional computing for a while, in the near-term it will offer impressive short-term experiences. Once people experience watching a movie on a 3D virtual display the size of a room or stepping into a virtual memory, traditional digital media will feel like newspapers and magazines today.

I think Apple's strategy of focusing on everyday things will improve frequent usage. Once people experience movies this way, they will want to watch all movies this way so it's less likely to end up in a drawer but the entry price is a big barrier for adoption just now.

It's clearly going to be an industry that gains traction slowly but it's one that will never go away, this is the future of interacting with digital content, it's just uncomfortable with the technology we have right now.

genovelle said:

mayfly said:

I hope Apple isn't chasing rainbows and unicorns in the AR/VR sector. From what I've seen, especially the failures of Meta and Google's offerings, the entire tech industry may have overestimated the demand for these products and services.

I believe it’s the end experience that will make the difference. If they figure out how to stream a real Apple employee and have a 3d image of them standing in your living room using spacial vision they put on the new iPhone, they will certainly succeed.

LOL! They could stream Jimi Hendrix in a 3D live concert in my living room, and dedicating it to me and I still wouldn't spend $3,500 for one. $999, mmmmmmaybe…