Apple crafting 'new application paradigm' for AR and VR experiences

Apple is recruiting software engineers for what it describes as helping to create "next-generation interactive computing platforms," using Apple AR and VR.

As expectations grow that Apple will soon announce an AR/VR headset — and that it will be a "game changer" — the company is looking further into the future. Apple's jobs site has included a new AR/VR role, and the company has taken the unusual step of also calling out for recruits through Twitter.

My team at Apple is hiring AR/VR framework engineers with a focus on multi-user systems.

If you want to make a big impact on the next generation of computing platforms please reach out.https://t.co/YZNLbGkRtb
— Hayden Lee (@HaydenLee37) December 6, 2021

"You will be researching and developing an entirely new application paradigm," says the full job posting, "- a challenge that will demand rapid experimentation and prototyping without sacrificing code quality or attention to detail."

This is one role, based in Cupertino, rather than Apple recruiting multiple people. The listing is also one of 65 current job openings in Apple's AR/VR division.

The tweet, and the new role, come as Hayden Lee became Apple's AR/VR Software Engineer Manager in October 2021. Prior to that, he spent around 14 months as a software engineer in the company, and previously co-founded two virtual reality firms, Bigscreen VR and Convrge.

Separately, analyst Ming-Chi Kuo has now reported that Apple's first AR headset will weigh just less than a pound.

8 Comments

swat671 155 comments · 9 Years

About 2 years ago

One thing I'm still interested to see is some sort of Apple version of Microsofts' Photosynth from about 15 years ago. It allowed you to take photos of an area, and it would processes them into a 3D model in the computer using photogrammetry. With a powerful smartphone that happens to have LIDAR, it should be easy to do something similar now.

badmonk 1328 comments · 11 Years

The paradigm construct here is going to be more important than the details IMHO, since public resistance to devices like this is huge (outside of gamers and pervs). Note the tepid and hostile reactions to Google Glass and FB’s Meta concepts).

Marvin 15330 comments · 18 Years

swat671 said:

One thing I'm still interested to see is some sort of Apple version of Microsofts' Photosynth from about 15 years ago. It allowed you to take photos of an area, and it would processes them into a 3D model in the computer using photogrammetry. With a powerful smartphone that happens to have LIDAR, it should be easy to do something similar now.

There are 3rd party apps that can do this capturing, it's neat how it can drop the capture directly into AR for viewing next to the real object:

https://www.youtube.com/watch?v=1FQQovfNKeU

These apps could use some smoothing/filling techniques to clean up the models a bit. That can always be done manually after capture but using some machine learning, it could do it in real-time.
The tweet, and the new role, come as Hayden Lee became Apple's AR/VR Software Engineer Manager in October 2021. Prior to that, he spent around 14 months as a software engineer in the company, and previously co-founded two virtual reality firms, Bigscreen VR and Convrge.

Separately, analyst Ming-Chi Kuo has now reported that Apple's first AR headset will weigh just less than a pound.
Looking at the Convrge site, they have a few images of standard VR hardware:

https://www.convrge.co

The PSVR in the second image weights 1.3lbs. A few others are closer to 1lb so this is probably why that weight is mentioned by Kuo, it's an obvious assumption that it will be lighter than the lightest available VR headset.

I think this kind of hardware is way too bulky for an Apple product.

fastasleep 6450 comments · 14 Years

swat671 said:

One thing I'm still interested to see is some sort of Apple version of Microsofts' Photosynth from about 15 years ago. It allowed you to take photos of an area, and it would processes them into a 3D model in the computer using photogrammetry. With a powerful smartphone that happens to have LIDAR, it should be easy to do something similar now.

There are several 3D scanner apps that use the iPhone/iPad LiDAR sensors for years now. Have you even looked?

Also, did you miss Apple adding Object Capture in RealityKit at this year's WWDC?
https://developer.apple.com/videos/play/wwdc2021/10076/

Marvin said:
The PSVR in the second image weights 1.3lbs. A few others are closer to 1lb so this is probably why that weight is mentioned by Kuo, it's an obvious assumption that it will be lighter than the lightest available VR headset.

I think this kind of hardware is way too bulky for an Apple product.

PSVR is 5 years old now. PSVR2 should be out sometime next year (hopefully) and should be much more compact.