Apple has reportedly approached Sony to partner on Apple Vision Pro controllers

AppleInsider said:

A new report claims Apple has approached Sony in an effort to bring handheld game controller support -- and presumably more games -- to the Apple Vision Pro. 

VR hand controller support could revitalize the software market for the Apple Vision Pro.

Top 10 VR games on Playstation reported by Sony for 2023 are on the following page:

https://www.roadtovr.com/sony-reveals-the-10-most-downloaded-psvr-2-games-in-2023/

These all probably work better with controllers, many simulate holding weapons or objects. The PSVR2 controllers are a bit bulky though:



There are more compact ones that do full finger tracking:

https://eteexr.com/



Apple could design their own compact controllers like these with full tracking and touch controls. It would be good if they could have wristbands that could track interaction with any passive objects. Then people could hold plastic replicas of any object.

Since games are already made to support the Sony controllers and millions of people already own them, it would help to have support for them too.

There are controllers here designed for AVP that have a nicer design, Apple could probably buy this company out or just sell their products in store:

https://www.surreal-interactive.com/
https://www.kickstarter.com/projects/surreal-interactive/surreal-touch-1-controllers-for-apple-vision-pro



https://www.youtube.com/watch?v=g9I4SvvyBrY

If Apple partners with Valve for SteamVR, AVP will be compatible with thousands of games, initially via streaming and then with native ports.

One very important thing hand VR controllers offer over controller-free hand gestures is haptic feedback.

For gaming this can be pretty important depending on the title. Even if you are grabbing something, the game developer can program some sort of feedback.

There are other controller features like gyroscopes, adaptive triggers and more. These are some of the same things that people playing with mouse & keyboard miss out on. There's really no way to implement an adaptive trigger for a standard keyboard and mouse combination.

Gyroscopic input is also extremely useful for certain situations. Many driving (and some flight) simulation games in VR utilize a gamepad as the handheld input device that essentially functions as a steering wheel. If you are just using hand gestures, you don't have a credible input experience just by grabbing some image projected in air.

While controller-less gesturing might be very freeing in some VR situations, often there's a benefit to having some sort of haptic feedback especially if you are simulating grabbing some object with your hand.

AVP has a looooooooong way to go before it escapes being a niche device. Even the controllers for my old Oculus Rift S VR HMD have vibration motors (just like a gamepad).

Wow can you imagine the PS 6 comes out and the psvr3 is actually an Apple headset 🤯. The main gripe I hear about the psvr2 controllers is that is was made for smaller hands, but the adaptive triggers are great so if Apple and Sony create a new controller for both the VP and PSVR it would possibly be the new industry standard (I have a very positive outlook in the future for unlikely things)

Hopefully, the deal primarily involves Sony porting PS5 games to Vision Pro and Mac.  :D

This looks like the slow incremental addition of Vision features Apple has been doing for the past year, filling out the feature set for visionOS. A nice to have, another reason to use the AVP, but Apple will have to subsidize games for awhile.

A very long road to go. They still need to add:

1. iPhone Virtual Display or iPhone Mirroring for visionOS
2. Xcode, Terminal, and the XQuartz equivalent.
3. Update the hardware. They really should update the AVP to M4, 16/32 GB, 0.5/1/2 TB, 12 MP main view cameras, WiFi 7, etc.
4. Continue with Continuity features as much as possible. 
5. Subsidize content.

tht said:

A very long road to go. They still need to add:

1. iPhone Virtual Display or iPhone Mirroring for visionOS
2. Xcode, Terminal, and the XQuartz equivalent.
3. Update the hardware. They really should update the AVP to M4, 16/32 GB, 0.5/1/2 TB, 12 MP main view cameras, WiFi 7, etc.
4. Continue with Continuity features as much as possible. 
5. Subsidize content.

Good analysis.

1. This first item is incredibly likely to arrive within the next few months.

2. I would guess Terminal is not coming soon, but Xcode/XQuartz should be.

3. I think that’s going to be announced at WWDC. On the outside not very different-looking, but with all or most of what you describe.
4. Amen
5. This is an area that is sorta new for Apple in terms of giving other big companies bribe money for development, but I concur that if they want new content fast, that needs to happen.