According to a new patent, Apple wants us to be able to dance, wave, gesture, or just enter and leave rooms, to have our smart devices like the HomePod do what we want.
I'm sorry, Dave, I'm afraid I can't do that.
Apple's HomePod already adjusts itself on setup to suit the acoustics of the room it's in. But now Apple seems to want that scanning and room awareness to be much more precise -- and always on.
According to a newly-granted patent called "Multi Media Computing Or Entertainment System For Responding To User Presence And Activity," Apple includes monitoring the activity of users in the room, too. And does it so that a device can do what a user wants, without that user having to necessarily even say it.
Apple's examples in the patent include a basic one, such as when the last person leaves the room. In that situation, the device could detect that the room is empty, and so stop playing music.
Or conceivably, if it's a large room with, say, a kitchen area at one end, then the HomePods there could know to pick up the music being played on other devices.
"[The] intelligent system acquires a depth image of a scene surrounding the system," says Apple. "A scene geometry may be extracted from the depth image and elements of the scene may be monitored."
"[Then] user activity in the scene is monitored and analyzed to infer user desires or intent with respect to the system," continues the patent.
In a similar way to how the HomePod senses its surroundings so it can alter acoustics, a future device may map out a room
Apple says that the "intelligent system" might recognize a user's wishes "expressed through hand gesture movements." For instance, just waving at a HomePod might tell it to change volume.
That would require that "such gesture movements [can] be interpreted based on real-time depth information obtained from, e.g., optical or non-optical type depth sensors," says Apple.
"When a system has an understanding of its users and the physical environment surrounding the user," continues the patent, "the system can better approximate and fulfill user desires, whether expressed literally or impliedly."
So a wave could change the volume, a military-style fist-up gesture might silence the music. In a professional setting such as delivering a presentation to an audience, Apple suggests a simple wave could move on to the next slide without the presenter relying on "inconsistent and imprecise" remote controls.
More than gesture recognition
But Apple isn't only interested in having the system follow set commands, it wants it to leverage all it knows so that it can produce the right result even when not specifically called for.
"For example," says Apple, "if [a] user is detected as older in age and that user expressly requests a higher volume, the system may decide that the user requires better differentiation of voice dialog in the system output."
Detail from the patent showing how a hand gesture might be identified.
"Therefore, the system may change the relative spectral distribution of the system output (to relatively amplify voice)," it continues, "rather than increase the average volume."
The patent is chiefly concerned with how a gesture, movement, or presence can be detected and interpreted, rather than necessarily on what the device then does with the information. Consequently, Apple doesn't say this, but such a system could detect a person dancing and provide appropriate music.
Note that while this patent has now been granted to Apple, the company applies for many hundreds of patents each year. Granted or not, there is no guarantee that a particular technology will result in product.