A patent application published by the U.S. Patent and Trademark Office on Thursday reveals Apple is actively researching methods to ensure a video's multi-channel audio is always played back correctly, no matter how a user holds the device.
Source: USPTO
Apple's patent filing for "Orientation-based audio" describes a system that leverages a mobile device's internal sensors, including accelerometers, gyroscope and cameras, among others, to synchronize audio output from multiple speakers.
According to one embodiment, the orientation of video being displayed on a device is used to determine the relative position of speakers to the left and right of the screen. Stereo audio is then routed to the correct speakers.
In another embodiment, a device is determined to be rotating based on various sensor data. The information is used to dynamically switch, or route, the audio to the appropriate speakers. For example, if an iPad mini user moves from viewing a movie with the tablet positioned home-button-down to home-button-up, the right and left audio channels would be switched to compensate for the 180-degree rotation.
The invention can be applied to devices with more than two speakers, and examples include implementations with three or more outputs.
Powering the system is an audio processing router that first matches audio channels of the source to the device's speaker configuration.
Typically, although not necessarily, the audio processing router attempts to ensure that left and right channel audio outputs (whether front or back) are sent to speakers on the left and right sides of the device, respectively, given the device's current orientation. Thus, front and rear left channel audio may be mixed and sent to the left speaker(s) while the front and rear right channel audio may be mixed and sent to the right speaker(s).
Alongside speaker mapping, the processor can mix what would normally be center channel audio across a set of two speakers if a third is not present. Gain is also controlled to compensate for the "missing speaker," while in another embodiment the channel may be suppressed entirely.
If the device is not locked to a particular mode, like "portrait" or "landscape," the processor will actively reroute audio depending on what orientation the video is being displayed. An alternative mode of operation bases audio output on user settings rather than a predefined map.
When rotation or other movement is perceived, the processor creates a new map of the device and redirects audio accordingly. User customization settings can limit the switching to certain angles, such as a 90-degree or full 180-degree rotation.
Finally, some embodiments call for the application of equalization techniques to normalize audio according to a user's perception. For example, if a device is moved laterally away from a user, gain can be adjusted to compensate. Sensors, such as accelerometers, cameras and infrared presence detectors can be used to determine a user's movement.
In one example, if a portion of the user's body, such as the head, appears smaller, it is likely that they moved away from the device. In this situation, gain or volume would be increased to normalize audio.
Time delays can also be used strategically to syncing audio from one set of speakers with a second if they are not in line with a user's head.
While Apple's current iOS device lineup holds only one product with stereo loudspeakers, the iPad mini, the tech may be implemented in future iterations of existing models. Alternatively, the filing's assets can be applied to MacBook Pros or other hardware Apple has yet to introduce. The technology would be well-suited for a dedicated television set, a product long rumored to be in the offing.
Apple's orientation-based audio patent application was first filed for in 2011 and credits Martin E. Johnson, Ruchi Goel and Darby E. Hadley as its inventors.