Apple patents physical controller input method for multitouch devices

Apple's U.S. Patent No. 8,433,828 for "Accessory protocol for touch screen device accessibility" covers specialized techniques for controlling a device with touchscreen inputs using an non-touch hardware accessory.

From the patent background:

For a user to interact with a touch screen device using touch input, the user must generally be physically present at the device in order to touch the screen. The user must also be capable of providing touch input and capable of viewing the screen of the device or otherwise knowing where input is needed.

The system basically allows for peripherals like a joystick, button pad or other non-capacitive touch accessory to communicate with and control the UI of a multitouch input device, such as Apple's iPhone or iPad. Communication can be wired or wireless, the patent explains, with the input signals from the accessory being processed by the touch input host device.

Key to the invention is the accessibility framework, an example of which would be Apple's VoiceOver technology. In one embodiment, the accessibility framework receives "accessibility packets" which include commands and one or more input parameters. The host device then translates the input and generates an accessibility command and the one or more parameters. Host device can then send an output command and parameter back to the accessory, for example sound or Braille data.

As called for in the patent language, the accessories provide specific inputs such as joystick movement or button presses, which are then mapped to a database on board the touch device.

The interactions between the host and accessory devices can be standardized, allowing for compatibility with a variety of apps. By standardized, the patent refers to screen dimensions and user interface graphics.

Samples of output to an accessory include responses to accessibility packet input, such as sound. Incoming data not generated by the user, such as phone calls and text messages, can also be sent to the accessory device over the same communication channels as the accessibility packet.

Going further, an accessibility event command can be initiated through the accessibility framework, allowing for the direct manipulation of the touch device's UI and apps. Event examples include page scrolling, text input, text cut, text copy, text-to-voice and other accessibility functions. Also supported is a cursor that can be used to navigate the UI.

While such an accessory does not yet exist, Apple's VoiceOver and iOS Accessibility options are robust, suggesting the company is taking the field seriously.

Apple's accessory protocol patent was first filed for in 2010 and credits Christopher B. Fleizach, Paul Holden, Eric Taylor Seymour, Emily Clark Schubert, Lawrence Bolton and Sylvain Rene Louboutin as its inventors.