Mega Apple filing details next-gen 'multi-touch input surface'

The latest patent filing from Apple Inc. hints at a next-generation multi-touch 'surface' that would combine typing, pointing, scrolling, and handwriting capabilities into a single ergonomic design aimed at replacing traditional input devices such as the keyboard, mouse, and drawing tablet.

Like last week's filing for an advanced multi-touch trackpad, the concepts outlined in the massive 80-page document are partially credited to Wayne Westerman of Fingerworks, a company absorbed by Apple several years ago as part of its quest to deliver iPhone and a new generation of input devices.

The filing, submitted four times on July 30, 2007 with varying title descriptions, calls for a generic device that tightly integrates yet clearly distinguishes the different types of input without providing excess strain or muscle movement.

"It should therefore appear modeless to the user in the sense that the user should not need to provide explicit mode switch signals such as buttonpresses, arm relocations, or stylus pickups before switching from one input activity to another," Westerman wrote. "Epidemiological studies suggest that repetition and force multiply in causing repetitive strain injuries. Awkward postures, device activation force, wasted motion, and repetition should be minimized to improve ergonomics."

Therefore, the multi-touch creator explained that an ideal implementation of his concept calls for multi-touch surface apparatus which is both compliant and contoured to be comfortable and ergonomic under extended use. It would "provide tactile key or hand position feedback without impeding [a] hand resting on the surface or smooth, accurate sliding across the surface."

The surface would include an electronic system which could provide images of flesh proximity to an array of sensors with such resolution that a variety of hand configurations can be distinguished. This would allow the device to identify different hand parts as they contact the surface so that a variety of hand configurations can be recognized and used to distinguish different kinds of input activity, Westerman said.

Yet another objective of the multi-touch surface would be to reliably extract rotation and scaling, as well as translation degrees of freedom from the motion of two or more hand contacts to aid in navigation and manipulation of two-dimensional electronic documents. Furthermore, it would be capable of reliably extracting tilt and roll degrees of freedom from hand pressure differences to aid in navigation and manipulation of three-dimensional environments.

In its preferred embodiment, Westerman said the surface would be large enough to comfortably accommodate both hands and would be arched to reduce forearm pronation. Text input, pointing, scrolling and some media manipulation functions of the surface would function similarly to that of Apple's existing implementation on the iPhone and iPod touch. In addition, a form of handwriting recognition would be added via a "pen grip detection module."

While in pen grip mode, the module would determine whether the inner gripping fingers of a hand in a pen-gripping mode are actually touching the surface. If so, the module would begin to generate inking events from the path parameters of the inner fingers and append them to the outgoing event queue of the host communication interface.

"These inking events can either cause 'digital ink' to be laved on the display for drawing or signature capture purposes, or they can be intercepted by a handwriting recognition system and interpreted as gestures or language symbols," Westerman explained. "If the inner fingers are lifted, [the module] sends stylus raised events to the host communication interface to instruct the handwriting recognition system of a break between symbols."

"In some applications the user may need to indicate where the 'digital ink' or interpreted symbols are to be inserted on the display by positioning a cursor," Westerman continued. "Though on a multi-touch surface a user could move the cursor by leaving the pen grip configuration and sliding a finger chord, it is preferable to allow cursor positioning without leaving the pen grip configuration. This can be supported by generating cursor positioning events from slides of the palm heels and outer knuckles. Since normal writing motions will also include slides of the palm heels and outer knuckles, palm motions should be ignored until the inner fingers have been lifted for a few hundred milliseconds."

Should the user actually pick up a conductive stylus and attempt to write with it, the tip of the stylus would essentially takes the place of the index fingertip for identification purposes, remaining at or above the vertical level of the knuckles, according to the filing. "Thus the pen grip detector can function in essentially the same way when the user writes with a stylus, except that the index fingertip path sent to the host communication interface will in actuality be caused by the stylus."

In addition to Westerman, the filing is credited to Apple employee John Elias. The concepts presented within the filing are eerily indicative of functions that would be present in a next-generation Apple Newton MessagePad or tablet slate.

Indeed, AppleInsider this past September exclusively reported on Apple's plans for such a device in a report titled: Up next for Apple: the return of the Newton. The project, upon last check, remains very much work-in-progress, as it has been met with the usual assortment of bumps and bruises.