Apple's virtual input tool patent could bring next-level UI interactivity to OS X

Source: USPTO

Apple's U.S. Patent No. 8,427,438 for "Virtual input tools" is a simple, yet effective tweak to the current user interfaces offered in modern computer operating systems. Unlike a traditional UI, which translates data from an input device and maps it to the movements of a cursor, the patent outlines a system that allows for more direct interactivity.

The patent first calls for a virtual representation of a physical input device, such as the multitouch trackpad found in Apple's MacBook Pro, to be displayed onscreen. Mapped to the coordinates of the virtual trackpad are the corresponding coordinates of the actual device, which in some embodiments are in proportion to the digital version. Presented within the virtual trackpad's boundaries are various interactive elements that can be directly selected and manipulated without searching for a cursor as they are represented in direct relation to the physical device. According to the patent, interactive objects can be application windows, directories and open media files, among others.

From the patent summary:

A virtual representation of an input device can be a two-dimensional area that increases an amount of data (e.g., virtual representations of objects) that can be presented at a particular time, thereby improving the user's experience.

By implementing this method, the entirety of a trackpad's surface can be effectively utilized to navigate a corresponding virtual environment. As noted in the patent language, a user can also use multitouch gestures, like one-finger tap-and-hold, to further enhance the system.

Six engines drive the virtual system: an identification engine that detects input on the physical device; a render engine that draws the virtual interface tools and/or content; a mapping engine to correlate physical coordinates with virtual coordinates; a preference engine to customize the virtual device; an interactivity engine that processes user interactions and dictates how they are translated onscreen; and a presentation engine that creates interactive object representations on the virtual device.

Perhaps most important of the six, and the most helpful in explaining the patent, is the interactivity engine. In some embodiments, the engine will receive a signal that a user has touched or tapped a certain area of the trackpad. From this information, the area which was touched is mapped to the virtual trackpad, and the interactive object associated with that space is activated.

In another example, a multitouch gesture such as a swipe is recognized. The associated object on the virtual device is manipulated according to a predefined set of rules, which could be the rendering of an interface window in the quadrant selected. Multitouch gestures can also be used to control the size and operation of the virtual display. For example, a four-finger swipe can minimize the virtual view, while another input can enlarge the interface.

Finally, a control object can be utilized and integrated into the virtual trackpad to deactivate or exit out of the UI. This mechanism can also be used in conjunction with other areas of the virtual tool. For example, a user can manipulate interactive content without making permanent changes if their finger is in continuous contact with the control object. Once a user's finger is lifted off this area, interaction is canceled.

As with many Apple patents, it is unclear if the virtual input device will make its way to a consumer product, but with a push toward multitouch input, the technology could be a useful stopgap if the company decides to create a laptop with a touchscreen display.

Apple's virtual tool patent was first filed for in 2009 and credits John O. Louch as its inventor.