Apple's Cover Flow-like 3D desktop UI uses tilt controls to peek around windows

Source: USPTO

The U.S. Patent and Trademark Office on Tuesday awarded Apple U.S. Patent No. 8,560,960 for "Browsing and interacting with open windows," which at first blush appears to be a modified solution that brings Cover Flow to the desktop. Deeper in the document, however, is language describing a fresh take on usability that implements a secondary mode of input more akin to controls used in an iOS device.

According to the patent's detailed description, a user can tilt the computer to present the three-dimensional desktop "with a raised view level, such that desktop items behind the browsable parade are revealed." This action not only allows for a more immersive interactive experience, but increases viewable desktop space and aids in navigating the UI.

While the current OS X desktop is already somewhat "3D," with drop shadows and other graphical tricks providing an illusion of depth, Tuesday's invention further pushes these established boundaries.

A conventional desktop displays two-dimensional windows and icons on a substantially flat background. Users interact with these on-screen items via a given mode on input, such as clicking on a window to make it active. This means the visible area for presenting windows, icons, and other UI elements is constrained by the size of a device's screen.

As noted in the patent, 3D desktops offer more perceived space with the introduction of a z-axis, but manipulating graphical assets within these environments has proven too complex for previously proposed systems. Apple itself experimented with such techniques in the past, though the technology has yet to be applied to OS X.

To work around the hurdles, the invention proposes an operating system that can seamlessly switch between two-dimensional and three-dimensional environments. In two-dimensions, a user can invoke the familiar expose mode to see all open windows in a tiled view. Alternatively, in a three-dimensional environment, a user can invoke a Cover Flow or "parade" viewing mode, browsable by flipping through active windows.

When in 3D mode, the individual application windows act much like their current OS X counterparts. For example, when a window is made active, that application's menu bar is provided directly below the system menu bar. Further, when new windows are opened, the UI places the assets in a stack that can be reshuffled, resized or moved by the user.

Aside from the Cover Flow capabilities, Apple's invention uses two inputs to control the window parade. This is accomplished with conventional computer inputs, like a mouse and keyboard, working in concert with tilt controls. While not specifically explained in the patent, tilting of a device can be recognized by an accelerometer, gyroscope, camera or any other suitable on-board sensor. Alternatively, a tilt command can be invoked via a special key combination or gesture.

By tilting the three-dimensional desktop space, users can view an entire group of open windows in the parade from a higher view point. Content previously obscured by an active top window is made viewable, allowing users to select another window without cycling through all open assets.

It is unclear if Apple will implement the Cover Flow window invention in a future version of OS X, but a similar technique uses iOS device sensors to create the "parallax effect" seen in iOS 7.

Apple's three-dimensional GUI patent was first filed for in 2010 and credits Thomas Goossens and Fabrice Robinet as its inventors.