Apple patent paves way for iPhone with full-face display, HUD windows

Apple on Tuesday was granted a patent detailing technology that allows for ear speakers, cameras and even a heads-up display to hide behind an edge-to-edge screen, a design rumored to debut in a next-generation iPhone later this year.

Awarded by the U.S. Patent and Trademark Office, Apple's U.S. Patent No. 9,543,364 for "Electronic devices having displays with openings" describes a method by which various components can be mounted behind perforations in a device screen that are so small as to be imperceptible to the human eye. This arrangement would allow engineers to design a smartphone or tablet with a true edge-to-edge, or "full face," display.

Until recently, smartphone aesthetics have been both defined and limited by their most dominant feature: the display. Conventional candybar designs made popular by iPhone put the screen front and center, with supporting hardware arranged along its periphery.

For example, iPhone features a large central display flanked by sensors and physical controls located on the device's "forehead" and "chin," or the areas above and below the screen, and its sides. Certain components like the front-facing camera, ambient light sensor and proximity sensor can be hidden below a cover glass or co-located in an inactive portion of the display. Other elements like the ear speaker and home button require unhindered access to the user, especially in the case of Apple's Touch ID, meaning space for these parts is literally cut out of the front surface.

With smartphones becoming increasingly more compact, there has been a push to move essential components behind the active — or light-emitting — area of incorporated displays.

Apple in its patent suggests mounting sensors and other equipment behind a series of openings, or through-holes, in the active portion of an OLED or similar panel. These openings might be left empty or, if desired, filled with glass, polymers, radio-transparent ceramic or other suitable material.

Positioning sensor inputs directly in line with said openings facilitates the gathering of light, radio waves and acoustic signals. Microphones, cameras, antennas, light sensors and other equipment would therefore have unimpeded access beyond the display layer. The design also accommodates larger structures like iPhone's home button.

According to the document, openings are formed between pixels, suggesting a self-illuminating display technology like OLED is preferred over traditional LCD structures that require backlight and filter layers. Hole groupings can be arranged in various shapes depending on the application, and might be larger or smaller than the underlying component. For example, an opening for a camera might be circular and only as large as the objective lens, while the sensor module is an order of magnitude larger.

Hole arrays (left) and spacing in relation to display pixels.

Alternatively, the openings themselves are of varying shape and size, allowing for the mounting of light-emitting modules like secondary displays.

Interestingly, Apple notes the patented technology might power a built-in heads-up display system. In some embodiments, windows are created by providing one or more transparent regions in the front and rear of the device. When a user looks through these windows, formed from a glass panel on the rear and the main transparent OLED display, they would see digital images overlaid atop real-world objects.

If implemented into a future iPhone, the window-based HUD could be Apple's first foray into augmented reality. Apple leaves the mechanics unmentioned, but the system could theoretically go beyond AR and into mixed reality applications.

For example, a rear-facing camera might gather scene information, which is then processed using local or cloud-based computer vision algorithms, similar to those employed by iPhone 7 Plus' Portrait mode. A digital image containing useful information about a user's surroundings might then be projected — in alignment with real-world objects — onto the sub-display window.

Whether Apple plans to implement the advanced display technology into a future iPhone is unclear, though the company is rumored to launch a special edition version of the smartphone with full-face display later this year. As for the HUD portion of Apple's patent, the company is said to be mulling an entry into AR within the next one to two years, with initial solutions potentially integrated into iPhone.

Apple's patent for a device with display openings was first filed for in February 2015 and credits Benjamin M. Rappoport, Jeremy C. Franklin, Fletcher R. Rothkopf, Scott A. Myers, Richard P. Howarth, Julian Hoenig, Christopher J. Stringer, John P. Ternus and Stephen Brian Lynch as its inventors.