Apple, Inc. gets its fingerprints on advanced touch sensor, appears difficult for Android to copy

Key to that strategy, it appears, is an advanced ultra-high resolution touchscreen technology it has developed over the last two years, shrouded in great secrecy.

The iPhone's defining multitouch touchscreen

When Apple first introduced the original iPhone in 2007, a key selling point was its capacitive multitouch screen. Steve Jobs contrasted the iPhone's full screen display against various models of then-popular button phones, inciting complaints from pundits and industry leaders who insisted that everyone really needed and wanted a physical keyboard.

Six years later, there are very few smartphones that even incorporate a physical keyboard, and virtually all of those that do also have a touchscreen. It may be hard to believe now, but in 2007 Microsoft's definition of a "Windows Mobile Smartphone" meant that it didn't have a touchscreen.

Sales of such phones, including the Samsung Blackjack II and Motorola Q (below), were obliterated when the multitouch iPhone appeared.

The new iPhone didn't just have a big touchscreen, it also used a capacitive rather than resistive touchscreen. This allowed it to support multitouch gestures like pinching (which Android 1.x phones of the next couple years would not), as well as being much more accurate and sensitive enough to recognize a gentle flick-to-scroll gesture or a very rapid double tap (most other touchscreen smartphones, like the Palm Treo, required a clumsy stylus or a slow, firm press).

A Capacitive touchscreen senses the electrons emitted by your living tissue. When you touch the screen, you are charging up tiny capacitors with your finger's electrons; as you move your finger away they discharge, providing an electronic trail of where you've touched. This kind of screen generally won't work with thick gloves on with a plastic stylus because there's nothing electrically-live for it to sense.

A Resistive touchscreen measures physical pressure. As you press against it with a finger or a stylus, you're changing resistance in the screen by pushing it inward. This is generally much less sensitive, so it requires a more deliberate push. Some, like supermarket checkout screens (or 2007-era phones), force you to pick up the stylus to press in on the screen firmly to make a selection.

While more expensive, capacitive screens gave the original iPhone a high quality feel and enabled Apple's Cocoa Touch environment to present an entirely new way for users to interact with mobile devices, incorporating subtle gestures that had only previously been possible on expensive business presentation devices, like those pioneered by Jeff Han and publicly demonstrated just a year prior to the iPhone making it an affordable mainstream technology.

Upping the ante to keep iPhone fresh and differentiated

Every year since, Apple has incorporated three kinds of features on its iPhone (and related mobile devices). The first are "catch up" features that incorporate the existing functionality of competitors. In its second year, for example, the iPhone got push messaging like BlackBerry, a mobile App Store like Danger, 3G and GPS support like Nokia and enterprise features like Microsoft.

Most recently, iPhone 5 got support for 4G LTE, a feature that had been an exclusive held by Android devices for two years. Every catch up feature Apple adds denies competitors of being able to claim an edge over the iPhone.

A second type of new feature Apple adds every year is a pacing of the state of the art. Every annual iPhone update has included a much faster Application Processor (like last year's A6, above), a more advanced Baseband Processor capable of faster data uploads and downloads and greatly improved camera sensors and optics. Everyone else in the industry is also matching these kinds of advances, making it hard to find ways to stand out. Everyone in the industry is desperately seeking new exclusives to stand out as new and fresh.

A third type of feature that Apple adds every year are new exclusives intended to keep the iPhone uniquely differentiated and exciting. Examples include the gyroscope, Retina Display and FaceTime on iPhone 4; Siri and AirPlay Mirroring on iPhone 4S and the precision design and advanced battery technology of iPhone 5.

Everyone in the industry is desperately seeking new exclusives to stand out as new and fresh. Apple needs these not just for differentiation with competing platforms, but also to encourage existing users to upgrade and entice new customers to buy a premium iPhone model.

A super sensitive touch screen for your fingertips

It's widely anticipated that one new feature Apple will be incorporating in its next high end model is a component that serves as an ultra precision, capacitive touchscreen. But rather than being part of the display, it appears it will be built into the Home button. And rather than being used to track your finger's movements, it will be used to uniquely verify your fingerprint.

Early last year, Apple began working with AuthenTec, a company that built advanced fingerprint scanners in addition to selling encryption-related embedded devices and encryption software for VPNs and secure video delivery.

While several companies, including HP, Dell, Lenovo and Fujitsu, had been incorporating AuthenTec fingerprint sensors in their PC notebooks (primarily as a secure shortcut for logging into Windows), only Fujitsu was using one in a smartphone.

Back in early 2011, Motorola Mobility had debuted its new Atrix 4G smartphone with an AuthenTec fingerprint sensor (at the top, on the back of the phone, visible above), but it had trouble getting it to work reliably.

By the end of the year, users were complaining that the feature had stopped working and that Motorola wasn't doing anything to support it. The company hasn't included fingerprint scanners on its subsequent products.

AuthenTec's new fingerprint sensor fails to find interest, apart from Apple

Nearly one year after Motorola began its now-abandoned experiment with smartphone fingerprint scanners, an SEC filing outlined that AuthenTec had developed a new technology and began shopping it around to "several leading consumer electronics companies." AuthenTec stated that the main reason Apple was the only interested party appears to have been the cost involved

None of the companies AuthenTec approached expressed an interest in developing the technology apart from Apple, which began a conversation with the firm in late 2011. AuthenTec stated that the main reason Apple was the only interested party appears to have been the cost involved.

Initial licensing and development talks in February 2012 shifted into acquisition deal in May, which fell apart only to be revived in July, when Apple moved to quickly initiate a full acquisition for $365 million while paying an additional $27 million for the "right to acquire non-exclusive licenses and certain other rights with respect to hardware technology, software technology and patents of the Company" including "commercialization of 2D fingerprint sensors for use in or with Apple products."

Fingerprint sensors inherently difficult and problematic

A report by an expert in the field of photographic fingerprinting seems to explain why nobody else but Apple expressed any serious interest in AuthenTec last year, including Motorola, which had been burned the year before.

Geppy Parziale is a researcher who cites more than 15 years of experience in pattern recognition and machine vision and who developed patented technology for full hand fingerprint capture with funding from the U.S. Federal Bureau of Investigation, Department of Defense and Department of Homeland Security.

In a blog entry for developer InvasiveCode, Parziale described the concepts behind biometric fingerprinting, contrasting the technology he developed for "touchless" photographic capture using "a camera with very sophisticated optical lenses and a complex lighting system" to the "touch" sensors sold by AuthenTec."Constant usage of the sensor starts to destroy the capacitors and over time, the fingerprint sensor stops to work" - Geppy Parziale

A primary engineering challenge for devices like AuthenTec's, Parziale describes, is that "constant usage of the sensor starts to destroy the capacitors and over time, the fingerprint sensor stops to work." While a protective layer can be added to prevent overloading the sensor's capacitors to prolong their lifespan, "the coating layer on the fingerprint sensor surface cannot be too thick otherwise the electrons from your body cannot reach the metallic surface of the sensor to generate a fingerprint image."

Complicating this issue, Parziale wrote, are real-world contaminants, including oils and salts naturally occurring on everyone's skin, that, together with heat and humidity, create "a killer combination for the sensor surface that speeds up the destruction of its surface."

He states that "fingerprint sensor manufacturers (including AuthenTec) never achieved great success in this issue," adding, "Unfortunately there is no existing solution to this. Manufacturers can only try to make the fingerprint sensor last longer, but sooner or later that device will stop working properly."

He concludes that this will greatly limit the potential usefulness of any fingerprint sensor on an iPhone, and prevent Apple from using the sensor for payments due to the risk of false acceptance, "when after a while [of use wears the sensor out] somebody else can be granted access to your device."

This all happened before

Six years ago, similarly dire predictions were made about the touchscreen on the original iPhone by Nomura analyst Richard Windsor.

In a report published by Eric Savitz of Barrons, a note by Windsor was cited as saying that the "iPhone screen uses a chemical deposition process to provide touch sensitivity based on heat."

Windsor wrote that rights to the technology had been acquired from "a bankrupt Finnish company" whose earlier products “encountered the problem that with extensive use, the film would begin to degrade and the screen would lose its sensitivity.” Windsor predicted iPhone screens would begin failing after three to six months, resulting in a massive, expensive recall.

That never happened, but more importantly the iPhone also never used a heat sensitive film. The next year, Windsor similarly issued a report saying that the second generation iPhone 3G had been equipped with faulty Infineon baseband chips and suggested that Apple might have to recall that product as well.

Analysts similarly reported that the new antenna design for iPhone 4 was so flawed that it wouldn't sell and would need to be recalled, but it again became the top selling smartphone model and Apple settled the media frenzy surrounding the issue with a free bumper offer.

Parziale's observations certainly seem more informed, but they also don't take into account the possibility that AuthenTec had developed a suitable solution to the issues he described.

The Smart Sensor: why Apple paid $356 million for AuthenTec

Apple paid far more for AuthenTech than it typically spends on acquisitions. There's only two known companies Apple has paid more for: $390 million for flash memory developer Anobit, and $400 million for NeXT, which formed the basis of Apple's 1997 turnaround, OS X and iOS.

Apple isn't known for paying astronomical billions for acquisitions that in retrospect, make little sense. While Microsoft paid $8.5 billion for Skype and Google paid $12.5 billion for Motorola, Apple built its own FaceTime technology in-house and has developed its own mobile hardware business.

It's also clear Apple bought AuthenTec specifically for its fingerprint sensors. After closing the deal, Apple sold off the company's encryption technology.

AuthenTec advertised at the time that its "semiconductor-based sensors are based on both capacitive and radio frequency (RF) technology that detects an image of the fingerprint ridge and valley pattern beneath the surface of the skin, thus capturing sharp and clear fingerprint patterns from the live layer.

"This approach, which is in marked contrast with optical, thermal and other solutions that simply read the surface of the skin, gives AuthenTec sensors significant advantages in image quality and in the protective coatings that the sensor can image through."

The company's new AES850 sensor, which it began selling to Fujitsu for use in a new smartphone that went on sale in late 2011, was called "the world’s smallest authentication/navigation smart sensor in a durable package."

AuthenTec's secret work for and within Apple

A contract between Apple and AuthenTec that described a secretive partnership that would go on as the acquisition progressed described "Apple Project Materials" that AuthenTec was forbidden to "disassemble, decompile, or otherwise reverse engineer," indicating that Apple also had valuable and significant work of its own in progress.

Earlier this year, Apple filed a patent titled "Integrally molded die and bezel structure for fingerprint sensors and the like," describing a "unitary encapsulation structure" designed to protect a "delicate biometric array," while still allowing for a portion of the sensor and bezel to be exposed or thinly coated with a protective seal.

"By encasing both the bezel and the sensor die in the encapsulation structure," the patent explained, "those elements may be brought closer together than heretofore possible. In addition, the encapsulation structure physically protects the bezel and sensor die, and in particular maintains the spacing there between, in a fashion not possible by currently known device designs."

Apple gets exclusive with AuthenTec

Last September, after Apple's acquisition closed, AuthenTec notified existing customers, including Samsung, HP, Dell, Lenovo and Fujitsu, that it would no longer be fulfilling their orders for parts in 2013.

Biometric Update cited reports last year describing those customers as being "in a state of panic" as they tried to source alternative suppliers.

Parziale's description of the difficulty of developing functional, reliable fingerprint sensors is clearly true, evidenced by previous failures like the Motorola Atrix and the fact that Samsung was reportedly working with Validity Sensors to add fingerprint scanning to its upcoming Galaxy Note 3 "phablet" expected this fall, but apparently had to pull the feature after running into production problems.

Similar issues were rumored to affect Apple as well, but the company has had more time to work on the issues, and is now the owner of the company that described itself as "the world's #1 provider of fingerprint sensors."

A tough act to follow on a crowded stage

This indicates that "an advanced capacitive touchscreen" may once again give Apple an advanced edge with a valuable feature competitors will have trouble matching for years.

In contrast, Samsung's Galaxy S 4 flagship smartphone appeared with novelty gimmicks that analyst Gene Munster of Piper Jaffray characterized as an "evolutionary" update, while Brian White of Topeka Capital Markets said it was "not a game changer" and simply "heavier, fatter and less refined than the iPhone 5."

The Galaxy S 4 was estimated to have sold less than half as fast as Apple's iPhone 5 launch.

Google's announcement of its forthcoming Moto X was similarly greeted by Munster as offering nothing special, writing in a note that the new handset isn't "significantly different than [the Galaxy S4 and HTC One] to change the current market dynamics between Android and iPhone."

Apple already has a series of distinct features, particularly its leading ecosystem and a lock on important markets such as the enterprise. It if can pull off a functional fingerprint scanner, it stands to increase its lead by standing out in an otherwise poorly differentiated market.