Apple acquires "dark data" specialist Lattice Data for $200M

The buy was reported by Tech Crunch, and Apple responded with its usual boilerplate confirmation that it "buys smaller technology companies from time to time and we generally do not discuss our purpose or plans."

Lattice, based in Menlo Park, California, was commercializing a Stanford University research project known as "DeepDive," acting as a framework for statistical inference. The firm's website says "our mission is to unlock the value in dark data for critical real-world problems."

"Dark Data" pertains to the mountains of raw information collected by in various ways (such as logs or photos) but which remains difficult to analyze.

Lattice built a platform of proprietary software on top of Stanford's open source DeepDive technology, which had been developed over a six year period with $20 million of DARPA funding, aimed at rapidly make sense out of unstructured data.

DeepDive is aimed at "extracting value" from various "dark data" sources, serving, as the firm's site explains, as a "programming and execution framework for statistical inference, which allows us to solve data cleaning, extraction, and integration problems jointly. We model the known as features and the unknown as random variables connected in a factor graph."

Lattice chief scientist Christopher R won a MacArthur Genius Grantfor his work on DeepDive, and cofounded the firm with Michael Cafarella (who also co-founded Hadoop), Raphael Hoffmann andFeng Niu.

The company contrasts its approach to traditional machine learning, explaining "we do not require laborious manual annotations. Rather, we take advantage of domain knowledge and existing structured data to bootstrap learning via distant supervision. We solve data problems with data."

The firm also emphasizes the "machine scale" of its platform in being able to "push the envelope on machine learning speed and scale," resulting in "systems and applications that involve billions of webpages, thousands of machines, and terabytes of data" at what it describes as "human-level quality."

While Apple doesn't usually comment on why it buys up talent and technology, Lattice was reportedly shopping itself around as a solution to enhancing voice-based assistance, and had been in talks with Amazon's Alexa team as well as Samsung.

The applications that Lattice outlines on its website also suggest potential use in analyzing data for use in Maps and self driving vehicles; HealthKit and ResearchKit; camera logic and processing as well as Internet data document search.