Apple's FoundationDB takes new Record Layer open source, confirms tech underpins CloudKit

Apple database arm FoundationDB on Monday announced the open source release of FoundationDB Record Layer, a relational database management technology that powers CloudKit.

Explained in a post to the official FoundationDB blog, FoundationDB Record Layer provides schema management, indexing facilities and query capabilities to the eponymous distributed datastore.

Beyond features commonly found in a relational database, Record Layer offers support for complex nested data types, indexes on the commit-time of records and indexes and queries that span different types of records, FoudationDB says. As a layer built on FoundationDB, the open source release incorporates ACID semantics and transactional semantics, the latter of which allows for a massive distributed datastore.

Record Layers is stateless, which allows for highly scalable implementations and speedy functionality. According to FoundationDB, database instantiation and subsequent operation can be performed in "milliseconds."

Apple acquired FoundationDB in 2015, taking the scalable NoSQL, ACID-compliant architecture in-house for previously undisclosed purposes. Reports at the time suggested the database software would be used as a basis for Apple's various cloud services, including iCloud and iTunes content management.

Today's announcement confirms FoundationDB and Record Layer act as a foundation for CloudKit. An accompanying white paper details Apple's integration, as well as additional nuances of Record Layer's underlying technology.

The open sourcing of Record Layer represents the second such release from Apple's distributed database subsidiary. In 2018, Apple released FoundationDB as an open source project, hoping to build a community around the scalable system.

8 Comments

applesauce007 1703 comments · 17 Years

About 5 years ago

Wow... This is awesome. Gotta check it out.
The C APIs for FDB were very low level, this layer should provide higher level APIs and hopefully seamless CoreData integration.
Keep the layers coming. SQL?
This DB has the potential to put some established DBs to shame in terms of scalability, performance, redundancy and ease of use.

Go Apple!

dick applebaum 12525 comments · 17 Years

By Codd, this is MegaBig...

From the Wait Paper:

The Record Layer is used by multiple systems at Apple. We demonstrate the power of the Record Layer at scale by de- scribing how CloudKit, Apple’s cloud backend service, uses it to provide strongly-consistent data storage for a large and diverse set of applications [43]. Using the Record Layer’s abstractions, CloudKit offers multi-tenancy at the extreme by maintaining independent record stores for each user of each application. As a result, we use the Record Layer on FoundationDB to host billions of independent databases shar- ing thousands of schemas. In the future, we envision that the Record Layer will be combined with other storage mod- els, such as queues and graphs, leveraging FoundationDB as a general purpose storage engine and remaining transac- tionally consistent across all these models. In summary, this work makes the following contributions:
An open source layer on top of FoundationDB with se- mantics akin to those of a relational database.
The record store abstraction and a suite of techniques to manipulate it, enabling billions of logical tenants to operate independent databases in a FoundationDB cluster.
A highly extensible architecture, clients can customize core features including schema management and indexing.
A lightweight design that provides rich features on top of
the underlying key-value store.

netmage 314 comments · 14 Years

Instead of SQL how about a LINQ driver?

LordeHawk 168 comments · 7 Years

Tantalizing read...

Interesting to note that Apple is building rich layers for abstraction. Got me thinking about a relational system for AI, AR and Siri.
Never mind, I know what they’re doing with it now.