Apple greatly reduced autonomous vehicle testing in 2019, fleet of 23 cars drove only 7,500 miles

Apple's autonomous vehicle testing program saw a significant drawdown in 2019, with its fleet driving 72,201 miles less than it did in 2018. The company also appears to have changed its disengagement reporting protocol, information commonly used to gauge system efficacy.

According to numbers submitted by Apple to California's Department of Motor Vehicles, 23 of the company's 69 registered self-driving testbeds were active between December 2018 and November 2019. Collectively, the fleet of modified Lexus RX450h SUVs logged 7,544 miles during the reporting period, down from 79,754 miles in 2018.

Apple reported 64 disengagements across the 7,544 miles driven, which equates to 8.48 disengagements per 1,000 miles. This compares to a whopping 69,510 disengagements, or 871.65 disengagements per 1,000 miles, recorded in 2018. At the time, Apple's system was reported, on a disengagements-by-mile basis, as the worst-performing test platform in California. By contrast, then-front runner Waymo managed 0.09 disengagements per 1,000 miles over the same evaluation period.

While the improvement seems impressive, it should be noted that disengagement reporting is not a hard science. Typically, disengagements are defined as instances in which a self-driving vehicle's autonomous systems are unable to process current conditions, forcing it to pass control back to the human driver. Human interventions also count as disengagements.

As explained by Apple in a letter to California's DMV last year, the company in 2018 took a "conservative" approach to disengagements that required "drivers to proactively take manual control of the vehicle any time the system encounters a scenario beyond our currently proven abilities." Further, the company said its "software self-monitors and returns control back to the driver when it encounters any errors or issues during operation."

The letter implies Apple's built-in threshold for failure is, or at least was, much less tolerant than competing systems, a thesis was borne out in total disengagement figures recorded during the period. From December 2017 through November 2018, the 47 other firms testing automated vehicles on California roads clocked a cumulative 3,040 disengagements over nearly 2 million logged miles.

That said, Apple appears to have narrowed its reporting methodology in 2019.

Of the 64 disengagements, 52 were initiated by the autonomous vehicle system, while human pilots overrode AV decisions 12 times. The most common reason for disengagement was a vague "controls discrepancy" issue.

According to numbers compiled by CNET, Baidu led the pack in 2019 with 0.06 disengagements per 1,000 miles, followed by Waymo and Cruise, both of which managed 0.08 disengagements per 1,000 miles. Toyota reported the highest rate of disengagements with 1,620 per 1,000 miles, though the Japanese carmaker fielded six cars that traveled 1,817 miles.

Apple's self-driving car initiative began under the "Project Titan" banner, with the program initially tasked with creating a branded car from whole cloth. Apple scaled back operations in late 2016 after hitting a number of snags and the Titan team has since refocused attention to autonomous vehicle subsystems. Whether an "Apple Car" will one day make a public debut is unknown.