Facebook shows signs of life after hours-long outage

Facebook, Instagram, WhatsApp, Messenger and Oculus VR all but disappeared from the internet at around 11:30 a.m. Eastern, due to what appears to be a misconfiguration of Facebook's servers. After nearly six hours of downtime, Facebook's services began to reappear for some users, though a full recovery is not expected for some time.

As detailed in a blog post from Cloudflare, the issue traces back to a Border Gateway Protocol (BGP) update that ran afoul, impacting traffic routing and resulting in Domain Name System (DNS) failures. BGP is a system used by networks to advertise their presence to other networks and route traffic accordingly. Without routing information announced by Facebook, DNS resolvers are unable to respond to queries for IP addresses like facebook.com and instagram.com.

A source who is reportedly working on the recovery effort told journalist Brian Krebs that the BGP update blocked remote access to Facebook's systems, meaning off-site technicians were unable to revert the change. Those who had physical access to the impacted systems were also unable to apply a fix because they lacked network access.

Compounding the problem, security engineers said they could not reach affected servers because their digital badges stopped working, reports The New York Times. Other employees reported Workplace, Facebook's internal communications platform, went offline with the wider outage.

An internal memo obtained by The Times reveals that Facebook sent a team of employees to its data center in Santa Clara, Calif., to attempt a "manual reset" of the servers. The effort apparently worked, as services are slowly coming back online.

Facebook has not elaborated on the matter, though executives have taken to other platforms, like Twitter, to apologize for the downtime.