Here's what Facebook says triggered the massive outage on its apps

Facebook Inc. blamed a global service outage that kept its social media apps offline for much of Monday on a problem with its network configuration, adding that it found no evidence that user data was compromised during the downtime.
 

“This disruption to network traffic had a cascading effect on the way our data centers communicate, bringing our services to a halt,” the company’s engineering team wrote in a blog post Monday night.

The technical issue took Facebook’s core social network, its photo app Instagram and its WhatsApp and Messenger services offline for hours, marking one of the longest and broadest failures in recent memory. It had a seismic impact, immobilizing a suite of services that more than 2.75 billion people rely on daily to communicate, do business and consume news. The outage dominated news reports and sent people across the globe scrambling to other apps and services in an effort to stay connected and informed.

Chief Executive Officer Mark Zuckerberg apologized on his Facebook page after the network had been restored. “Sorry for the disruption today -- I know how much you rely on our services to stay connected with the people you care about,” he wrote.

Facebook’s internal tools and communications systems were impacted by the disruption, adding to the challenge for engineers working to identify and resolve the issue. Its internal work product, Workplace, was also affected. Fixing the underlying problem involved visiting a physical server facility and manually resetting some servers, a spokesman said.

“To every small and large business, family, and individual who depends on us, I’m sorry,” Chief Technology Office Mike Schroepfer tweeted Monday afternoon. His apology was reiterated by Facebook’s engineering blog.

Facebook’s loss turned into a gain for other social media. Twitter Inc.’s network stayed online, with CEO Jack Dorsey tweeting his endorsement of private messenger Signal as a WhatsApp alternative. Signal signed up millions of new users on the day, while Telegram, whose functionality closely mirrors that of WhatsApp, surged 55 places to top the U.S. iPhone download chart, according to Sensor Tower.

Mike Schroepfer, chief technology officer of Facebook | Bloomberg

While it’s not uncommon for Facebook’s apps to have occasional glitches, they rarely last more than a few minutes. Downdetector, which monitors internet problems, said the Facebook outage was the largest it had seen, with more than 10.6 million reports worldwide.

Monday’s protracted outage was the latest in a cascade of difficult events for Facebook. A former employee turned whistle-blower appeared Sunday on CBS’s “60 Minutes” to accuse the company of prioritizing profits over user safety. The former employee, Frances Haugen, also handed over thousands of damning documents to U.S. lawmakers and the Wall Street Journal, which wrote a series of articles last month on Facebook’s struggles with content moderation and Instagram’s negative psychological impact on teenagers. 

The whistle-blower is set to testify before a Senate subcommittee on Tuesday and plans to tell lawmakers the “frightening truth” about Facebook.

“Facebook became a $1 trillion company by paying for its profits with our safety, including the safety of our children,” Haugen will tell the Senate Tuesday, according to a copy of her prepared testimony. “I came forward because I recognized a frightening truth: almost no one outside of Facebook knows what happens inside Facebook. The company’s leadership keeps vital information from the public, the U.S. government, its shareholders, and governments around the world.”

Facebook shares dropped 4.9% to $326.23 at the close in New York. They had declined before the service disruption was reported, hurt by the whistle-blower’s “60 Minutes” appearance. 

Facebook’s internal apps also stopped working in 2019, following a dispute with Apple Inc. that halted some of the functionality of those apps on the iPhone maker’s platform. The extent of Monday’s disruption was more severe. Some workers even struggled to use Facebook’s identity-badge system at offices during the outage.

--With assistance from Vlad Savov.