It started around 3:30 in the morning on the East Coast, reports flooding in about an AT&T service outage. Customers complained across Reddit and X and logged their issues on Downdetector, a site that, well, detects when services go down.
The impact was widespread; AT&T users from New York to Atlanta to Dallas claimed no signal, phones stuck in SOS mode. Multiple police departments, including in San Francisco, reported that some users were unable to contact 911 as a result of the outages. By 9 am ET, Downdetector was showing more than 72,000 AT&T outages across the US; the site’s baseline for AT&T service issues is 42.
“Some of our customers are experiencing wireless service interruptions this morning,” AT&T spokesperson Jim Greer said in a statement. “We are working urgently to restore service to them. We encourage the use of Wi-Fi calling until service is restored.”
It’s not just AT&T’s customers, though. Every cell carrier on Downdetector showed spikes Thursday morning, including behemoths Verizon and T-Mobile. But both companies confirmed that there’s nothing wrong with their networks; the complaints are instead collateral damage, people trying to reach their AT&T contacts and not getting through.
“We did not experience an outage,” a T-Mobile spokesperson told WIRED over email. “Our network is operating normally. Downdetector is likely reflecting challenges our customers were having attempting to connect to users on other networks.”
Similarly, a Verizon spokesperson said over email that “Verizon’s network is operating normally. Some customers experienced issues this morning when calling or texting with customers served by another carrier.”
The news isn’t all bad. AT&T confirmed that FirstNet, the first responder network that AT&T built out, is operating normally. And while it’s an imperfect substitute—or totally unusable, depending on where you are—Wi-Fi calling should suffice as a workaround for now. (To turn on Wi-Fi calling, go to your smartphone’s Settings, then Network & internet, then switch the Wi-Fi Calling toggle to on. Exact wording might vary depending on your phone model and operating system.)
Cell network outages happen with some frequency, and can stem from any number of causes. “Three things come to mind with major network outages,” says Erik Keith, senior research analyst at S&P Global Market Intelligence, “cyber attacks, fiber cuts at critical points in the network, and software or system upgrades that don’t go as planned or have unforeseen issues.”
In this case, the latter explanation seems the most likely, says Doug Madory, director of internet analysis at Kentik, a network monitoring company. “It's quite unusual,” Madory says, noting that even within the same household some AT&T devices are affected while others aren't. “I'm guessing that they did some sort of internal software push and it didn't agree with some subset of these devices, and they're having trouble reverting it."
While interruptions of this scale are rare, they do happen globally once or twice a year. Last month, Spanish carrier Orange España lost half of its network for hours due to a cyberattack. T-Mobile had a massive service interruption of this scale a year ago that it ultimately attributed to a “third-party fiber interruption issue.” That issue appeared more self-contained, though, than the ripple effects seemingly caused by AT&T’s problems.
Thursday afternoon, the FCC said on X that it was “actively investigating” the outage. “We are in touch with AT&T and public safety authorities, including FirstNet, as well as other providers,” the agency said.
By that time, the incident appeared close to being resolved. By 11 am ET, DownDetector incidents had begun to taper off, although over 60,000 people were still reporting issues. Shortly thereafter, AT&T updated that it had made significant progress with the issue. “Our network teams took immediate action and so far three-quarters of our network has been restored,” Greer said in an emailed comment at 11:14 am ET. “We are working as quickly as possible to restore service to remaining customers.” As of 2:30pm ET, DownDetector showed over 3,000 reports, well below the peak of the trouble. Less than an hour later, AT&T reported that it had restored service to all of its customers.
It wasn't until later on Thursday evening that the company gave an indication of the underlying cause. “We believe that today’s outage was caused by the application and execution of an incorrect process used as we were expanding our network, not a cyber attack,” the company posted to a network update page. “We are continuing our assessment of today’s outage to ensure we keep delivering the service that our customers deserve.”
Updated: 2/23/2024, 10:40 am EST: This story has been updated to reflect that AT&T resolved the issue and provided an initial explanation.
