Another 911 outage. Does any accountability exist?

911 call processing ground to a halt nationwide for AT&T Wireless customers last week. What failed? The public may never know.

911 accountability—does it exist?
Nick Barber

AT&T Wireless customers who tried to reach 911 in the evening hours of Wednesday, March 8, were left stranded in more than a dozen major cities. Based on unconfirmed, but widespread reports, the problem may have affected callers nationwide, with AT&T only sending out tweets again and again, stating:

[AT&T is aware] of issue affecting some calls to 911 for wireless customers. Working to resolve ASAP. We apologize to those affected. 

With no root cause offered, several public safety agencies and citizens were left confused on what to do. In what has become a natural reaction to any traumatic event, both the general public and several agencies took to Twitter, Facebook and other social media outlets on the web as the go-to place for help, as well as an outlet for frustration—with many announcing their intentions to switch carriers.

Who regulates 911?

The National Emergency Number Association, a U.S.-based association that serves its members and the greater public safety community as the only professional organization solely focused on 911 policy, technology, operations and education issues. With more than 11,000 members in 48 chapters across the United States and around the globe, NENA promotes the implementation and awareness of 911, as well as international three-digit emergency communications systems.

NENA works with 911 professionals nationwide, public policy leaders, emergency services and telecommunications industry partners, like-minded public safety associations, and other stakeholder groups to develop and carry out critical programs and initiatives, facilitate the creation of an IP-based next-generation 911 system, and establish industry-leading standards, training, and certifications.

Based on the organization's estimates, 240 million calls are made to 911 in the U.S. each year. Of those calls, 70 percent (about 168 million) are from cellular devices, with that number increasing every day.

What do the 911 numbers say?

The supposed gold standard level of network service is measured in the percentage that system is working normally—what is known as 5 Nines Reliability. In reality, this level is difficult to reach, as it means the system is up 99.999 percent of the time. Each year contains 525,600 minutes of "up-time" availability. To maintain 5 Nines Reliability, centers would be allotted only 315 seconds of downtime per year.

Working backwards, based on estimates from NENA, cellular calls to 911 arrive at an average of just over five per second nationwide. Based on that, it would mean nearly 1,700 calls a year could still fail, with an average of over four a day—again nationwide.

Are rules enough to solve the 911 uptime issue?

Unfortunately, no. Rules and mathematical probability will not correct the core issue. Actual outages do occur, and people are affected, with some ending very badly. In most cases, these incidents remain rare and are often spread out over time and geography.

While that might minimize the emotional sting of tragedy, it does nothing to increase the level of reliability, the carrier network diversity that must be there, or the willingness to transparently report outages instead of locking away the information while claiming proprietary information and security risks.

Most people want to believe the 911 networks are built to the highest standard of resiliency and reliability possible. While standards from NENA and APCO exist prescribing these levels of service, they themselves go only so far. To be effective, standards need to be promulgated by industry experts and agreed upon by those providing services. They then need to be meticulously implemented by PSAPs and, most important, include a process that tracks compliance and accountability.

The FCC's role in 911 outages

Rules and regulations. Policy and penalties. It is only in the enforcement of these where the mighty FCC steps into the picture. The organization collects information on communications service disruptions that transpire in the nation's communications infrastructure. This information aides them in fines and penalties.

At the core of this collection activity of data is the Network Outage Reporting System (NORS). This provides an interface for carriers to report information regarding communications disruptions that have exceeded the thresholds that are outlined in the FCC's rules that are published and defined in the Code of Federal Rules: 47 C.F.R. Part 4

NORS is managed by the Cyber Security and Communications Reliability Division of the FCC's Public Safety and Homeland Security Bureau. The bureau is run by Acting Bureau Chief Lisa M. Fowlkes, who was appointed by FCC Chairman Ajit Pai as part of his transition activities. The repository information is classified as confidential and therefore not available to the public.

This can make accountability and transparency difficult, however. The FCC closely monitors the data submitted to NORS, and the Public Safety and Homeland Security Bureau can then engage in additional studies as needed. Using this information from these studies, further actions and task forces, such as TFOPA and the EAAC, can be requested where applicable, and assessments can be made requiring additional action, such as recommendations for changes in policy and rules.

Personally, I had the honor of participating on both of those initiatives, as well as the DAC, dealing with access and accessibility issues for persons who are deaf, are blind or have a speech impairment.

Allowable 911 outage tolerances

The nation’s First Responders and Emergency Management officials all agree that our 911 communications networks are critical infrastructure. Based on this, the FCC has defined specific thresholds for 911 outage reporting.

In 47 C.F.R. Part 4.5, 911 centers are designated as “special facilities” and that a reportable outage has occurred if:

“There is a loss of communications to PSAP(s) potentially affecting at least 900,000 user-minutes and: The failure is neither at the PSAP(s) nor on the premises of the PSAP(s); no reroute for all end users was available, and the outage lasts 30 minutes or more.”

Clearly, this last outage was a failure in the AT&T network. The outage was experienced across the nation,  specifically on the AT&T wireless network. In fact, Text to 911, where deployed, remained as an option to the 10-digit PSTN numbers being advertised on Twitter and Facebook.

Tick tock—your time starts NOW! And it may cost you

As soon as an outage is noticed, the proverbial clock starts. Carriers have only 72 hours to submit an Initial Communications Outage Report to the FCC via NORS. Final reports, with full root cause analysis and remediation plans to prevent further incidents, are due in 30 days.

While these reports and standards provide valuable insight, they are confidential and not available to others who could benefit by not making the same mistakes. For the carrier that is responsible for the outage, penalties for their actions incent them to remedy to the situation. In recent years, the FCC has imposed more than $42 million in penalties—a significant amount, as noted here for the FCC Enforcement Bureau website:

July 2006 — FCC Reaches $2.4M Settlement with GCI - Wireless 911 Outages in Alaska

September 2015 — COs Fined $1.4M For Failing To Accept 911 Calls For Hearing Impaired

July 2015 — T-Mobile to Pay $17.5 Million to Resolve 911 Outage Investigation

April 2016 — FCC Fines CenturyLink/Intrado $17.4M for Multi-State 911 Outage

March 2015 — Verizon to Pay $3.4 Million to Resolve 911 Outage Investigation

The question remains what penalty will be handed down to AT&T for this outage. Given the number of users impacted and the past financial penalties, it is very conceivable that fines for this incident could reach $50 million, considering AT&T’s estimated 30 percent market share of wireless subscribers.

The best advice: Be prepared for a 911 outage

At the end of the day, personal safety is the goal of 911. Always know where you are, be prepared to provide landmarks to help dispatchers locate you, and program the local 10-digit number of the police department where you live and work into your cell phone so that they are handy in an emergency.

In case voice calls are not going through, find out if and when Text to 911 will be rolled out in your area. With the AT&T Wireless outage, users quickly figured out it Text to 911 did work and used social media to get the word out.

Be cognizant of the fact that coverage across the U.S. is still sparse in many areas, though. According to the most recent FCC Public Safety and Homeland Security Bureau information, Text to 911 is still a future capability or not currently planned in most areas.

 [Click to down load]  Text to 911 Availability Map as per the FCC PSHSB
Created by Mark J. Fletcher, ENP with FCC PSHSB

In the end, I still believe our best defense is awareness. In today’s world of autonomous computer assistance, with everything from Alexa turning on lights to Siri dialing our phones, nothing beats a good level head and a well-practiced plan in the event of an emergency. At least when I called my 18-year-old daughter at college to remind her to do this, I got a “Thank you Daddy – I love you.

She knows just how to push my buttons in any direction—no worries.
I love you, too, Haley.

Copyright © 2017 IDG Communications, Inc.

The 10 most powerful companies in enterprise networking 2022