Weekly internet health check, US and worldwide

ThousandEyes, which tracks internet and cloud traffic, provides Network World with weekly updates on the performance of three categories of service provider: ISP, cloud provider, UCaaS

thousandeyes map
ThousandEyes

The reliability of services delivered by ISPs, cloud providers and conferencing services (a.k.a. unified communications-as-a-service (UCaaS)) is an indication of how well served businesses are via the internet.

ThousandEyes is monitoring how these providers are handling the performance challenges they face. It will provide Network World a roundup of interesting events of the week in the delivery of these services, and Network World will provide a summary here. Stop back next week for another update, and see more details here.

Update April 12

The number of outages last week across all three categories increased slightly from 210 to 214, up 2% compared to the week prior. In the US they decreased from 93 to 88, down 5%.

Globally, the number of ISP outages decreased from 143 to 137, a 4% decrease, and in the US they decreased from 74 to 73.

Cloud-provider network outages worldwide increased from nine to 12, up a third, while in the US they decreased from three to one, down two thirds.

Globally, collaboration-app network outages increased from one to two. In the US they dropped from one to zero.

There were two notable outages during the week. At 2:35 a.m. EDT on April 8, NTT America, experienced a 34-minute outage that affected some customers and downstream partners across countries including the US, Australia, Canada, France, India, Germany, UK, Switzerland, Japan, Hong Kong, and the Netherlands. The outage appeared initially to be centered on NTT America nodes in Newark, New Jersey, and Paris, France. The issue was cleared around 3:10 a.m. EDT.

About 10 p.m. EDT on April 6, AT&T experienced an outage on its network affecting customers in countries including the US, UK, Japan, Germany, Canada, Australia, India, Brazil, Republic of Korea, Switzerland, and the Netherlands. The outage centered on AT&T nodes in Phoenix, Arizona, lasted 24 minutes, and was cleared around 10:25 p.m EDT.

Update April 5

Global outages across all three categories decreased over the last week from 282 to 210, down 26%, and in the U.S., fell from 119 to 93, a 22% decrease.

The number of ISP outages globally dropped from 204 to 143, a 30% decrease and decreased in the US from 96 to 74, which is 23%.

Globally, cloud-provider outages went from 20 to 9, a 55% decrease. In the US outages went from four to three.

Worldwide, collaboration-app network outages dropped from seven to one and decreased from 2 to 1 in the US.

There were three notable outages during the week.

At 7 a.m. EDT on March 30, Cogent Communications experienced a 44-minute outage that affected multiple downstream providers, as well as Cogent customers globally. The outage appeared to be centered on Cogent nodes in El Paso, TX, Washington DC, and Phoenix, AZ. Five minutes in, the number of Cogent nodes exhibiting outage conditions increased to include nodes located in Salt Lake City, UT, Houston, TX, San Francisco, CA, and Los Angeles, CA. Fifteen minutes in, just those in Los Angeles, CA, San Francisco, CA, and Washington DC still exhibited outage conditions. Twenty minutes in, nodes in San Francisco, CA, and Los Angeles, CA recovered, but the Washington DC nodes remained out for a further 24 minutes.

Around 9:45 p.m. EDT on March 31, the AT&T network experienced an outage that impacted AT&T customers in multiple countries, including the US, UK, Japan, Germany, Canada, Australia, India, Brazil, Republic of Korea, Switzerland, and the Netherlands. IT centered on AT&T nodes in Phoenix, AZ, and lasted 18 minutes.

On April 1, Microsoft experienced an interruption that impacted customers in multiple countries, including the US, UK, Germany, Poland, Belgium, the Netherlands, Australia, Sweden, Japan, France, Ireland, China, Turkey, and the Ukraine. First observed around 5:30 p.m. EDT, the outage appeared to impact availability of Microsoft Azure DNS services. The outage lasted 24 minutes, with full availability being restored around 6:00 p.m. EDT.

Update March 29

Global outages across all three categories decreased from 300 to 282, down 6% from the previous week, and in the US they dropped from 143 to 119, a 17% decrease.

ISP outages globally increased from 197 to 204, a 4% increase. In the US they dropped from 106 to 96, a 9% dip.

Cloud-provider network outages went down from 26 to 20, a 23% decrease, and in the US decreased from five to four.

Globally, collaboration-app network outages increased from four to seven, a 75% increase. In the U.S., they moved up from one to two.

There were two notable outages during theweek. On March 23, Level 3 Communications, experienced an outage that impacted multiple downstream partners and customers in multiple countries including the US, Malaysia, the UK, the Netherlands, Brazil, India, the Czech Republic, Canada, France, Japan, and Australia. The 18-minute outage was first observed around 11:30 a.m. EDT and appeared centered on Level 3 nodes located in London,UK. During the outage, the number of affected nodes in London incrementally decreased, with the outage cleared around 11:50 a.m. EDT. Click here for an interactive view of the outage.

March 24, Zayo Group experienced a 24-minute outage that affected some of its partners and customers in the US. It was observed around 2:35 p.m. EDT and appeared to center on Zayo Group nodes located in Los Angeles, CA. The outage was cleared around 3 p.m. EDT. Click here for an interactive view of the outage.

Update March 22

Globally, outages in all three categories increased from 281 to 300, up 7%. In the US they increased from 137 to 143, 4 %.

The number of ISP outages decreased from 203 to 197, a 3% decrease, while in the US, the drop went from 108 to 106, a 2% decrease.

Cloud-provider network outages went up from 11 to 26, a 136%, but in the US they decreased from six to five. 

Collaboration-app network outages increased two to four, and in the US remained at one.

A notable outage occurred on March 17 when Cloudflare suffered an interruption that impacted its customer in the northwest Pacific region of the US and Canada. The 33-minute outage over a one-hour period, was first observed around 10:20 a.m. EDT and appeared to center on Cloudflare nodes located in Kansas City, MO. This first portion of the outage lasted around three minutes. Fifteen minutes later there was a 22-minute incident centered on Cloudflare nodes located in Seattle, WA. Forty minutes after the outage was first observed, two more were observed, again centering on Cloudflare nodes in Seattle, WA. It was cleared around 11:25 a.m. EDT. Click here for an interactive view of the outage.

Update March 14

Outages in all three categories worldwide during the previous week were down from 385 to 281, a 27% decrease. In the US, they dropped from 168 to 137, an 18% change.

Globally the number of ISP outages decreased from 281 to 203, down 28%, and from 132 to 108 in the US, down 18%.

Cloud-provider outages fell from 26 to 11 worldwide, a decrease of 58%. In the US, they rose from four to six.

Collaboration-app network outages worldwide fell from five to two, and in the US from five to one.

On March 10, Dynamic Network Services experienced an interruption that resulted in DNS-resolution degradation on their Dyn Managed DNS service. The disruption affected users in countries including the UK, South Africa, Singapore, Australia, Ireland, France, Spain, and Portugal. The 55-minute outage was first observed around 6:40 p.m. EST and appeared to be centered on Dyn nodes located in London, UK. Twenty minutes later, a second Dyn node in Manchester, NH, showed outage conditions. The appearance of this second Dyn node coincided with a Dyn notification that their engineers had identified the issue and had implemented a fix. Twenty-five minutes into the outage, only the Dyn node located in London, UK, was exhibiting outage conditions and the number of affected services began to reduce, indicating the service was recovering. The outage was cleared around 7:35 p.m. EST. Click here for an interactive view of the outage.

On March 11, NTT America experienced an outage affecting some of its customers and downstream partners across countries including, the US, Australia, Canada, France, India, Germany, UK, Switzerland, and the Netherlands. The 20-minute outage was first observed around 3:05 p.m. EST and appeared to be centered on NTT America nodes located in Ashburn, VA, and Los Angeles, CA. Five minutes into the outage, the countries affected were reduced to just the US, UK, the Netherlands, and Germany, accessing downstream NTT networks. Five minutes later the outage cleared at the Ashburn, VA, node leaving just the node in Los Angeles, CA, exhibiting outage conditions. That outage was cleared around 3:30 p.m. EST. Click here for an interactive view of the outage.

Update March 7

Outages in all three categories worldwide fell from 393 to 385, a 2% decrease compared to the week before. In the US, they decreased from 184 to 168, 9% fewer.

Globally, the number of ISP outages decreased from 311 to 281, a 10% decline,while in the US they decreased from 166 to 132, down 20%.

Cloud provider outages worldwide increased from 22 to 26, an 18% increase, and the change in the US was an increase from two to four.

The number of collaboration-app network outages jumped from two to five, all of them in the US.

There were two notable outages during the week. On March 3, UUNETVerizon experienced an outage that impacted many of its peers and customers, including, Bank of America, JP Morgan Chase, Nomura, Samsung, and Zoom. The outage, lasting around 36 minutes over a 75 minute period, was first observed around 9:00 a.m. EST and appeared to center on UUNETVerizon nodes in Philadelphia, PA, and Ashburn, VA. This initial part of the outage lasted around 4 minutes and appeared to have a cascading impact on Cogent infrastructure located in New York, NY, and affected Cogent’s path to the JP Morgan Chase network. Approximately five minutes after the initial outage cleared, a second was observed that lasted around 13 minutes. It was observed on UUNETVerizon nodes located in Seattle, WA and Dallas, TX, as well as appearing to have a cascading impact in Level 3 Communications infrastructure located in Seattle, WA, and affecting Level 3 customers and partners in Canada. Five minutes into this second period, the Level 3 infrastructure direct outage cleared and after another five minutes, the only UUNETVerizon nodes exhibiting the issue were located in Dallas, TX. Around 9:50 a.m. EST, the third occurrence of the outage was observed 20 minutes after the second. This outage lasted around 19 minutes and was initially focused on UUNETVerizon infrastructure in Dallas, TX. Five minutes into the third period of the outage, UUNETVerizon infrastructure exhibiting problems expanded to include Seattle, WA. Approximately 10 minutes into this third period of the outage, UUNETVerizon infrastructure located in San Jose, CA was added to those in Seattle, WA, and Dallas, TX. At around 10:10 AM ET, the UUNETVerizon infrastructure located in San Jose, CA, was the only infrastructure exhibiting issues. The outage was cleared  around 10:15 AM ET. Click here for an interactive view of the outage.

On March 3, PCCW experienced an outage affecting some of its U.S. customers and networks, including Flagstar Bank, Target, Bloomberg, Morgan Stanley, and Dell. The outage lasted around 31 minutes and was divided into three periods over an hour and 20 minutes. The outage was first observed around 8:45 a.m. EST and appeared to center on PCCW infrastructure located in Ashburn VA. The first period of the outage lasted around 9 minutes, before recurring 15 minutes later, again centered on PCCW infrastructure located in Ashburn, VA. This second outage lasted around 19 minutes. The third period was observed 30 minutes after the second ended and lasted around 9 minutes. The outage was cleared around 10:05 a.m. EST.

Click here for an interactive view of the outage.

Update March 1

Global outages across all three categories jumped from 279 to 393, a 41% increase over the week before. In the US outages went from 138 to 184, up 33%.

ISP outages rose from 233 to 311, a 33% increase worldwide, and from 123 to 166 in the US, a 35% increase.

Cloud-provider network outages globally jumped from 5 to 22, a 340% increase. The US accounted for two of them, up from one the week before.

Collaboration-app network outages dropped from four to two globally and from two to zero in the US.

There were three notable outages this week.

On Feb. 23, LinkedIn experienced a service disruption affecting its mobile and desktop global user base. The outage was first observed around 1:50 p.m. EST, with users attempting to connect to LinkedIn receiving server-unavailable error messages. Around 45 minutes later, services to some regions began to return, although others were still unable to use the services. After another 45 minutes, the server unavailable messages were replaced with content not available errors. The total disruption lasted around two hours, during which no network issues were observed connecting to LinkedIn web servers, further indicating the issue was application related. Service was restored around 3:40 p.m. EST. Click here for an interactive view of the outage.

Related:
1 2 3 4 5 6 Page 1
Page 1 of 6
IT Salary Survey 2021: The results are in