On May 17, Hurricane Electric experienced an outage that was divided into three instances over an hour and a half that affected users across countries including the US, Australia, Singapore, Hong Kong, the UK, Brazil, Germany, South Africa, the Netherlands, and Canada. The first period was observed around 1:43 p.m. EDT centered on Hurricane Electric nodes in San Francisco and San Jose, California. After five minutes, the San Francisco nodes appeared to recover, reducing the scope of the outage. But five minutes after that, San Francisco nodes exhibited outage conditions again. Five minutes after this occurrence cleared, a second one lasting three minutes was observed centered on the San Jose and San Francisco nodes. Around 2:15 p.m. EDT, the nodes appeared to recover, temporarily clearing the outage, but an hour later, those two nodes exhibited outage conditions again before clearing after eight minutes. The total outage lasted around 26 minutes and was cleared around 3:25 p.m. EDT.
Update May 17
Global outages across all three categories last week increased from 237 to 252, up 6%, while in the US, they increased from 95 to 123, a 29% jump.
ISP outages globally increased from 168 to 180, and in the US they increased from 76 to 98.
Cloud-provider network outages worldwide increased from 13 to 14 but dropped from four to one in the US.
Globally, collaboration app network outages dropped from four to three, and in the US from two to one.
There were three significant outages this week.
About 3:30 p.m. on May 12, NetActuate experienced an outage affecting downstream partners and customers in the US. It lasted around 13 minutes overall, divided into two occurrences spanning a 30-minute period. The first lasted four minutes and appeared to center on NetActuate nodes located in Raleigh, North Carolina. The outage reappeared 15 minutes later and lasted nine minutes and centered on the Raleigh nodes and nodes in Durham, North Carolina, increasing the number of customers affected. The nodes in Durham cleared five minutes into the second period of the outage. The outage was cleared around 4:05 p.m. EDT.
About 10:15 p.m. EDT on May 13, TATA Communications (America) experienced an outage affecting downstream partners and customers in countries including the US, the UK, Australia, India, China, Germany, the Netherlands, Japan, Hong Kong, Brazil, Switzerland, Republic of Korea, and Canada. The outage lasted 35 minutes and was divided into two periods over 55 minutes. The first period lasted around nine minutes and appeared to be centered on TATA nodes in Tokyo, Japan. About 10 minutes after it cleared, the outage reappeared, centering on TATA nodes located in Hong Kong and after another five minutes expanding to TATA nodes in Los Angeles, California. The nodes in Hong Kong appeared to clear after 15 minutes, leaving just the Los Angeles nodes exhibiting outage conditions. This second period of the outage lasted 26 minutes and was cleared around 11:10 p.m. EDT.
About 5:10 p.m. EDT on May 11, Salesforce experienced an interruption that left users able to reach the Salesforce front-end, but experiencing issues logging on and navigating to the Salesforce Sales Cloud, Marketing Cloud, Commerce Cloud, and Experience Cloud. That indicated internet and network connectivity for end users was functioning. Salesforce identified issues with its domain name system that had a cascading effect on its services. A fix was implemented around 7:45 p.m. EDT, and the outage was cleared about 10:15 p.m.EDT.
Update May 10
Global outages across all three categories last week decreased from 243 to 237, while in the US they decreased from 117 to 95.
Globally, the number of ISP outages increased by one, from 167 to 168. In the US they dropped from 100 to 76, a 24% decrease compared to the week prior.
Cloud provider network outages dropped 52% from 27 to 13 worldwide. In the US, they doubled from two to four.
Collaboration app network outages increased from three to four globally and stayed at two in the US.
There were two notable outages during the week.
Around 6 p.m. EDT on May 3, Cloudflare experienced a disruption to its Magic Transit service when some customers began experiencing significant packet loss at Cloudflare’s network edge. The outage appeared to impact Cloudflare's infrastructure across the globe, with packet loss occurring at varying levels for approximately two hours. At around 8 p.m. EDT, Cloudflare began implementing a fix for the issue, and announced that it was resolved just after 9 p.m. EDT.
Around 3:35 a.m. EDT on May 7, PCCW experienced an outage impacting some of its customers and networks in multiple countries including, the US, Australia, Brazil, and China. The outage lasted around 22 minutes and was divided into two periods over a half-hour span, the first of which appeared to center on PCCW infrastructure located in Ashburn, Virginia. It lasted about 18 minutes. Five minutes later it recurred, again centered in Ashburn, but with the outage condition including infrastructure in New York, New York. It lasted lasted around 4 minutes and was cleared around 4:05 a.m. EDT.
Update May 3
The number of outages globally across all three categories decreased slightly last week from 246 to 243. In the US, outages decreased from 123 to 117.
The number of ISP outages worldwide increased from 162 to 167, while in the US they increased from 92 to 100.
Cloud-provider network outages remained at 27 overall and went down in the US from five to two.
Globally, collaboration-app network outages remained at three and dropped from three to two in the US.
There were three notable outages during the week.
At 7:36 a.m. EDT on April 27, TATA Communications (America), experienced an outage affecting many of its downstream partners and customers in countries including the US, Australia, India, Japan, and the Philippines. The outage affected TATA nodes in Ashburn, Virginia, and appeared to clear after five minutes, but came back around 8 a.m. EDT centered on TATA nodes in Chicago, Illinois, and Los Angeles, California. The Chicago nodes appeared to recover 10 minutes later, leaving only the nodes in Los Angeles exhibiting outage conditions. About 10 minutes after that, nodes in Hong Kong began exhibiting outage conditions. In total, the outage lasted around 29 minutes, divided into two occurrences over the course of an hour and was cleared around 8:25 a.m. EDT.
AT 12:40 a.m. EDT on April 29, Hurricane Electric experienced an outage affecting users across countries including the US, Spain, Russia, and Ireland. It affected Hurricane Electric nodes in Ashburn, Virginia, and New York, New York. After five minutes, the nodes in New York appeared to recover, reducing the impact to US users only. Around 12:50 a.m. EDT, the nodes in Ashburn appeared to recover, temporarily clearing the outage. But five minutes later the nodes located in New York began exhibiting outage conditions again before clearing after three minutes. The total outage lasted around 11 minutes, consisting of two periods over half an hour. The issue was cleared around 1 a.m. EDT.
Around 6 a.m. EDT on April 27, Microsoft experienced an outage that affected its Teams users globally for about an hour and a half. The outage occurred outside of business hours for much of the Americas, but its global nature resulted in service disruption for users connecting from Asia and Europe. During the outage other Microsoft services continued to be reachable and available, but Teams services appeared unable to authenticate connection requests. Check out the ThousandEyes Internet Report for a deeper dive into the outage.
Update April 16
Globally the number of ISP outages moved from 160 to 162. In the US they moved from 85 to 92, an 8% increase.
Cloud provider network outages overall dropped by one, from 28 to 27. In the US they increased from two to five.
Collaboration-app network outages globally dropped from eight to three, and in the US from five to three.
There were two notable outages during the week.
At 11:10 p.m. EDT on April 20, Internap experienced an outage that hit many of its downstream partners and customers in countries including the US, the UK, Japan, Australia, Singapore, Germany, and Hong Kong. It lasted 18 minutes and centered on Internap nodes in New York, New York. The outage peaked 10 minutes later and was cleared around 11:30 p.m. EDT.
At 11:35 a.m. EDT on April 21, Zayo Group experienced an outage affecting partners and customers in countries including the US, China, Mexico, Canada, Hong Kong, Germany, Sweden, Brazil, India, and Singapore. It lasted around 24 minutes over a one-hour period and appeared to initially center on Zayo Group nodes in Atlanta, Georgia; Salt Lake City, Utah; and Denver, Colorado. A second occurrence started about 25 minutes later and lasted about four minutes. The outage expanded to include Zayo nodes located in Toronto, Canada, and that coincided with an expansion of affected countries and partners. Ten minutes after that, a third, three-minute occurrence centered on Zayo nodes in San Francisco, California, and affected a handful of countries. The final period of the outage was observed around 12:20 p.m. EDT centered on Zayo nodes in Phoenix, Arizona, and lasted 15 minutes. It appeared to affect only US-based customers and partners. The outage was cleared around 12:40 p.m. EDT.
Update April 19
Global outages in all three categories rose from 214 to 245, up 14% over the previous week, and from 88 to 106, up 20%, in the US.
The number of ISP outages worldwide increased from 137 to 160, a 17% increase, and in the US from 73 to 85, a 16% increase.
Cloud-provider network outages globally went from 12 to 28, a 133% jump, and in the US they increased from 1 to 2.
Worldwide collaboration-app network outages increased from two to eight, a 300% increase, while the US, outages jumped from zero to five.
There were two notable outages during the week. Around 4:46 p.m. EDT on April 12, TATA Communications (America) experienced an outage that impacted many of its downstream partners and customers in countries including the US, the UK, Australia, India, Brazil, Germany, the Netherlands, Japan, Switzerland, Republic of Korea, and Canada. The outage, lasting around nine minutes, appeared to be centered on TATA nodes located in Tokyo, Japan. The outage was cleared around 5:10 p.m. EDT.
Around 8:45 a.m. EDT on April 14, Zayo Group experienced an outage that affected some of its partners and customers in multiple countries. The outage lasted around 36 minutes, was first observed in Zayo nodes in Atlanta, Georgia. Five minutes later the outage expanded to include nodes in Seattle, Washington, and Chicago, Illinois, which expanded the area affected from just the US to include the UK, Russia, Singapore, India, and Canada. Five minutes after that, nodes in Houston, Texas, became involved and customers in Australia were affected. Around 9:10 a.m. EDT, nodes located Denver, Colorado, were affected. This appeared to be the peak of the overall effects of the outage. Thirty minutes into the outage, the Denver node appeared to recover, reducing the number of affected countries and downstream partners. The outage was cleared around 9:25 a.m. EDT.
Update April 12
The number of outages last week across all three categories increased slightly from 210 to 214, up 2% compared to the week prior. In the US they decreased from 93 to 88, down 5%.
Globally, the number of ISP outages decreased from 143 to 137, a 4% decrease, and in the US they decreased from 74 to 73.
Cloud-provider network outages worldwide increased from nine to 12, up a third, while in the US they decreased from three to one, down two thirds.
Globally, collaboration-app network outages increased from one to two. In the US they dropped from one to zero.
There were two notable outages during the week. At 2:35 a.m. EDT on April 8, NTT America, experienced a 34-minute outage that affected some customers and downstream partners across countries including the US, Australia, Canada, France, India, Germany, UK, Switzerland, Japan, Hong Kong, and the Netherlands. The outage appeared initially to be centered on NTT America nodes in Newark, New Jersey, and Paris, France. The issue was cleared around 3:10 a.m. EDT.
About 10 p.m. EDT on April 6, AT&T experienced an outage on its network affecting customers in countries including the US, UK, Japan, Germany, Canada, Australia, India, Brazil, Republic of Korea, Switzerland, and the Netherlands. The outage centered on AT&T nodes in Phoenix, Arizona, lasted 24 minutes, and was cleared around 10:25 p.m EDT.
Update April 5
Global outages across all three categories decreased over the last week from 282 to 210, down 26%, and in the U.S., fell from 119 to 93, a 22% decrease.
The number of ISP outages globally dropped from 204 to 143, a 30% decrease and decreased in the US from 96 to 74, which is 23%.
Globally, cloud-provider outages went from 20 to 9, a 55% decrease. In the US outages went from four to three.
Worldwide, collaboration-app network outages dropped from seven to one and decreased from 2 to 1 in the US.
There were three notable outages during the week.
At 7 a.m. EDT on March 30, Cogent Communications experienced a 44-minute outage that affected multiple downstream providers, as well as Cogent customers globally. The outage appeared to be centered on Cogent nodes in El Paso, TX, Washington DC, and Phoenix, AZ. Five minutes in, the number of Cogent nodes exhibiting outage conditions increased to include nodes located in Salt Lake City, UT, Houston, TX, San Francisco, CA, and Los Angeles, CA. Fifteen minutes in, just those in Los Angeles, CA, San Francisco, CA, and Washington DC still exhibited outage conditions. Twenty minutes in, nodes in San Francisco, CA, and Los Angeles, CA recovered, but the Washington DC nodes remained out for a further 24 minutes.
Around 9:45 p.m. EDT on March 31, the AT&T network experienced an outage that impacted AT&T customers in multiple countries, including the US, UK, Japan, Germany, Canada, Australia, India, Brazil, Republic of Korea, Switzerland, and the Netherlands. IT centered on AT&T nodes in Phoenix, AZ, and lasted 18 minutes.