The reliability of services delivered by ISPs, cloud providers and conferencing services (a.k.a. unified communications-as-a-service (UCaaS)) is an indication of how well served businesses are via the internet.
ThousandEyes is monitoring how these providers are handling the performance challenges they face. It will provide Network World a roundup of interesting events of the week in the delivery of these services, and Network World will provide a summary here. Stop back next week for another update, and see more details here.
Updated March 13
Global outages in all three categories last week decreased from 337 to 271, down 20% decrease compared to the week prior. In the US they increased from 95 to 105, up 11%.
Globally, ISP outages decreased from 205 to 191, down 7%, and in the US they increased from 66 to 83, up 26%.
Globally, cloud-provider network outages dropped from 12 to three, and in the US they decreased from six to two.
Globally, collaboration-app network outages dropped from 11 to three, and in the US they dropped from nine to one.
Two notable outages:
On March 6, Twitter experienced a service disruption affecting users globally. First observed around 11:45 a.m. EST, many users were unable to access service, although the application remained reachable from a network perspective. During the incident, users were receiving 403 forbidden errors, which is indicative of a backend application issue. Around 12:19 p.m. EST, Twitter announced the disruption was caused by an internal system change that had unintentional consequences, and they were working to resolve it. The disruption lasted 60 minutes, with access restored to users around 12:50 p.m. EST. Click here for an interactive view.
On March 12, Okta experienced an issue that disrupted access to its service for a number of users globally. First observed around 11:56 a.m. EDT, connection requests appeared to return HTTP 403 forbidden status codes, indicating an application issue rather than internet or network problems connecting to Okta. The incident appeared to resolve for most users approximately 48 minutes later around 12:45 p.m. EDT. Click here for an interactive view.
Updated March 6
Global outages across all three categories last week increased from 316 to 337, up 7% from the week prior. In the US, outages increased from 62 to 95, up 53%.
Globally, ISP outages decreased from 228 to 205, down 10%, while in the US they increased from 47 to 66, up 40%.
Globally, cloud-provider network outages increased from 10 to 12, and in the US they increased from four to six.
Globally, collaboration-app network outages jumped from two to 11, and in the US jumped from one to nine.
Two significant outages:
On February 27, TATA Communications (America) Inc. experienced an outage affecting many of its downstream partners and customers in multiple countries, including the US, Germany, the UK, France, the Netherlands, India, Canada, Singapore, Switzerland, Norway, China, Mexico, Portugal, and Hong Kong. The outage lasted 53 minutes, divided into three segments over an hour and 55-minute period. The initial period of the outage was observed around 1:50 a.m. EST and appeared centered on TATA Los Angeles nodes. Five minutes after that occurrence cleared, nodes in Los Angeles and San Francisco, appeared to exhibit outage conditions and cleared after three minutes. Ten minutes later, the third occurrence was observed, affecting nodes in Los Angeles; Newark, New Jersey; and Paris. Ten minutes into the third occurrence, nodes in London and Amsterdam exhibited outage conditions. Twenty-five minutes later, Newark and Amsterdam nodes appeared to clear. The remainder of the outage was cleared around 3:05 a.m. EST. Click here for an interactive view.
On March 1, Cogent Communications experienced an outage affecting downstream providers and Cogent customers in countries including the US, China, Singapore, Taiwan, Indonesia, Turkey, the Republic of Korea, India, Germany, the UK, and Australia. The 23-minute outagewas first observed around 12:05 a.m. EST, initially centered on Cogent nodes in San Francisco and San Jose. Five minutes later, nodes in Oakland, California, Kansas City, Missouri, and Washington, DC exhibited outage conditions. Twenty minutes into the outage, the Kansas City, Washington, DC, and San Francisco nodes appeared to recover. The outage was cleared around 12:30 a.m. EST. Click here for an interactive view.
Updated Feb. 27
Global outages across all three categories last week decreased, from 339 to 316, down 7% compared to the week prior. In the US, they decreased from 76 to 62, down 18%.
Globally, the number of ISP outages decreased from 258 to 228, down 12%, and in the US they decreased from 53 to 47, down 11%.
Globally, cloud-provider network outages decreased from 14 to 10, and in the US from eight to four.
Globally collaboration-app network outages decreased from four to two, and in the US they decreased from two to one.
Two notable outages:
On February 21, Arelion experienced an outage affecting customers and downstream partners across the US. The disruption, lasting a total of 24 minutes, was first observed around 4:40 p.m. EST and appeared to center on nodes located in San Jose, California. Fifteen minutes the number of nodes appeared to reduce, and the outage was cleared around 5:05 p.m. EST. Click here for an interactive view.
On February 24, Oracle experienced an outage on its network affecting customers and downstream partners interacting with Oracle Cloud services in the US and Canada. First observed around 10 p.m. EST, it appeared initially to center on Oracle nodes in Ashburn, Virginia, and Washington, DC. Five minutes later they appeared to clear and nodes in Toronto and Montreal, Canada, exhibited outage conditions. Around 10:10 p.m. EST, the Ashburn and Washington nodes exhibeted outage conditions again. The outage lasted 17 minutes and was cleared at around 10:20 p.m. EST. Click here for an interactive view.
Updated Feb. 20
Global outages across all three categories last week increased from 301 to 339, up 13% compared to the week prior. In the US, outages increased from 73 to 76, up 4%.
Global ISP outages increased from 215 to 258, up 20%, and in the US they remained the same at 53.
Global cloud provider network outages increased from 10 to 14, and they increased from three to eight in the US.
Global collaboration app network outages decreased from five to four, and increased from zero to two in the US.
There were two notable outages:
On February 18, Zayo Group experienced an outage affecting partners and customers in the US, Canada, China, Australia, and Malaysia. The 14-minute outage was first observed around 4 p.m. EST, and appeared centered on Zayo Group nodes located in Houston, Texas; New York, New York; and Newark, New Jersey. Five minutes after being observed, the New York and Newark nodes appeared to recover. The outage cleared around 4:15 p.m. EST. Click here for an interactive view.
On February 14, Hurricane Electric experienced an outage affecting customers and downstream partners in the US, Japan, Mexico, and Hong Kong. The outage, first observed at around 8:01 p.m. EST, lasted a total of 19 minutes and was divided into two occurrences over a 29-minute period. The first occurrence, lasting six minutes, appeared centered on Hurricane Electric nodes in Paris, France. Around 8:15 p.m. EST, five minutes after the Paris nodes appeared to clear, the second occurrence was observed. It lasted around 13 minutes and appeared centered on nodes in New York, New York. As the second occurrence progressed, the number of New York nodes exhibiting outage conditions dropped, which coincided with a reduction in the number of affected regions and customers. The outage was cleared around 7:35 a.m. EDT. Click here for an interactive view.
Updated Feb. 13
Global outages across all three categories last week decreased from 331 to 301, down 9% compared to the week prior. In the US, outages decreased from 117 to 73, down 38%.
Globally, ISP outages decreased from 231 to 215, down 7%, and in the US they dropped from 85 to 53, down 38%.
Globally, cloud-provider network outages decreased from 12 to 10, and in the US they decreased from seven to three.
Globally, collaboration-app outages dropped from 10 to five, and in the US dropped from three to none.
Global outages across all three categories last week decreased from 331 to 301, down 9% compared to the week prior. In the US, outages decreased from 117 to 73, down 38%.
Globally, ISP outages decreased from 231 to 215, down 7%, and in the US they dropped from 85 to 53, down38%.
Two notable outages:
On February 6, Microsoft experienced an outage that affected services in North America, Europe, and Asia. The outage, first observed around 10:55 p.m. EST, appeared to affect user access to Microsoft Outlook services. Microsoft confirmed that a change to some Microsoft 365 systems had contributed to the outage, and used targeted restarts to parts of their infrastructure to restore service. The bulk of the incident lasted about an hour and 39 minutes, although access issues could be seen for about four hours after that. The incident was similar to the January 25th event in terms of global reach and duration, but it did not appear to be network related, as no significant packet loss or latency or unusual routing behavior was observed. Click here for an interactive view.
On February 12, Comcast Communications experienced an outage that affected a number of downstream partners and customers across the US. The 21-minute outage was first observed around 1:30 p.m. EST and appeared to center on Comcast nodes in Denver. The outage was cleared around 1:55 p.m. EST. Click here for an interactive view.
Updated Feb. 6
Globally, outages across all three categories last week decreased compared to the week prior, from 373 to 331, down 11%. In the US they increased from 102 to 117, up15%.
Globally, the number of ISP outages decreased from 278 to 231, down 17%, and in the US they increased from 81 to 85, up 5%.
Globally, cloud-provider network outages increased from 10 to 12 and increased from two to seven in the US.
Globally, collaboration-app network outages increased from four to 10 and increased from one to three in the US.
Two significant outages:
On January 31, Level 3 Communications experienced an outage affecting multiple downstream partners and customers in countries including the US, Canada, Brazil, the Philippines, China, Mexico, the UK, Japan, India, Singapore, Taiwan, and Australia. The outage lasted a total of 47 minutes divided into three occurrences. First observed around 4:25 a.m. EST, the outage initially appeared centered on nodes in San Jose, California. Five minutes later, nodes in Los Angeles and San Francisco, California; Chicago, Illinois; Denver, Colorado; Dallas, Texas; and São Paulo and Rio De Janeiro, Brazil, also exhibited outage conditions. Fourteen minutes after initially being observed, the outages appeared to clear. Around 10 minutes after that, San Jose, Los Angeles, Chicago, Dallas, Denver, and San Francisco nodes appeared to exhibit outage conditions again. The second occurrence lasted 19 minutes and appeared to clear around 5:10 a.m. EST. Twenty minutes after that, a third occurrence was observed, this time initially appearing to center on nodes located in San Jose, Los Angeles, San Francisco, Rio de Janeiro, and Sao Paul. Five minutes into the third occurrence, the Rio de Janeiro and São Paulo nodes appeared to clear, leaving the San Jose, Chicago, Los Angeles, and Denver nodes exhibiting outage conditions. The outage appeared to clear completely around 5:45 a.m. EST. Click here for an interactive view.
On February 3, Okta experienced an issue that disrupted access to its service for a number of users globally. First observed around 1:10 p.m. EST, connection requests appeared to return HTTP 403 forbidden status codes, indicating an application issue rather than internet or network problems connecting to Okta. The incident appeared to resolve for most users approximately 30 minutes later at around 1:40 p.m. EST. Click here for an interactive view.
Updated Jan. 30
Global outages across all three categories last week increased from 245 to 373, up 52% over the week prior. In the US, they jumped from 57 to 102, up 79%.
Globally, ISP outages increased from 187 to 278, up 49%, and in the us they increased from 42 to 81, up 93%.
Globally, cloud-provider network outages increased from six to 10 outages, while in the US decreased from three to two.
Globally, collaboration-app network outages increased from one to four, and in the US they remained the same at one.
There were two significant outages.