Weekly internet health check, US and worldwide

ThousandEyes, which tracks internet and cloud traffic, provides Network World with weekly updates on the performance of three categories of service provider: ISP, cloud provider, UCaaS

1 2 3 4 5 Page 5
Page 5 of 5
  • “Cogent Communications suffered yet another significant outage this week — its fifth major outage this month. The outage occurred within parts of Cogent’s network in Northern California and Oregon and impacted users connecting to sites and services in those regions, including projectbaseline.com, the website of Verily’s much-publicized COVID-19 testing program.”
  • ”For approximately 20 minutes on March 25th, ThousandEyes observed that some users located on the East Coast may not have been able to reach Google services due to 100% traffic loss. A short time later, Google’s SVP of Engineering tweeted that the incident was due to a router failure in Atlanta, Georgia. US users outside of the Northeast were also impacted intermittently, although they would have experienced the incident as site errors when trying to reach some Google sites, such as google.com. The HTTP server errors seen during this period are consistent with an inability to reach the backend systems necessary to correctly load various services. Any traffic traversing the affected region — connecting from Google’s front-end servers to backend services — may have been impacted and seen the resulting server errors.”

With the increased use of remote-access VPNs, major carriers are reporting dramatic increases in their network traffic – with Verizon reporting a 20% week-over-week increase, and Vodafone reporting an increase of 50%.

While there has been no corresponding spike in outages in service provider networks, over the past six weeks there has been a steady increase in outages across multiple provider types both worldwide and in the U.S., all according to ThousandEyes, which keeps track of internet and cloud traffic.

This includes “a concerning upward trajectory” since the beginning of March of ISP outages worldwide that coincides with the spread of COVID-19, according to a ThousandEyes blog by Angelique Medina, the company’s director of product marketing. ISP outages worldwide hovered around 150 per week between Feb. 10 and March 19, but then increased to between just under 200 and about 225 during the following three weeks.

In the U.S. those numbers were a little over 50 in the first time range and reaching about 100 during the first week of March. “That early March level has been mostly sustained over the last couple of weeks,” Medina writes.

Cogent Communications was one ISP with nearly identical large scale outages on March 11 and March 18, with “disruptions for the fairly lengthy period (by Internet standards) of 30 minutes,” she wrote.

Hurricane Electric suffered an outage March 20 that was less extensive and shorter than Cogent’s but included smaller disruptions that altogether affected hundreds of sites and services, she wrote.

Public-cloud provider networks have withstood the effects of COVID-19 well, with slight increases in the number of outages in the U.S., but otherwise relatively level around the world. The possible reason: “Major public cloud providers, such as AWS, Microsoft Azure, and Google Cloud, have built massive global networks that are incredibly well-equipped to handle traffic surges,” Medina wrote. And when these networks do have major outages it’s due to routing or infrastructure state changes, not traffic congestion.

Some providers of collaboration applications – the likes of Zoom, Webex, MSFT Teams, RingCentral – also experienced performance problems between March 9 and March 20. ThousandEyes doesn’t name them, but does list performance numbers for what it describes “the top three” UCaaS providers. One actually showed improvements in availability, latency, packet loss and jitter. The other two “showed minimal (in the grand scheme of things) degradations on all fronts — not surprising given the unprecedented strain they’ve been under,” according to the blog.

Each provider showed spikes in traffic loss over the time period that ranged from less than 1% to more than 4% in one case. In the case of one provider, “outages within its own network spiked last week, meaning that the network issues impacting users were taking place on infrastructure managed by the provider versus an external ISP.”

“Outage incidents within large UCaaS provider networks are fairly infrequent; however, the recent massive surge in usage is clearly stressing current design limits. Capacity is reportedly being added across the board to meet new service demands,” according to the blog.

Meanwhile, ThousandEyes has introduced a new feature on its site a Global Internet Outages Map that is updated every few minutes. It shows recent and ongoing outages

Google outage unrelated to COVID-19

On March 26 Google suffered a 20 minute outage on the East Coast of the U.S., apparently from a router failure in Atlanta, ThousandEyes said, agreeing with a statement put out by Googe to that effect.

That problem affected other regions of the U.S. as evidenced by Google sites such as google.com intermittently returning server errors."These 500 server errors are consistent with an inability to reach the backend systems necessary to correctly load various services," ThousandEyes said in a statement. "Any traffic traversing the affected region — connecting from Google’s front-end servers to backend services — may have been impacted and seen the resulting server errors."

ThousandEyes posted interactive results of tests it ran about the outage here and here.


Join the Network World communities on Facebook and LinkedIn to comment on topics that are top of mind.

Copyright © 2021 IDG Communications, Inc.

1 2 3 4 5 Page 5
Page 5 of 5
IT Salary Survey: The results are in