YouTube outage underscores big Internet problem

Sunday's inadvertent disruption of Google's YouTube video service underscores a flaw in the Internet's design that could some day lead to a serious security problem, according to networking experts.

The issue lies in the way ISPs share Border Gateway Protocol (BGP) routing information. BGP is the standard protocol used by routers to find computers on the Internet, but there is a lot of BGP routing data available. To simplify things, ISPs share this kind of information among each other.

And that can cause problems when one ISP shares bad data with the rest of the Internet.

That's what happened with YouTube this weekend, according to sources familiar with the situation. BGP data intended to block access to YouTube within Pakistan was accidentally broadcast to other service providers, causing a widespread YouTube outage.

The chain of events that led to YouTube's partial black-out was kicked off Friday when the Pakistan Telecommunication Authority (PTA) ordered the country's ISPs to block access to YouTube because of an alleged anti-Islamic video that was hosted on the site.

According to published reports, the clip was from a film made by Geert Wilders, a Dutch politician who has been critical of Islam. Wilders is hoping to air a 15 minute anti-Islam film, called Fitna on Dutch television in March.

ISPs in Pakistan were able to block YouTube by creating BGP data that redirected routers looking for YouTube.com's servers to nonexistent network destinations. But that data was accidentally shared with Hong Kong's PCCW, who in turn shared it with other ISPs throughout the Internet.

In San Francisco, David Ulevitch first noticed the problem Saturday morning. "I was trying to watch cats falling off roofs... and I couldn't get to YouTube," he said. Ulevitch, who runs an Internet infrastructure company called OpenDNS, was soon able to connect with engineers at Google, who also experienced similar problems, he said. "They were like, 'Holy crap, we can't get to YouTube either.'"

Because Pakistan's BGP traffic was offering very precise routes to what it claimed were YouTube's Internet servers, routers took it to be more accurate than YouTube's own information about itself.

Larger service providers typically validate BGP data from their customers to make sure that the routing information is accurate, but in this case, PCCW apparently did not do that, according to Ulevitch. When the Pakistani ISP sent the bad data, PCCW ended up sharing it with other ISPs around the globe.

This kind of accidental denial of service attack has happened before. In early 2006, for example, New York's Con Edison caused data intended for a number of networks to be misrouted following a similar mistake.

There wasn't anything that Google could have done to prevent the problem, said Danny McPherson, chief research officer with Arbor Networks. "They can't keep someone on the Internet from announcing their address space," he said. "It's a huge vulnerability."

By intentionally propagating bad BGP data, an attacker could knock a Web site off the Internet or even redirect visitor's traffic to a malicious server, security experts said.

The IDG News Service is a Network World affiliate.