A day in the life of the Internet: Chicago

CHICAGO, JANUARY 19 -- If you were pulled off the streets of downtown Chicago and plunked down in the middle of Ameritech's Chicago NAP, you'd most likely say to yourself: ''So what's the big deal?''

Stay for five minutes and you'd probably be thinking: ''This is boring. When can I leave?''

But probe behind the scene at the Chicago NAP and your interest would surely perk up. The quiet, stark surroundings belie the fact that this is one of the busiest spots on the Internet. The NAP's three Ascend ATM switches hum just as quietly as any other switch you've ever heard, but this gear pumps some 6Tbytes of data through the ‘Net daily.

Even at the slowest time of day – 4 a.m. central standard time – the Chicago NAP typically handles an astounding 175G bytes of data. Traffic volume climbs steadily throughout the day, with the heaviest loads between 10 a.m. and 7 p.m. On Jan. 15, for example, volume peaked at 2 p.m., with 312G bytes passing through the exchange. By 11 p.m., the load had dropped to 236G bytes.

The Chicago NAP hasn't always handled such incredible volume, but daily traffic patterns have been consistent since the exchange opened four years ago. The skyrocketing volume – traffic load doubled from three terabytes in early 1998 to the six terabytes of today – still fascinates Andy Schmidt, product manager for Ameritech's NAP service and author of three Internet-related books.

As part of his daily routine, Schmidt spends a good chunk of time, often from his home office while wearing his ''jammies,'' running reports for and talking to NAP customers. Prior to heading to the NAP today, for example, Schmidt was having lunch with the chief information officer for the University of Chicago, one of the big universities connecting to the NAP as part of the Midwest Research and Education Network (MREN). (He donned a casual suit for the occasion.)

Because of its MREN support, the Chicago NAP also serves as an interconnection point for international research organizations that want access to U.S. educational networks. And it has become an exchange point for the Department of Defense, NASA, the Department of Energy, the National Institute of Health and other U.S. governmental networks.

But the bulk of connections, of course, come from the ISP community. The number of ISP connections stands at about 60 today, but grows by about four per month, Schmidt says.

That makes for a lot of work provisioning circuits, theoretically making the activity level at the NAP comparable to the bustling streets just three floors below. That's not the case, though.

Most days, the NAP looks more like a ghost town, with rows and rows of switches and Sonet multiplexers, only a mere fraction dedicated for the NAP service, humming away. Aside from a clunky metal desk and a couple of beat up chairs, there's no sign of human inhabitance. The desktop is barren, the walls undecorated.

That's because most of the circuits are provisioned remotely from an Ameritech data center in Southfield, Mich., and the switches are monitored at the company's network operations center (NOC) in Hoffman Estates, Ill. That makes a cast of thousands involved in the NAP, even though only eight sales, support and operational people are dedicated to the service.

Even though it's not his home base, Kevin Peterson, operations manager, can usually be found at the NAP testing circuits – but only during normal business hours. After hours, NOC engineers handle NAP operations. If an emergency crops up, they can page Peterson or one of the other operations managers. .

But neither Peterson nor Schmidt can really recall such an event. The vast majority of the ISPs and other organizations using the NAP are great about upgrading their connections and engineering their lines so congestion doesn't occur, says Schmidt, pulling out a printed report to prove his point.

Charts show that 47% of the time, most NAP connections fall in the 80% utilization band. ''We maintain this type of trending information from the minute a circuit is turned on so we can see right away when a port is going to saturate.''

To further prove his point, Schmidt heads down a row of switches to the workstation running the H-P OpenView graphing program used to count ATM cells passing through the exchange. A quick screen scan proves Schmidt's point. Of the 79 ATM switch ports, only two are losing any cells due to congestion.

And, given that 96,000 cells pass through a DS-3 connection each second, the cell loss shown translates into just a handful of packets, Peterson points out. ''When you think about the number of terabytes of data going through, that's nothing.''

More often than not, congestion occurs because an ISP's peer sends it too much – say a full DS-3's worth – traffic, say Schmidt and Peterson, both of whom proudly assert that ATM is the key to keeping their NAP congestion-free.

Schmidt isn't privy to peering agreements among those organizations connecting to the NAP. Ameritech simply uses the ATM infrastructure to create a full mesh of permanent virtual circuits (PVC) over which any ISP, university, research organization or government agency can connect to any other NAP customer. Peering can be cut off by request.

The Chicago exchange is the largest ATM-based NAP in the world. ATM's scalability makes it easy to handle the constant upgrades in the number of ports and port speeds required at the NAP – and that ability makes up for the heat Ameritech got four to five years ago for going the ATM route, Schmidt says.

Most of Ameritech's NAP customers have outgrown their original DS-3 connections and are now accessing the exchange point at the OC-3 rate of 155 Mbit/sec. And on Nov. 1, 1998, the Chicago NAP became the first to offer access at the Sonet rate of OC-12, or 622 Mbit/sec, to meet the needs of various ISPs and MREN. ''I don't know of any other exchange that supports that now,'' Schmidt says proudly.

Schmidt looks forward to the Chicago NAP becoming a major hub that provides connectivity not only among the existing types of customers but also big corporate users as well. It's inevitable that corporate customers begin buying Internet access directly through the NAP, Schmidt says. ''We're starting to cross the threshold where a different business model makes sense.''

Read the stories of individual NAPs by clicking on their city name below:

Chicago NAP overview

Forum: The state of the 'Net
Discuss it with Dan Lasater, director of broadband applications and MAE architect for MCI WorldCom.