* Q&A with Energy Sciences Network officials
I recently had the opportunity to interview U.S. Department of Energy officials responsible for the Energy Sciences Network, one of the fastest IP networks in the world and a pioneer of IP technology.
ESnet is a nationwide IP backbone linking Energy Department’s headquarters in Gaithersburg, Md. with more than 40 sites including many national laboratories. ESnet’s ring-shaped backbone network operates at 10G bit/sec in the northern states and 2.5G bit/sec in the southern states.
In the next two issues of the ISP News Report, I’ll provide excerpts from my conversation with Mary Ann Scott, ESnet program manager, and Daniel Hitchcock, a senior technical advisor on Esnet, about the challenges of keeping one step ahead of a demanding user base.
Carolyn Marsan: What can you tell me about the user base of ESnet?
Dan Hitchcock: Every scientist in the world who accesses an Energy Department facility is an ESnet user at some level. Every major university in the country has bits going over ESnet. The number of ESnet users is somewhere between the number of employees in our laboratories – which is tens of thousands of people – and the number of people on the global Internet. This is not a closed system.
CM: Given that you have such a broad user base, how do you handle security?
DH: The ESnet architecture is designed so that at all of our peering points, we have monitoring. At each of our 42 sites, ESnet has a router on one side and the site has a router on the other side. The only thing between these two routers is one connection. So the site can filter anything that comes in from ESnet, and ESnet can filter anything coming out from the site. We manage all of this centrally. So if something bad happens, ESnet can shut off a particular site. Or if the Internet is doing something strange, ESnet can isolate itself from the Internet to enable the laboratories to continue to talk to each other. And we monitor this 24/7.
CM: What are the biggest security threats you see?
DH: ESnet is just a carrier, so ESnet doesn’t get attacked as much. But the sites at the ends get attacked, and ESnet has to help them respond and protect themselves. We’ve done some work in public key infrastructure for science to help us try to manage security, but this is just hard. With viruses and system vulnerabilities, we don’t control the end points.
CM: How would you distinguish your user base in terms of what they demand out of the network?
DH: Our users demand the ability to transmit huge volumes of data rapidly and in big chunks. It’s not like it’s one Web page at a time. They may want to transmit a 5T-byte file.
CM: What is the main challenge you face in supporting the ESnet user community?
Mary Ann Scott: Keeping up with their needs to move large data files. At the moment, all we have is pure bandwidth. But we see the need in the future to have various kinds of QoS. End-to-end performance is very important to the scientists.
CM: Why is end-to-end performance such a challenge?
DH: The average packet on ESnet going end-to-end crosses between seven and 12 networks. So getting end-to-end performance is not just having big pipes somewhere. The routers have to be configured right, and the software and systems on the ends have to be configured properly, so the performance actually happens. That turns out to still be a black art.
CM: Your main backbone and access contract with Qwest expires next year. What will you do then?
DH: The detailed management and day-to-day operations of ESnet is handled through our contract with the University of California, which operates Lawrence Berkeley Laboratory. They will award a competitive contract. We think we will need two 40G-bit/sec rings by the end of the decade, with [Metro-Area Networks] in the 10G- to 20G-bit/sec range in each city where we have major laboratories. That’s how much bandwidth we will need just to do the science.
CM: You recently awarded a contract for a high-speed MAN in San Francisco to Qwest. Will you run separate contracts for the rest of the MANs?
DH: Each one will be bid out separately because in each case there are different carriers with dark fiber in the ground. Our major contract is with Qwest, but we have 40 other contracts with telecom vendors for various parts of the ESnet infrastructure.
CM: How would you describe your relationship with Qwest?
DH: The subcontract is between the University of California and Qwest, so we’re not directly involved. We look at how Lawrence Berkley manages their subcontracts, and we have people out there at the site making sure they’re acting legally. And we look for contingency plans. They’re supposed to give them to us if they thought Qwest was in significant financial trouble. We have to make sure we can continue to have operations.
CM: So the whole situation of Qwest maybe buying MCI, you’re hands-off on that?
DH: It is not an issue for us. We’re not in the middle of that. Remember that ESnet doesn’t support the business operations of the federal government. ESnet supports scientific research. So it’s not like we’re carrying accounting data. We just do science.
CM: What’s the biggest lesson you’ve learned from ESnet?
DH: The biggest lesson we’ve learned from ESnet is that you have to have the right sociology between the networks and the people at the ends in order to make this happen. If you want end-to-end performance, you have to be willing to make the investment in money and people to make that happen. You have to have a way for people to talk frequently.
Tune in next week to learn about the latest IP technologies that ESnet is deploying.
