Built for power

NFL Films San Diego Supercomputer Center TeraGrid Kodak Theatre

Phil Andrews is no stranger to heavy lifting. To prove it, he'll show you the latest rankings from Powerlifting USA magazine, which ranks him No. 2 in the U.S., after hefting 837 pounds at a competition in Las Vegas.

Small wonder, then, that the National Science Foundation (NSF) asked this guy to oversee more than 500 terabytes of storage capacity for TeraGrid, which will be the fastest, most computationally rich research network in the world when it goes live early next year. Andrews clearly knows how to deal in big volume, whether it's in the gym or inside the San Diego Supercomputer Center (SDSC) where he works, on the campus of the University of California at San Diego.

In addition to the supercomputing center in San Diego and one in Pittsburgh, TeraGrid connects two sites in Illinois - the National Center for Supercomputing Applications in downstate Champaign and Argonne National Laboratory, outside of Chicago in Argonne. It also connects to the Center for Advanced Computing Research at the California Institute of Technology in Pasadena.

The TeraGrid backbone between Internet hubs in Chicago and Los Angeles screams along at 40G bit/sec, with links from those two main nodes to the five sites operating at 30G bit/sec. Fiber capacity from Qwest and gigabit routers from Juniper make up the major pieces of the TeraGrid network.

Massively large data

TeraGrid will deliver access to 20 teraflops (trillion floating point operations per second) of computing power, facilities capable of managing and storing nearly 1 petabyte (1 quadrillion bytes) of data, high-resolution visualization environments and toolkits for grid computing. Researchers will tap the TeraGrid to run applications such as biomedicine, global climate and astrophysics research.

"This will move scientific research into areas it hasn't gone before," Andrews explains, because scientific modeling data can be massive.

Data is coming from a variety of sources such as microscopes, telescopes and sensors, all generating terabytes of data. The NSF's National Virtual Observatory already keeps its digitized map of the night sky at SDSC.

SDSC's role is to "crunch and store," Andrews says, meaning his organization will provide processing power and data storage. Inside TeraGrid's storage network at SDSC, three Brocade Communications SilkWorm 12000 switches will sit in front of the 500 terabytes of data. Using Fibre Channel over IP, SDSC should be able to write 10G byte/sec, while its tape drives will write at 1G byte/sec. That's at least one exponent above most storage systems.

Before now, computing grids were overlaid on existing networks. But because TeraGrid is being built specifically for grid computing, developers can ensure all components are appropriate and as powerful as budgets will allow, Andrews says. He predicts that this sort of distributed processing and storage across the WAN will be the model for future networks, whether research or corporate. "The future of grid computing is in providing these kinds of data services. We just have to figure out the best ways to ship so much data across the network," he says.