Routers/Switches /

Genome Project meets Gigabit Ethernet

Incyte fashions high-speed network to support historic scientific endeavor.

By Jim Duffy
Network World, 09/18/00

PALO ALTO - When completed this summer, a "working draft" of the human genome was hailed as a major milestone in the efforts to combat illness and disease by better understanding the genetic makeup of the human body.

For one company, the project also provided insight into how a fully redundant, meshed Gigabit Ethernet network can hold up through the strain of decoding strands of human DNA.

Incyte Genomics relied on its Gigabit Ethernet network to provide DNA sequencing information for the Human Genome Project as well as for private medical and pharmaceutical companies. This data is useful not only for information on the human genetic code, but also for research on drugs to better treat ailments and afflictions.

"We were one of the large players doing a lot of the decoding," says Phil Kwan, associate director of network infrastructure at Incyte Genomics. "We believe [the human genome] should be completely uncovered and be made available to everybody to start using this information to produce a better drug."

Incyte's network transports between 6 and 8.5 terabytes of data daily in a configuration that supports multigigabit links between switches. The network consists of 88 Foundry Networks routers and switches with 2,000 10/100M bit/sec Layer 2 Ethernet ports and up to 560 Gigabit Ethernet ports. Between 150 and 200 of the Gigabit Ethernet ports are fully routed using Open Shortest Path First.

The Foundry switches support high-speed access and data retrieval to and from a 1,200- to 1,300-strong server farm, which includes one of the biggest Linux farms in the world, with 850 to 900 Linux machines in clusters, Kwan says.

Incyte also has 350 to 400 Sun, Silicon Graphics and Digital Equipment servers, as well as 75 to 100 Windows NT servers and a smattering of Apple Macintoshes.

All links and devices in the Incyte network are redundant.

Gigabit links are logically grouped into multigigabit trunks using Foundry's multilink trunking feature and IP load balancing. The smallest multilink trunk is 2G bit/sec, and the largest is between 6G and 8G bit/sec.

This configuration not only ensures reliability, but also balances loads across multiple paths for better performance, Kwan says.

"Because of this design, we haven't had a network outage that's affected the backbone for over three years," he says. "The design helped a lot, the OSPF helped a lot, but also the gigabit redundancy really did its job."

Applications on the network perform sequencing, raw data analysis, high-volume data transfers, and product and software development. Application quality of service (QoS) is based on some unsophisticated queuing, Kwan says. The network supports two or three levels of priority with the high-bandwidth servers having the highest-priority QoS, he says.

Once the information is sequenced, it gets put into a database that is sold to large pharmaceutical companies for target drug research. Some of this data is also available freely for the genome project work.

Incyte last month licensed its database and gene-patent portfolio to Motorola so that the company could develop "biochips" that contain data on biological samples of genes.

Incyte has been sequencing genetic information for about four years. Initially, its network was based on 100M bit/sec Fast Ethernet and FDDI.

The firm began its upgrade to Gigabit Ethernet three years ago to keep its network from collapsing under the weight of the bandwidth-intensive sequencing data.

"One terabyte to 1.5 terabytes of information was being pushed around three and a half years ago, and our older legacy network was buckling an average of three to four times per month," Kwan says. "Too much traffic was congesting the backplanes of our switches and routers."

Routing topology updates would trigger management modules on Incyte's switches and routers to shut down transmission of that information during network rediscovery. That shutdown would cause a lot of the sessions on Incyte's Unix server farm to hang or to die, Kwan says.

"That was a very costly adventure," he says.

Incyte upgraded to Gigabit Ethernet after a long study comparing that technology with others, namely ATM and High-Performance Parallel Interface (HIPPI). The high cost of ATM and the limited vendor support for HIPPI convinced Incyte to go Gigabit Ethernet.

"We decided Gigabit Ethernet looked like a very positive upcoming standard because so many vendors were behind it," Kwan says. "Three and a half years ago, ATM was very costly. We couldn't achieve the benefits down to the server farm unless we put all of our servers on ATM.