Skip Links

Medical research group skips 40G, makes 'right move' to 100G Ethernet

Howard Hughes Medical Institute upgrades core Ethernet network to boost performance, eliminate Spanning Tree

By , Network World
November 15, 2011 02:12 PM ET

Network World - Howard Hughes Medical Institute (HHMI), a nonprofit medical research organization, has implemented a 100G Ethernet network to boost bandwidth for advanced data analysis.

The institute swapped out a Force10 network in favor of Brocade MLXe routers. Fifty-six ports in HHMI's two-router core run 100G Ethernet, making HHMI's Janelia Farm Research Campus the largest single site deployment of 100G Ethernet in a research institute, according to Brocade.

The network will be used by 250 scientists in Janelia Farm. Applications include data analysis for neuroscience and imaging research.

IN DEPTH: 100G Ethernet: Bridge to Terabit Ethernet 

HHMI implemented the 100G network the first weekend in September. The 10G Force10 E1200 network had been deployed in March 2006, says Spartaco Cicerchia, director of network infrastructure systems for the Janelia Farm Research Campus in Ashburn, Va.

"We have a five-year benchmark for life cycle of gear itself," Cicerchia says. "Looking at the challenges we could potentially face for the next three to five years is large generation and movement of data. There is a large amount of data that is constantly moving. In order for us to be as transparent as possible, we needed to deploy a network that was capable of producing no bottleneck and very transparent between data generation, storage and movement."

HHMI felt 40G Ethernet was not going to be a large enough bandwidth leap and would not give it enough capacity for the next three to five years, Cicerchia says.

"Going straight to 100G would prove to be the right move 36 months from now," he says.

The Janelia network has 200Gbps connections out to each of its wiring closets and 10G Ethernet directly to researchers' systems. From the core, Janelia Farm Research Campus aggregates, or bonds, two ports of 100G Ethernet to create a single 200-gigabit logical connection, allowing line rate bandwidth in both directions.

Previously, Janelia bonded together multiple 10G links to achieve 20G to 40G connections from core to closet, Cicerchia says.

At the aggregation layer, two fully populated Brocade MLXe-32 routers support more than 2,400 Gigabit Ethernet ports and a large subset of 10G Ethernet ports. The core and desktop connections are 10 times that of the previous network, which was 10G at the core and Gigabit Ethernet to the workstations.

The Brocade routers - MLXe32s and MLXe16s - are configured using Brocade's Multi-Chassis Trunk (MCT) capability, in which all paths between the routers are active for increased resiliency. This is designed to overcome the traditional active-passive redundancy of Spanning Tree. MCT lets two MLXe router chassis operate as a single fully redundant active/active logical router with no Spanning Tree overhead.

"We wanted to be Spanning Tree-free," Cicerchia says. "It can be challenging at times in large networks. [In Spanning Tree] you basically have 50% of your ports in non-forwarding mode. We wanted to have all of our links up."

MCT, though, produced the only glitch HHMI encountered during the 10G-to-100G cutover. There were inconsistencies in the way legacy Force10 switches at the top-of-rack performed IEEE 802.1w - rapid reconfiguration of Spanning Tree - and the way Brocade implemented it while simultaneously running MCT.

Our Commenting Policies
Latest News
rssRss Feed
View more Latest News