Los Alamos builds big InfiniBand cluster

LOS ALMOS, N.M. - Los Alamos National Laboratory has turned to server maker Promicro Systems to build what it says is the largest cluster that uses the fledgling InfiniBand interconnect.

Los Alamos recently said it was building a 128-processor cluster that includes Promicro servers running on Intel's Xeon chips and Linux. The government lab picked InfiniBand for the high-speed interconnect between servers, as it looks to test the technology for possible use in larger systems. InfiniBand provides a high-bandwidth, low-latency interconnect between systems.

The use of InfiniBand marks an effort by Los Alamos to shy away from proprietary interconnects from Myricom and Quadrics, says Steve Tenbrink, a group leader of network engineering group at Los Alamos.

"We are trying to push open standards for interconnects where we can," Tenbrink says.

This new system, equipped with faster Xeon chips, joins a growing family of Beowulf Linux clusters at the laboratory. Los Alamos used low-power Transmeta processors in an RLX Technologies-based server blade cluster to test ways to lower cooling costs and raise the stability associated with large computers.

While Los Alamos has yet to get the Xeon-based cluster up and running, it plans to use the system to run some of its nuclear simulation software. But it will depend on how well InfiniBand works with a large number of computers, Tenbrink says.

"The problem is that as you scale higher and higher, interconnect performance tends to get worse and worse. You really have to be careful how you address that problem," Tenbrink says.

Los Alamos expects to have the new system up and running in a few months.

Vance is a correspondent with the IDG News Service's San Francisco bureau.