Gigabit Ethernet dominates supercomputer line-up

Gigabit Ethernet is the interconnect of choice for a majority of the top 500 supercomputers in the world, according to the latest list from Top500.org.

Supercomputers are most often collections of thousands of individual processors working together. Obviously, those many processors need to communicate with one another, and there are various technologies they could use.

The primary alternative is InfiniBand, though there are many other techniques used by the computers on the list.

The interconnect used by the top computer on the list is a mix of both InfiniBand and Gigabit Ethernet. The so-called IBM Roadrunner is the world's first machine to compute in petaflops, a thousand trillion calculations per second. The exact speed of Roadrunner is 1.5 petaflops.

If that isn't enough to blow your mind, consider this statistic: Roadrunner has "10,000 InfiniBand and Gigabit Ethernet connections requiring 57 miles of fiber optic cable," as this slideshow notes. 

The Top500 list indicates that Gigabit Ethernet is used in 285 - that is, 57% - of the 500 top systems, while InfiniBand is used in 120. The next-highest category is "proprietary," with 40 systems going that route.

That's the highest number ever for Gigabit Ethernet. A few years ago Myrinet was the interconnect of choice; now a mere dozen systems out of the top 500 use it.

Gigabit Ethernet has speed going for it, of course, but it also has low cost and the upgradeability to 10 Gigabit Ethernet.

The vast majority of the systems have between 1,025 and 4,096 processors; Roadrunner uses 6,948 dual-core AMD Opteron chips and 12,960 Cell engines.

Jeff Caruso is site editor at Network World.