The fact that Purdue was able to build its supercomputer by lunchtime Tuesday is pretty incredible. But the network equipment used is pretty interesting as well.
Purdue bills the Coates Cluster as “the first internationally ranked academic supercomputer that is wired solely by superfast ten-gigabit network connections.” It is using those 10 Gigabit Ethernet connections as a “unified wire,” where all communications into and out of each node – including storage, networking, clustering, management and boot traffic - happen on a single 10G Ethernet connection.
Chelsio notes that each node, therefore, has only the 10G Ethernet and the power cord connecting to it, (what, no Power over Ethernet?) which no doubt helped the university get its supercomputer together in such a hurry.
Each of the 1,280 AMD-based HP dual quad-core compute nodes was paired with a Chelsio Communications iWARP RDMA adapter. The technology used in the adapters can offload a lot of the communications processing so that the CPU doesn’t have to do it. Chelsio says the performance is comparable to that of InfiniBand.
The adapters were connected to Cisco Nexus switches. Matrix Integration provided the nodes and Verizon Business Network Services provided the integration services for the switches.
The next ranking of supercomputers worldwide happens in the fall, and Purdue expects the Coates supercomputer to make the top 50 with its 90 teraflops of performance. Purdue notes that the supercomputer it built last year was initially ranked 105th in the world, but has since fallen to 196th, which gives you an idea of how quickly the state of the art is advancing.
Purdue expects to use the new supercomputer for research into climate modeling and weather forecasting.