TACC’s 300-node cluster is just the beginning

Recently I visited the Texas Advanced Computing Center at the University of Texas, where there are four clusters installed and running a variety of compute-intensive applications.

Among the clusters is a 300-node cluster of Dell PowerEdge servers, connected with a Myricom Myrinet 2000 fabric switch and 42 Dell PowerConnect switches. The Myrinet switch has a bandwidth of 250M bit/sec.

Each server in the cluster runs Linux, and they were chosen because they are inexpensive to deploy and own, says Chris Hempel, TACC associate director. Dubbed Lonestar, the cluster consists of two PowerEdge 2650s that function as master nodes, where users log in; 16 servers that perform I/O; and 282 PowerEdge 1750 servers acting as compute nodes. All the servers in the cluster have dual Xeon processors running at 3.06 GHz, operating collectively at 3.67 teraFLOPS (trillions of floating point operations per second). The total disk space is 39 terabytes.

The cluster will be used to predict weather changes and locate petroleum reserves.

TACC expects to add 200 more servers to the cluster in the next year and has designed the power and cooling in its data center to support the cluster.

Hempel also has a cluster of IBM pSeries servers that run AIX and use IBM’s Power4 processor. They are interconnected with IBM’s Colony switch. The Power4 cluster has 224 processors each running at 1.3 GHz, and 500G bytes of memory across four symmetrical multiprocessing nodes. The cluster consists of three IBM p690 servers, one p690 node that shares memory with the others and 32 p655 server nodes.  Each of the nodes is connected to a p690 logon node.

In addition, Hempel has installed a 64-processor Intel Pentium III cluster running Linux that operates at 64 gigaFLOPS. Each server runs at 1 GHz. They are connected with a 100Base-T Cisco Catalyst switch for network services and a Myrinet 2000 switch that passes messages among the nodes.