On the grid at Sun

I visited Sun’s Austin Compute Ranch last week and learned a little more about grid computing.

Sun’s Austin facility is just one of three “compute ranches” the company uses – the others are in Sunnyvale, Calif., and Burlington, Mass. The Sunnyvale compute ranch is the largest of Sun’s server collections, with 5,000 CPUs. At the Austin Compute Ranch, Sun has 2,300 CPUs; at Burlington, 1,200.

The compute ranches each have grid software to interconnect the CPUs, and they’re used by Sun programmers and engineers to develop the UltraSPARC processor. The grids run Sun’s own electronic design automation applications to lay out chips, perform what-if scenarios on them, perform modeling and do verification on them. Austin’s grid processes over 1 million jobs per month.

While there are 300 employees at Sun’s Austin Compute Ranch, it serves employees throughout Sun, who submit jobs to a ranch based on the type of data they want to use or analyze.

Developers submit a job, and a resource-management application schedules the job, allocates computer resources and executes it, returning the results to the engineers on completion. Unused compute cycles are ganged together to perform jobs.

Sun’s Austin facility is stocked with SunBlade 1000 and 2000, Sun Fire V3800 and V4800, UltraSparc 60 and Enterprise 4500 computers, and T3, as well as A5x00, storage devices connected with Gigabit Ethernet. Across the three sites there are as many as 50 high-availability clusters. I didn’t see any Sun Fire V1280s, Sun’s newest upcoming V Series server. 

The compute ranches hold more than 410 terabytes of data, with machines running at 98% utilization 24 hours a day, 365 days a year.

Sun claims that as much 15 years of single-processor computing can be accomplished each day at its compute ranches. The company says one of its “power users” ran 1.2 million jobs in one month – for a total of 650,000 hours, or 75 years, of computer time. The company consumes over 300 years of computer time a month, or 10 years per day.