HPC Spinoffs

HPC features trickle down to regular IT.

Big Linux systems plot climate change, simulate nuclear explosions, and secure bragging rights. But IT customers are starting to find that high-performance computing technologies make a difference in the real world, from clustered processing to data center greening.

High Performance Computing (HPC) has long been the watchword of supercomputing centers, where huge government and university research centers, such as the Lawrence Livermore and Sandia national laboratories, use Linux to process huge data sets. Now, HPC spinoff technnology is arriving in the enterprise and SMB Linux markets.

Just as the U.S. space program afforded such innovations as scratch resistant sunglasses, all-weather winter radial tire for cars, and equipment for hospitals to monitor patients' vital signs, so has HPC been the source of innovation for smaller scale architectures. The technology used to run HPC is spilling over to mainstream IT, reaching smaller entities and companies with a much smaller scale than Livermore use or Sandia National Laboratory might have. Enterprise and SMB market have an abundance of resources to draw from traditional HPC computing.

"Today, many more organizations are able to take advantage of High Performance Computing, due to the ready availability of inexpensive compute clusters powered by Linux running on off-the-shelf x86 hardware, as opposed to the proprietary hardware and software of yesterday’s supercomputers," says Sam Charrington, Vice President of Product Management and Marketing for Appistry, Inc. The accessibility and availability of HPC technology is a big driver for scaled down markets, and so is another consideration: physical environmental controls.

Bill Thirsk, CIO of Marist College in Poughkeepsie, New York, believes that innovations from HPC that avail themselves to mainstream SMB and enterprise IT environments are many: in physical environmental controls, efficient processors, scalability with lower power consumption, and less need for network fabric.

An emerging topic of concern for today's CIO is a trend to a "green" data center environment and innovations developed from HPC are aiding this direction.

"The 'greening' of the datacenter was once a good idea to save a little money," says Thirsk. "It is now an imperative. HPC cooling and environment control technologies are leading the way."

Thirsk points out that with a server farm or blade center, many independent power supplies are running equipment that is not running at capacity, and they are all generating heat. "The supercomputers and mainframes have evolved to include integrated cooling, smaller footprints, more efficient power allocation for CPUs, and consolidated network connections, making the need for multiple network wires running back to multiple switches and other network gear obsolete," he explains. "These technologies are finding their way to the smaller capacity servers, network gear, and most notably self-cooling network racks."

Justin King, systems administrator for the Human Neuroimaging Laboratory at Houston, Texas based Baylor College of Medicine in the Department of Neuroscience concurs with Thirsk that advances in power efficiency developed in a large-scale HPC environment are available to mainstream IT computing environments. "Machines are insanely powerful now and customers are demanding improvements in energy consumption," says King. "We're currently deploying a cluster which has 16 cores and 32GB RAM in 1U. In 1.7" of rack height you have as much power now as you did in 5-8U just 4 years ago. With the density now, everyone is noticing lots of spare CPU cycles—hence the move towards virtualization. With that said, I think we are starting to see the move towards utility-on-demand and cloud computing, and I believe that trend will continue. Given an abstract problem and the ability to scale out as needed, I think the power and cooling issue will present less of an issue as adding compute power to an application will be as simple as adding a few more servers to the cloud."

"The HPC market is where vendors test out the ideas that will drive tomorrow's commercial products,” says Meike Chabowski, Technical Product Marketing Manager at SUSE Products, Novell. "The current generation of servers from all of the major vendors —IBM, Sun Microsystems, HP, Fujitsu [and] Siemens —may look on the outside like vanilla symmetric multiprocessing (SMP) servers, but under the covers, they more closely resemble the parallel machines favored by supercomputer buyers today."

According to Alanna Dwyer, marketing manager for high-performance computer solutions at HP, HPC technology has enabled industry users to achieve the highest efficiency and utilization rates for their systems. This development coupled with accessibility, ease of use, and affordable hardware and software has prompted SMBs and smaller workgroups to get onboard with some of the same tools and methods that the big players use.

"With higher utilization on their servers, the enterprise/SMB market can benefit from the Linux tools and methodologies developed in HPC for managing groups of systems in light-weight, efficient manner, and also from storage management solutions to enable higher I/O and file capacity required as utilization grows," says Dwyer. "With new and innovative high-performance computing hitting the market on a constant basis, Enterprise/SMB customers are being forced to learn how to best utilize HPC technologies to better their business outcomes."

These same tools are also being used on a smaller scale to design, test and analyze new product concepts.

"These companies are trying to accelerate their time to market with more innovative and reliable products," says Len Rosenthal, Chief Marketing Officer at Panasas, who provides a storage system for scalable Linux clusters. "Financial firms are using HPC to determine trading strategies and analyze risk to improve their profitability. Media and Internet companies are using HPC technology to scale their file serving infrastructure to handle massive user demand."

Chabowski further adds that companies like 3M, Proctor & Gamble, Boeing, BP and ConocoPhilips are using HPC clusters every day in business-critical applications.

"What you are increasingly seeing is commercial application of HPC technologies to help companies bring new products to market, and many of them through areas of simulation," says Chabowski. "Companies like BMW and Boeing will do more computer-based simulation rather than continue to use wind tunnels, which take longer and are more expensive. In the commercial world, it's things like, How do I mix a liquid? How do I model how the chemicals are mixed in there? How do I make sure that bottle will withstand any shock it may receive as it goes down the production line?"

There are numerous other examples. HPC is being used for real-time trading, particularly for complex risk analysis in finance applications. Web based applications for extreme scale web based technology demands by companies such as Google and eBay are being implemented with sophisticated technology thanks to HPC. Pharmaceutical and Biotech firms are performing research at the enterprise level that was once performed mostly by government and university research labs, also courtesy of HPC technology. Even the animation industry has adopted HPC solutions for computing, visualization and storage applications.

Suse says that if you take go-go companies like eBay, Google, or Amazon, you will see technology that bears a semblance to the HPC environment. Say Suse, "even if the web-based transactions or searches are not traditional HPC applications or workloads as such, these companies use HPC or supercomputing technologies to deal with all the data processing and run them at extreme scale."

Learn more about this topic


This story, "HPC Spinoffs " was originally published by LinuxWorld-(US).

Join the Network World communities on Facebook and LinkedIn to comment on topics that are top of mind.

Copyright © 2008 IDG Communications, Inc.

SD-WAN buyers guide: Key questions to ask vendors (and yourself)