Variations on a virtualization theme

Which strategy is right for your data center: consolidation, clusters or grids?

To virtualize or not to virtualize -- that is no longer the question when it comes to deploying Linux in the data center. Today, the question is which virtualization approach to take.

The trendy data-center virtualization scheme among Linux users is server consolidation, which aims to address a problem that has roots in the economic downturn of 2001 to 2003, when cash-strapped enterprises started favoring smaller servers over larger ones, Bozman says.

"Over that time, there was a proliferation of volume servers, the likes of which has never been seen," he says. Before 2001, Linux server shipments were around 3 million to 4 million units per year. Now they top 7 million. For customers who built out data centers using hundreds of machines, there is now a push to pare down the amount of "pizza box" hardware.

"Customers who run a data center with 50 or 100 physical servers may need 500 or 1,000 of those machines someday," says Kevin Lehay, director of virtualization at IBM. "How do you manage all of that environment? That's where the scale-up environment takes advantage of that."

The drivers behind the scale-up model include the ability to manage and provision servers more easily, with virtualized servers all running inside one box. Cost savings on power consumption of one large machine, vs. hundreds of single-rack-unit boxes, can be significant. A recent study by Gartner found that the cost of energy in data centers is in some cases almost equal to the cost of the server hardware itself.

For Nationwide Insurance, consolidation of 416 Linux servers onto a single Big Iron box means less walking around and pushing buttons. This is not insignificant when considering wide-scale server maintenance, such as applying Linux kernel or application software patches, according to Steve Womer, senior IT architect at Nationwide Insurance.

"Let's say it takes you 45 minutes per server to apply patches and software fixes, to reboot them and get them back up," Womer says. "Forty five minutes, with 418 servers - that's 315 man-hours. I've got eight people to do all this. That's a long time."

Womer uses a single shared-root file system, which the 418 servers share, running on top of the IBM z/VM virtualization layer of the mainframe. "If you only have one root, it's only two man-hours to patch the copy of the shared read-only root, then you start rolling it through."

Hype over hypervisors

Several key Linux kernel and system-tool advancements over the last several years are helping these virtualized data-centers-in-a-box and grid-style deployments to evolve.

"The introduction of hypervisor technology you might say is the single-most important virtualization advancement over the past five years," says Justin Steinman, Novell's director of product marketing for Linux and open source.

The hypervisor is a software layer that sits between the guest operating system and the physical server. "The best way to think of it is as the traffic cop," Steinman says. The software controls the different operating systems that are running on a virtualized server and manages the flow of the hardware resources, such as I/O, storage, and processor use and memory access. Open source and vendor-specific products in this area include Xen's open source virtualization technology, IBM's z/VM and VMware's ESX Server.

Virtualization via a hypervisor layer is called paravirtualization, Steinman says, as opposed to standard VMware-style virtualization, in which a guest operating system runs inside a host, without any knowledge of the host system. Novell's SUSE Linux Enterprise Server 10 has a Xen hypervisor built into the Linux distribution, and Red Hat's forthcoming update to its Enterprise Linux Server also will have this virtualization piece built in.

"You need to put software drivers [in the guest Linux systems] to make them aware that they're being virtualized," he says. This enables the virtualized Linux systems to use processor resources more efficiently. Otherwise, the systems would compete for resources, with the software functioning as if running on a weak hardware system.

In clustering and distributed computing, some of the important advances have happened inside the Linux kernel, as well as with system and management tools offered by vendors to harness and control dozens, hundreds or thousands of Linux-based processors.

"That's a challenge for high-performance computing users," Steinman says. "How do you make sure all those processors are the exact same operating system with the exact same patch, with all the different tweaks there? If one box is out of sync, it could bring the whole system down."

Tweaks in the Linux kernel over the last few years also have expanded possibilities for distributed, virtualized Linux.

"Some of the advancements inside of Linux that have helped this stuff are improvement in scalability and performance," Steinman says. Linux software can now scale to 10TB of memory across a grid or cluster, and as many as 1,024 processors. "That's an advantage where the open source technology has improved to enable that. You could go out this afternoon and download the code and find the exact code tweak that was made to implement that kind of advancement."

Linux virtualization also is being used to consolidate Windows servers in some IT shops. Success Apparel, a children's clothing company in New York, recently boiled down its 17 separate Windows servers to nine servers running SUSE Enterprise Linux, VMware and virtual Windows instances on top.

The move "has reduced operating expenses by 25% while allowing our IT staff to concentrate on other projects," says Steven Golub, the company's IT manager.

Scale out with clustering

"It's funny with all the excitement about virtualization, people have sort of almost forgotten that clustering is a form of virtualization," Bozman says. "Clustering was one of the earliest forms of virtualization, in the sense that when an application is cluster-aware, it views all of the attached server nodes as being resources that it can use, as if it were on a big SMP [symmetric multiprocessing] machine.''

Users of large, high-powered Linux cluster systems say the mix of proprietary virtualization management software, along with low-cost hardware and free Linux, are opening up the processing-power floodgates.

CIS Hollywood is a digital special-effects house that produced digital images for "Pirates of the Caribbean," the fantasy epic "Eragon" and the most recent "X-Men" movie sequel, among dozens of other movies. Much of CIS Hollywood's rendering work -- in which large computer files are processed and crunched down into a viewable digital movie format -- is done on a cluster of 40 Linux PCs, running the free 64-bit version of the CentOS Linux distribution, which are managed by software from Linux Networx.

"The big key with Linux Networx is manageability," says Matt Ashton, systems manager for CIS Hollywood. "Instead of having to maintain individual nodes -- which can be done with a variety of scripts -- they've got all of that all set up to go. I can update all 40 machines with a few mouse clicks without having to do it by hand."

To CIS' users -- artists, graphic designers and computer technicians -- the Linux cluster appears as one large virtual machine. Fronting the cluster is a scheduling application written in-house, which distributes rendering jobs to the 40 machines. "Users don't interact with individual nodes," Ashton says. "They just submit jobs, and the queue management software takes care of it."

CIS has used a clustered, virtual rendering system for more than four years as a way to process the work of its artists more quickly and inexpensively. Ashton says nodes in the cluster -- dual-processor AMD Opteron boxes with 4GB of memory -- cost about $4,000 each. CIS' large SMP Linux machines -- four-processor, dual-core machines with 32GB of memory -- cost between $30,000 and $40,000 each. The cost savings on a per-node basis is between $2,000 and $3,000 when scaling the system out, as opposed to up, he says.

PayPal, the online payment system owned by eBay, uses thousands of Linux machines to run its Web presence. The Web company replicates a single Linux/Apache image, bundled with its own transaction software, across these servers that appear as a single system to customers.

"Rather than have a monolithic box, we just have so many [nodes] that the breakages are irrelevant," says Matthew Mengerink, vice president of core technologies for PayPal. (For more about PayPal's Linux technology, see  here >>.)

However, few enterprises need the kind of computing power of a CIS Hollywood, or the scale of a global payment system, such as PayPal's.

Google is another example of the scale-out model, Steinman says. Its search engine runs on thousands of distributed Linux computers, which provide its signature fast, accurate search results. "But will an enterprise run its SAP platform on that model?" Steinman asks. "Probably not."

Griddy up

However, this does not preclude the use of distributed, virtualized computing in enterprises.

"Businesses tend to use [a distributed Linux] model in certain specialized enterprise applications, such as actuary or risk management applications," IBM's Leahy says. "You could build a stand-alone environment, which could deliver these processes in minutes or hours, but it would be pretty expensive and dedicated to one thing." This single-purpose system also would remain idle most of the time, he adds.

This is popular in Wall Steet firms, where trading desks have very powerful workstations that often sit idle during the hours when the markets are closed.

"Some people would like to have a series of distributed resources, the kind of work you used to do on a mainframe," IDC's Bozman says. "This is a work in progress, but clearly people would like to do that."

Whether Linux users deploy virtualization in a consolidated deployment, or in clustered applications or grids, Bozman says there's a common thread shared among trends.

"It's like back to the future. What we're doing is reinventing the economics of computing, but we still want the same results that we had before" in the mainframe and large-system days -- "lots of reliability and lots of availability and utilization. But we're doing it today at lower price points than we did in the early '90s."

< Previous story: Linux virtualization options | Next story: Linux, open source software pay off for PayPal >

Copyright © 2007 IDG Communications, Inc.

The 10 most powerful companies in enterprise networking 2022