Server virtualization technology from vendors such as VMware and Xen is causing IT managers to address management challenges related to server provisioning, patch management and disaster recovery.
Server virtualization makes it possible to run multiple applications and operating systems on fewer hardware resources, which appeals to many IT managers looking to improve utilization. According to a recent Forrester Research poll, respondents have virtualized about one-quarter of their servers and plan to have close to 50% virtualized in two years. As enterprise IT teams look to broaden their server virtualization deployments, it’s important to get in front of the management challenges, according to industry watchers and IT managers.
“Virtualization is a rather addictive technology and IT organizations are spinning out virtual machines faster then they can manage them. The technology warrants a management investment from the start," says Stephen Elliot, a research director with IDC.For those who are struggling with how to manage virtual machines, here are answers to six important questions.
1. What’s so tough about managing virtual servers?
Some will tell you that managing virtual machines varies little from managing physical servers, and others will say it depends on what you're managing. But all agree you need to have a comprehensive management plan in place before widely deploying virtualization in production environments.
"Management is not a single discipline. It can range from business continuity planning to patch management," says Andi Mann, a research director at Enterprise Management Associates (EMA). In the case of business continuity planning, virtual servers could be considered easier to manage than physical servers, Mann explains, but when it comes to patching multiple systems, the virtual world introduces complexities. "You can't always be certain if all virtual systems are patched, and obviously that's a problem," Mann says.
Consistency and standardization also become a bigger issue when managing virtual servers alongside physical machines. The perks of virtualization include easy-to-deploy resources, and that demands IT managers have predefined configuration parameters for application and database servers, for instance. Experts say keeping configurations accurate and up-to-date becomes more critical in the virtual environment because configuration drift is more apt to happen on virtual machines. The same goes for patching.
"The focus shifts to managing templates and preventing drift," says Jasmine Noel, principal analyst with Ptak, Noel and Associates. IT managers would ideally create a standard template that details the operating system, vendor software, patch levels, custom code and more. The template would be maintained so that every new virtual server deployed remained consistent with the predefined standard. Patching would also become part of the template, Noel says.
Beyond maintenance and availability management, another key management issue is performance. The complexity of a virtual environment makes determining the root cause of performance issues a more daunting task, industry watchers say.
"Performance management becomes trickier because for the more difficult problems you'll need to understand how physical server issues manifest in the [virtual machines] and vice versa," Noel says.
While virtualization provides flexible resources, multiple virtual machines residing on one box compete for the same resources and IT managers need to keep that in mind.
"No longer do you have the limitations of just what is in the server shell, but you have to worry about what else is in that box and what it needs," says Edward Christensen, director of technical operations at Cars.com in Chicago.
The online automotive company uses VMware to virtualize servers on HP boxes in its development and quality-assurance environments. "Where you used to think, 'I have a two processor/16GB server,' you are now thinking, 'Do I have two processors and 16GB I can use or is [a particular virtual machine] taking that up or needing it also?’" he adds.
Virtualization offers ease of deployment, which can become a bit of a Catch-22 scenario for IT managers. The faster servers can be provisioned, the more it seems they are in demand — and that quickly leads to too many virtual machines.
"We find that new machines are spun up and down on a more than once-a-day basis. Lots of those machines are put up for development, testing and staging," says Adam Gray, CTO of Novacoast, an IT professional services company in Santa Barbara, Calif.
IT managers and industry watchers say controlling virtual server sprawl requires the same processes and auditing that would be used in physical server deployments to ensure only as many machines as needed get provisioned.
"We have it set up so that no one has the rights to add virtual servers without requesting them through IT," says Marc Kraus, manager of IT infrastructure at Merkle in Lanham, Md. "We run weekly scans as well to keep that in check."
While policy-based management and inventory tools can help IT stay on top of the number of servers, IT has to be disciplined about putting processes in place to prevent virtual sprawl from corrupting the success of a deployment.
"People know we are able to bring up a new virtual server and turn that around quickly so the requests have increased. We basically have had to push back a bit against server creep," says Albert Ganzon, director of network services and engineering at international law firm Pillsbury Winthrop Shaw Pittman LLP in San Francisco.
Ganzon says he is in the middle of a VMware deployment with a dozen virtual servers running, and his team created a "bureaucracy of sorts" in which departments must formally request servers to control an outbreak of unauthorized virtual machines. "It's really about pointing out the resources they already have and asking, 'Do you really need another server?'" he adds.
Tim Antonowicz, senior systems engineer at Bowdoin College in Brunswick, Maine, says he uses a basic "sanity check" to keep his VMware ESX virtual servers from running amuck. "When a new server is requested, barring emergencies, make sure you go through enough evaluation to determine if it is truly needed or simply convenient," he says.
Industry watchers suggest adopting a server life-cycle management process in which a virtual or physical server's purpose and status from creation to retirement is tracked. Failure to curb lax deployment habits can exacerbate other challenges around managing virtual machines, such as patch management.
"If you don't know how many machines you have or how many are being rolled out, there is no way you can patch them," EMA's Mann points out.
The existing management vendors would answer, "Yes!" and for the most part, management vendors have stepped up their support for virtual environments.
From systems management market leaders such as CA to data center management players such as BladeLogic, vendors have partnered with or built APIs into VMware's tools to enable the exchange of data and provide some metrics around the health and availability of virtual servers. Several vendors promise to provide virtual and physical management metrics such as CPU, disk and memory usage side by side, but IT managers need more than the basic information provided with some tools.
"Yes, my existing management tools work just as well with virtual servers as any other server. The difference, however, is you don't have the advantage of seeing the whole machine and manipulating that in the same tools you do the [virtual machines]," says Cars.com's Christensen. "Visual representations of environments and good dashboards are key in managing a virtual environment."
Start-ups such as PlateSpin, Scalent Systems, Veeam, Vizioncore and several others have emerged to fill the virtual management gap they say incumbent vendors can't address. For instance, some of the areas that start-ups focus on are identifying applications running on the virtual machines and gaining visibility into the requests and responses in the virtual stack. Innovative virtual server management tools can help IT managers more quickly identify which application on which virtual machines is performing poorly.
"Going with a start-up comes with its own headaches, but for the short-term it can help you do what you can't do today with your management tools," says George Hamilton director of Yankee Group’s Enabling Technologies Enterprise group. "But there is always the chance these start-ups will be acquired by your existing management vendor."
VMware in September acquired virtual management start-up Dunes Technologies to augment its own tools, for example. For IT managers who aren’t ready to invest in specialized software for virtual management, there are things they can do to make their tried-and-true techniques better suited to a virtual environment.
For instance, Ganzon increased his investment in Network General products to monitor traffic to and from virtual servers. He coupled the traffic analysis from Network General (recently acquired by NetScout) with physical server performance metrics from Compuware's ServerVantage software.
"We had instances where Compuware would report the server is responding appropriately, but we couldn't get visibility into the virtual stack and see how well it was processing requests," he explains. "Network General inspects packets and looks for requests and acknowledgements to make sure there are no delays in the virtual layers and the server is behaving as Compuware is reporting."
The consensus is that the management tools that come bundled with VMware or Xen hypervisors won't cut it in a large virtualization deployment.
"If your virtualization provider is your main supplier of management gear, then you are in for a world of hurt," Noel says. For instance, the software provided with the hypervisor can tell IT managers how many virtual machines are running on a host, but provides little visibility into applications and their performance. "Remember the only reason servers exist is to give business applications the power to perform," she adds.
While the software provided with, say, VMware's hypervisor, enables management of the hypervisor and that environment, industry watchers say the capabilities don’t go much beyond availability to cover performance or other vendors’ products.
"I think IT managers have to decide if 'good enough' tools will cut it for robust management in large environments," IDC’s Elliot says. "IDC predicts most IT organizations will have more than one hypervisor in their environment in the next three years and that will demand a heterogeneous approach to virtual server management."
Plus the technology available today from virtualization vendors won't work as well when IT managers look to scale their virtualization deployments from dozens to hundreds of servers. While virtualization vendors are expected to differentiate themselves with management capabilities in the future, today's tools aren't up to snuff for large multi-vendor, multi-site networks.
"Most of the virtualization vendors have a built-in scalability issue, which they are working on, but their management tools will need to work across an entire environment with subnets and multiple sites," EMA’s Mann says. "Currently hypervisor providers are not equipped to perform management tasks across a large environment."
But that doesn't mean IT managers getting started with virtualization can't put the tools to use.
"Out of the box, ESX Server can connect to [VMware's] management suite, Virtual Center. You automatically get hardware monitoring for CPU, disk, memory and [network interface card] resources. These metrics are available for both the ESX hosts as well as the [virtual machines] hosted within," Bowdoin’s Antonowicz says.
Mark DiPofi, network administrator of the Cohoes New York School District, uses a combination of the HP Integrated Lights Out, or iLO, and VMware management tools that came with the respective server products along with Network General Sniffer products to track network bandwidth issues that could originate from additional traffic coming out of the server environment. He has about 20 virtual servers running and doesn't see an immediate need to add more management tools.
"I use iLO to remotely manage hardware; it has features that tell me if a physical box will overheat. And I use VMware to get into the back door of the ESX servers," DiPofi says. "For our needs right now, the management tools that came with my servers work excellent."
Network World Lab Alliance member Tom Henderson explores this question an a deeper technical level in his testing based article.
Whatever your opinion of Microsoft, you can't deny the company knows how to generate excitement over products. The operating system vendor's much-anticipated Windows Server Virtualization hypervisor technology, code-named Viridian, isn't expected to be released until 2008 at the earliest — which has some wondering if they should hold off their virtualization investment until then.
"Microsoft may want you to wait, but why wait? Whatever Microsoft does will be Microsoft-specific," Yankee Group’s Hamilton say. Others agree, saying that Microsoft's product could make or break decisions in smaller Windows-centric shops, but not for large heterogeneous environments.
"I'm unconvinced it is worth the wait for most large enterprises with a specific server virtualization project that they want done now," Noel says.
But if you are a Microsoft shop, you should take into consideration the vendor's plans, Mann says. Waiting would be counterproductive, but planning a short-term tactical approach until Microsoft reveals its bigger plans makes sense. While users question if Microsoft will broaden its reach to manage hypervisors other than its own, industry watchers are positive the vendor will couple its virtualization play with more management technologies.
"Microsoft will absolutely come out with management tools around managing its and other hypervisors so it will be worth checking into when the time comes," Mann says.
IT managers agree Microsoft's technology should be evaluated with due diligence when it is released but that is no reason to hold off a compelling server virtualization project needed now. "You have good tools out there now, and they will only get better as Microsoft is releasing their toolset and aging it," Cars.com’s Christensen says.
Others say Microsoft seems to be planning a different level of hosting for its hypervisor, which could make VMware more appealing for some.
"[Microsoft] is still basing its design on a scheme that runs on a basic Windows kernel. [Microsoft] cannot get the same performance out of their virtual layer when it sits on top of a full operating system kernel," Antonowicz says. "Virtual machines running on the Microsoft Virtual Server are still treated as an application running on a Windows server. Only with VMware's ESX or Xen will you get true 'bare-metal' performance."
Industry watchers have no doubt that when Microsoft enters the virtualization market, the landscape will change. The one platform that doesn't have a hypervisor built into its operating system today is Windows and when that happens, IT managers will need to reassess their virtualization strategies, experts say.
"Microsoft has key questions it needs to answer around performance and reliability, and if it can nail down those two factors, it will be headed in the right direction to compete directly with VMware," IDC's Elliot says. "Microsoft has a pretty big footprint and while it is behind in some areas, it is quickly making up ground. So the question will be about if VMware functionality is so superior to Microsoft's that it can overcome Microsoft's lower price points?"
Companies such as Hyperic and Veeam have released products designed to manage virtual environments. Hyperic, which released its Hyperic HQ for VMware software last year, built capabilities to extend the company's flagship software into virtual environments. The vendor wrote integrations into VMware's APIs and Virtual Center interface to discover both physical and virtual servers and incorporate virtual instances into an inventory of all systems. If something changes, the software detects it, updates the repository and alerts IT. HQ performs what the company calls "physical to virtual mapping" that shows IT managers the virtual machines, their hosts, as well as operating systems and applications running within the virtual machines.
In Veeam's case, the start-up is building a commercial software business off of the success of its freeware application. FastSCP 2.0 for VMware is a freeware file management product that helps customers move virtual machines and copy instances from one server to another. FastSCP was originally released in October 2006 and "became the de facto standard for ESX file management," says Veeam President and CEO Ratmir Timashev.
IT managers say in a pinch, freeware fits the bill. Mark Devlin, virtualization consultant at Auracom Technologies in Perth, Australia, uses FastSCP because when he needed the virtual capabilities, Veeam's application was the only available tool to address his needs.
"It is the fastest way to achieve real-time file system management of the ESX environment; in fact, there is nothing passive about this product — it really does deliver cost savings, reduces labor overhead and expedites the day," Devlin says.
Others say with limited feature sets, the low-cost alternatives may only have a short life in the virtual environment.
"There are not really any viable freeware solutions that can manage a VMware infrastructure. Because of the tight relationship between VMware and the physical server hardware, it would be difficult for a freeware product to really integrate at that level," Bowdoin's Antonowicz says. "With Xen virtualization, because it is open source, there are a few basic management offerings. However to get the features that are most valuable as well as technical support, the best offerings in this are from commercial entities."
EMA's Mann encourages putting open source software up to the task, with the caveat that they won't be the only tools needed to provide complete management of a slew of virtual servers. "You are going to need multiple solutions and open source should be part of the mix, but you won't be betting your entire environment on open source," he says.
As they do with traditional management tools and those that come bundled with hypervisors, other industry watchers shy away from advocating freeware or open source applications for full-blown virtual server management.
"The risk in using freeware or open source is really low if it fills a gap in existing management tools, but I'd be nervous about trying to extend the capabilities or scale the application to cover an enterprise-level deployment. You don't want to get too far down the path with the freeware or open source application and realize it will not meet all the needs," Yankee Group's Hamilton says.
In the end, industry watchers warn IT managers that the problems of managing physical servers don't disappear in the virtual world — they multiply and become obscured by the intangible boundaries between systems. IT managers say realizing upfront that going virtual doesn't lessen the management workload helps ease the adjustment to manage volumes of virtual machines.
“Managing virtual machines works the same as it would if they were physical. You still have to patch the operating system on all of them, you still need to monitor your applications and you still need to maintain them," Antonowicz says. "It adds another layer of process for the IT staff to master, but the benefits are well worth the cost.”