The virtual blind spot

Virtual machine traffic presents a new challenge for data center security professionals

Malicious hypervisors. Subversive virtual machines. Live migration impersonators. Welcome to the world of server virtualization, where the threats are new and the traditional security tools like firewalls and intrusion-prevention systems don't cut it anymore. Unfortunately, at many enterprises, security strategies haven’t kept pace with the shift to x.86 server virtualization . “Many companies that have virtualized environments haven’t contemplated the security ramifications of what they’re doing yet,” says Forrester analyst John Kindervag.

Malicious hypervisors. Subversive virtual machines. Live migration impersonators. Welcome to the world of server virtualization, where the threats are new and the traditional security tools like firewalls and intrusion-prevention systems don't cut it anymore.

Desktop virtualization vs. PC

Unfortunately, at many enterprises, security strategies haven't kept pace with the shift to x.86 server virtualization. "Many companies that have virtualized environments haven't contemplated the security ramifications of what they're doing yet," says John Kindervag, a Forrester analyst.

Gartner's Neil MacDonald agrees. "The general awareness level of issues related to virtual security isn't quite where we need it to be," he says.

For their part, IT pros tend to look at it this way: Since physical and virtual servers run the same Linux and Windows operating systems on the same hardware, then security for the former is adequate for the latter. "They'll argue that nothing has changed -- and that's a dangerous mistake," MacDonald says.

"When you virtualize, you introduce a new layer of software and all of the Windows and Linux workloads running on top of it rely on its integrity. The first and most important thing you need to do is acknowledge this new layer and establish basic security hygiene around the configuration and vulnerability management of it," MacDonald says. "That's basic block and tackle."

Secondly, IT needs to figure out what to do about the network blind spot that virtualization creates, he adds.

Patrick Quinn

"None of our network-based firewalls or IPSs in the physical world can see the traffic being switched between two virtual machines (VM) in the same box," MacDonald says. "The question we need to answer is, 'Do we need security controls inside of the virtual server to see this virtual network traffic?’ Maybe you do or maybe you don't – but you've got to acknowledge that you can't see the traffic and if something bad happens, like an inter-VM attack, you won't be able to see it."

Many enterprises haven't focused on virtual server security because their virtualization deployments are immature. When virtual servers are just used for test and development purposes or for running non-critical, low-priority applications, security doesn't much matter.

But that changes as a virtualization layer moves into the production environment to host mission-critical applications. The deeper entrenched virtualization becomes, the greater the need to deploy security technology specifically aimed at protecting the virtual infrastructure.

Awakening to a new reality

"We did originally go through a phase where we thought physical security would do. But as we started to grow our virtualization deployment, we felt we needed to make sure we were taking proactive steps to secure our customer information," says Patrick Quinn, assistant vice president and network administrator at Thomaston Savings Bank, in Connecticut.

In doing so, the bank set up secure network segments in the virtual environment much as it would do on physical infrastructure. It uses Catbird Networks' vSecurity TrustZones virtual security technology, which allows VMs of varying trust levels to share a common host.

TrustZones lets Quinn control traffic moving between VMs based on policy. For example, Quinn says he has established trust zones for each branch, as well as several for the main office.

Likewise, Interior Health Authority, a regional health agency in Kelowna, British Columbia, is hoping to incorporate a virtual server layer into its overall security architecture, says Kris Jmaeff, information security specialist.

"Definitely one of our goals is to have visibility within the virtualization layer," Jmaeff says. "We've got certain areas where we need to use virtual sensors to monitor traffic within our virtual server world or cluster."

Toward that end, Interior Health is beta testing HP TippingPoint's Security Virtual Framework, which lets security teams monitor vSwitch – the virtual switch within VMware's platform -- and VM changes to identify tampering or disablement of security controls.

In addition, HP TippingPoint virtual IPS integrates with the vTrust virtual security technology from Reflex Systems. Similar to Catbird's TrustZones, the Reflex technology lets users create trusted network segments and enforce policies, as well as monitor, filter and control VM-to-VM traffic.

"Our goals for the beta test are to increase our knowledge, obtain more insight and visibility on infrastructure, and develop pre-engagement, pre-planning ideas of what we're going to do with security in the future. This is a good opportunity to learn and be on the cutting edge of virtual security," Jmaeff says.

Virtual security vendors step up

Catbird and Reflex are but two companies that are targeting virtual server security. Others include start-ups such as Altor Networks, Apani and HyTrust, as well as well-established security vendors. Besides HP TippingPoint, this latter group includes CA Technologies, for security functions such as access control and log management; Check Point Software Technologies, for virtual firewalls; Juniper Networks, which has a strategic alliance with Altor; IBM, for IPS; and Trend Micro, which acquired virtual security start-up Third Brigade.

"As bigger companies jump in, this signals that there is a need for these types of products. It's just a matter of time before they all have virtualized offerings of security enforcement," Gartner's MacDonald says.

It might seem logical to think that you would defend the hypervisor layer the same way you would defend physical servers -- by plugging in IPS or anti-virus software.

But MacDonald disagrees. "We don't believe you need to go run IPS or a copy of antivirus in the hypervisor. That would defeat the whole purpose of this layer being very thin and hardened. Rather, good configuration, vulnerability and patch management disciplines are enough at that layer," MacDonald says.

Forrester's Kindervag adds, "They say about 40% of issues in modern networks relate to configuration or other types of human error. That leads me to believe that how you do security management is more critical [than hypervisor security] at this moment," he says.

"What vendors really are talking about now is protecting the VMs and traffic between them just as you'd protect workloads in the physical environment," MacDonald adds. "This becomes especially important when you start combining virtual workloads of different trust levels on the same physical servers. You're going to need that visibility, that separation and that policy enforcement."

When evaluating virtual security products, he advises, select those that are optimized to run inside the virtualization environment and have been integrated into virtualization frameworks from Microsoft, VMware and Xen-based virtualization vendors.

For its part, virtualization leader VMware provides virtual security companies visibility into VM operations via its VMsafe API.

"About seven major security vendors have participated as VMsafe partners. They've developed virtualization-aware network and endpoint solutions that work through the hypervisor in a privileged fashion with high security," says Venu Aravamudan, senior director of product marketing for VMware's server business unit.

But that's just for starters, he adds. Earlier this year, at the RSA Conference 2010, VMware previewed how it envisions next-generation virtual server security technology might work. Working in conjunction with Trend Micro, it showed the ability to run antivirus processing on a host machine rather than VM by VM as current-generation products do.

"Once this technology becomes real, in terms of a shipping product, we don't have the need for an agent in each VM. That means better performance, less to manage, lower cost and so on," Aravamudan says.

It also means new capabilities. "You can look at this model to drive solutions such as being able to detect rootkits in the files hypervisors are running on, discover credit-card and other sensitive information in VMs and check the integrity of files, for example," he says.

Baked-in security

Morgan Keegan & Co., one of the nation's largest regional investment firms, is one of the few companies quite comfortable with its virtual security posture. "We don't have any security concerns today in the way that we've deployed the virtual environment," asserts Luke McClain, a systems engineer with the Memphis firm.

That's because Morgan Keegan took security into consideration from Day One of its virtualization project, launched in March 2008. That the company already has virtualized 75% of its server infrastructure – roughly 515 VMs running on 52 VMware ESX hosts across three data centers -- is in part attributable to this fact, McClain says.

A particular IT operational goal was collapsing the company's traditional firewalled DMZ into the virtual environment. "We felt that we could really benefit by bringing those physical machines into the virtual environment and manage them while still leaving them in this protected pocket," says Parker Mabry, managing director of network systems engineering at Morgan Keegan.

This required close planning with the information security group, which compared virtual firewalls against what it knew of their physical counterparts – in its case, Cisco's firewalls. "They compared feature to feature, looking for things like robust logging, forensics and the depth and granularity of locking down machines," Mabry says.

"I like to tease that usually the first response we get from corporate information security is 'No' – it's that tight," he says. "So actually getting information security to see the value of being able to use a virtual firewall in the virtual environment was a big win for us."

To harden the virtual DMZ, Morgan Keegan uses Reflex's vTrust Security product.

From an operational standpoint, the company secures VMs through tight authentication, McClain adds. With VMware's vCenter virtualization management tool and the management interface, "We're very cognizant of who has rights to any virtual machine and keeping close track of that specifically and especially in the DMZ environment," he says.

VMware encourages its partners and field service organization to ensure that all enterprises bake security into their planning and designs, as Morgan Keegan has, Aravamudan says.

While the security-first encouragement doesn't always stick with customers just starting out on their virtualization journeys or who are using the technology in limited scenarios, larger enterprises do get it, he says.

"Especially at those customers with large percentages of workflows deployed on virtual servers, we clearly see a lot more discipline in adhering to our best practices and security hardening guidelines," he adds.

VMware believes that just as virtualization enabled massive cost savings and efficiency gains, it is a real game-changer when it comes to security, Aravamudan says. "It's definitely one of our goals -- and we've already started to prove this – that security for environments based on virtualization will be better than physical security as it exists today in IT."

Gartner's MacDonald agrees. "What we see clearly is that virtualization is not inherently insecure, but that it gets deployed insecurely today. But this problem will go away over the next three to four years as IT staffs, vendors, the tools and skills mature," he says. "People will be deploying securely -- ideally even more securely -- than they have been in their physical environments."

Schultz, author of the Network World’s Network/Systems Management Alert, is a longtime IT writer and editor in Chicago. You can reach her at

Copyright © 2010 IDG Communications, Inc.

The 10 most powerful companies in enterprise networking 2022