The cloud is revolutionizing networking, and this overhaul presents enormous challenges for IT managers who are used to being able to see, monitor and control their networks and systems.
Network and systems management software has been heading in this general direction for years and is better positioned than you might think to take this next step – but there are several areas that require more work by the industry.
IN DEPTH: Guide to cloud management software
To understand the size of the problem, let’s take a look at what cloud computing is. There are many definitions, but at its heart, cloud computing is an abstraction of things that have not been abstracted before. Instead of having servers, software, applications and storage dedicated to certain tasks, all of that is abstracted to the user and even the IT manager.
Instead of being concerned about individual servers, the focus is on the services they provide – services like email or a sales application. Under the covers, resources (like servers, network devices, storage and operating systems) are shared for these services. Automation software can set up and tear down resources as needed – provisioning a virtual machine with an operating system and an application, for instance, and then tearing it down later. But the person using the service is unaware of the resources being used underneath, and they can be changing all the time.
There are two major divisions of cloud computing – public clouds and private clouds. In a private cloud, IT departments build their own clouds in their own data centers. They set up the automation, they provide all of the hardware and software to support the service, and they provide the abstraction through virtualization and automation techniques.
In a public cloud setup, a company uses an outside firm to provide the service. The most abstract type of public cloud is software-as-a-service (SaaS), where the software is hosted by the outside firm on an Internet site, and the client company usually accesses it through a Web browser. One step down in abstraction is platform-as-a-service (PaaS), where customers can create the applications to run on the platform, but all of the software and hardware is managed by the outside firm, and the applications are tied to the application platform provided. Another step down in abstraction and you get infrastructure-as-a-service (IaaS), where the hardware and the operating system are still managed by the provider, but the applications are completely determined by the client.
You can imagine how these developments could throw enterprise management software for a loop. Management software’s roots are in the physical world, providing visibility and control into all of the hardware and software components of a network. Now, that visibility is being deliberately taken away.
Past is prologue
Cloud computing, however, did not arise out of nothing, and management software vendors have not stood still for the past decade. Although management software still has much to do to keep up with the changes, two key developments have helped prepare management software for the cloud.
Before cloud computing came, there was service-oriented architecture, or SOA. While the buzzword is no longer buzzy, it did introduce the concept of abstracting resources. The key message was that servers, storage, software and network components are not as important as the business-related service they provide. These services could be anything from email to an HR application to a customer service application.
This was a big shift in thinking for the IT industry, and it took some time to absorb. But major management platforms across the board reoriented to support this concept, whether the platforms were from BMC, CA, HP, IBM or others. The management software would roll up the supporting hardware and software and give IT managers a view into which business services were being affected, and how. You could still easily drill down into the individual components, to track down the server that was slowing your email response, for example. But the emphasis had moved.
The second key development was virtualization. Virtualizing machines abstracted the operating environment from the hardware it ran on. Once you have done that, the physical location of the software is less important, and you’ve just planted the seed for cloud computing. Indeed, private clouds have been called the next generation of virtualization. Again management software vendors had to cope with this new reality and had to adapt to monitor and manage virtual machines.
HP, in particular, took on the provisioning and de-provisioning of virtual machines, seeing the task as in line with its strengths in servers. “When virtualization came along, it fit neatly on HP’s platform,” says Mark Shoemaker, executive program manager of cloud compute and BTO operations software, at HP. “We didn’t have to reinvent the wheel.”
Hurdles
Still, there is some evidence that management software vendors haven’t done as good a job at managing server virtualization as they could have. Enterprise Management Associates published a survey of 151 IT professionals in February 2011 that showed a “surprisingly high” number of companies either buying all-new tools or building their own to cope with server virtualization, says Jim Frey, managing research director at EMA. In the survey, where respondents could choose more than one answer, 50% say they were buying and deploying new management tools, and a striking 33% said they were building their own. Contrast that with 54% upgrading their existing tools.
“What that says is that we haven’t solved this problem,” Frey says.
Indeed, even with foundations laid in service management and virtualization, the long history of traditional enterprise management platforms is both a blessing and a curse. While the longevity brings experience, it also brings baggage and an inability to adapt to the new cloud reality as quickly, says Jeff Kaplan, managing director at consulting firm THINKstrategies.
“They have been cumbersome in their own way,” Kaplan says. “They’re very expensive, very complex to deploy, and not easy to administer.”
Of course, the same thing happens every time there is a major technology shift, whatever the technology. The old guard struggles to bring their installed base into the new paradigm, while a host of start-ups appear to try to solve the new breed of problems from scratch.
But both old and new vendors have work to do – because what’s going to happen is that most companies are going to move to a hybrid approach, with some services performed in-house, and some services farmed out to outside providers. Management software is going to have to be able to handle both kinds of services in an intelligent way, and in a unified way.
TESTED: First-ever test of public cloud management wares
“In terms of bursting to the cloud and bursting back from the cloud, I don’t see any [vendors] doing that,” says Jon Oltsik, principal analyst at Enterprise Strategy Group and blogger for Network World.
CA has made some strides in this direction, particularly with its 2010 acquisition of 3Tera and its AppLogic software, which endeavors to help IT managers “know the health and wellness of an application no matter where it resides,” says Adam Famularo, general manager of the cloud computing business at CA. In a “smart cloud environment,” it shouldn’t matter whether the application is on a private cloud or delivered by an outside provider – the policies should be consistent, he says.
The private cloud is the “easiest of the scenarios” to manage because the management software can reach all of the components directly, EMA’s Frey points out. With SaaS, there’s not much you can do other than test the availability and responsiveness of the service.
The trickiest areas are IaaS and PaaS. In those scenarios, “rarely do you know where your workload will be placed,” Frey says. One open question is if you want to do packet-based remote monitoring, where in the vast cloud do you put the agent? “No one has cracked that yet,” he says.
Getting their arms around it
There are several things that vendors are doing to crack all of these issues.
The first is acquisitions. The more established management software vendors are grabbing up younger companies that have been able to address cloud-specific problems in innovative ways. As mentioned, CA acquired 3Tera in 2010 – but that was just one in a string of cloud-related buys in the last couple of years, including Nimsoft for $350 million, Arcot Systems for $200 million, Oblicore, Cassatt, Hyperformix and 4Base Technology. And then in June 2011 it announced a $330 million purchase of Interactive TKO, which helps developers build applications for cloud environments.
Other vendors have acquired cloud-related companies as well, though perhaps not as much as CA. For IBM, its 2010 acquisitions of Intelliden and Cast Iron Systems helped its cloud management efforts. BMC Software says its cloud management was aided by acquisitions of Coradiant, Gridapp Systems, Neptuny Software, and Phurnace Software. Even Cisco is getting into the act, with acquisitions of cloud management-related companies such as LineSider Technologies in 2010 and newScale in 2011.
The second big thrust is in building an “ecosystem of relationships” with cloud service providers, as Kaplan points out. Expect all of the major management software vendors to form relationships with the big cloud providers if they haven’t already. These providers include Amazon, the big dog, which offers the services but has left the management piece alone, Kaplan says. Presumably, such partnerships will give the management platforms better visibility into the clouds.
Thirdly, an effort is under way to try to somehow qualify the services provided by these outside firms. Carnegie Mellon University in May 2010 launched an initiative to develop a Service Management Index to try to get a standard method for comparing cloud services. The resulting index would be used by members of the Cloud Commons, which is a community supported by CA.
Lastly, vendors are also fleshing out management-as-a-service offerings. Since management is just an application, it can be put into the cloud as well – and that means you could use a cloud-based service to manage your cloud-based services.
The plight of management software vendors mirrors that of IT managers – if they all can stay on top of cloud computing as it invades the enterprise, if they can manage it, then their prospects for survival are that much better.
