How IT managers are banking on intelligent infrastructure to ease their layered burden.
Bob Kelly is the first to admit it; he’s knee-deep in an enterprise complexity crisis.
As CIO at New York state’s Division of Housing and Community Renewal in Albany, Kelly has seen his operating expenses for the agency’s vast network balloon to almost double what he spends on hardware and software capital expenses.
VoIP realities
The secret sauce: Interoperability
Kelly attributes this rise – to $4 million – to the layers upon layers of technology that exist throughout the three main offices and nine remote sites his team supports, from security tools to server farms to application patches and updates. And he sees trouble on the horizon.
“We see the state is going to have difficulty getting resources so we’re turning to centralized management,” he says. “It’s easier to get money for products than staff – the state doesn’t like to look like it is hiring people – so we need to consolidate.”
Kelly isn’t alone. To reduce costs driven by complexity, companies are turning to automation tools from major systems and network players as well as smaller suppliers of advanced management tools, says Andreas Antonopoulos, principal analyst at Nemertes Research.
“The largest cost is the operations overhead – cost in salaries, training, attracting specialized people,” Antonopoulos says. To fight back, he says IT managers are looking to blade servers, virtualization tools for server and storage resources, bandwidth allocation programs, and application management software to boost intelligence in the network and reduce the need for human intervention. Eventually, IT shops will feature self-configuring, self-healing, self-protecting networks.
“The fewer people that stick their fingers into the network, the better it runs overall,” he says. “Automation tools help the network be as flexible as possible, allowing users to deploy applications with minimal overall operational costs.”
Automation has significantly reduced the number of people it takes to run networks, especially in data centers, Antonopoulos says. “Networks are becoming less specialized and as commoditized as possible.”
The evidence: VoIP, Web and file servers all can be deployed on the same server or moved from one server to another, which lets IT managers more easily expand or shrink the resources they need.
Getting here from there
The IT complexity crisis has been building for some time, exasperated in part by the buying frenzy unleashed to deal with Y2K. More technology has been layered on without much consideration for network and operation dynamics.
Consider the emergence of the major corporate software suites. “Companies spent a lot of money putting in new applications that created islands of integration – such as ERP and CRM systems,” says Tony Bacardi, senior vice president of software at IDC. Then they lumped on complex management tools that didn’t work together and featured disparate data dictionaries and schemas, he says. “It’s just been one unproven layer of software after another.”
Bacardi says there is so much complexity that company executives often wonder if they are getting their money’s worth. A recent IDC study found that less than 20% of chief officers felt they were using the full capabilities of their myriad enterprise systems.
One way to deal with complexity is to turn to specialized tools designed to address specific problems.
“IT shops have gone out to solve discrete problems, and they’ve bought discrete appliances,” says Anthony James, product marketing manager at Netscaler. “They’ve had three choices: upgrade my servers, redo my infrastructure by adding bandwidth or tweak the applications. They’ve rolled out distributed denial-of-service protection systems, compression technologies for the WAN, TCP offload, caching.
“They’ve been justifying single boxes to solve single problems,” James says. “But managing all that is incredibly complex. And you’ve got personnel costs, gear costs, support costs, maintenance fees and training fees for each of those boxes.”
Netscaler is just one of many start-ups trying to consolidate multiple operations like DoS protection, encryption and compression into one device. “Reducing the number of boxes you’re managing in effect reduces your overall operating expenses,” he says.
New intelligent devices also reduce the amount of gear an IT shop needs. For instance, rolling out automated TCP offload or TCP multiplexing takes the load off servers. “A server that had to deal with 10,000 connections can cut down to 3,000 connections. Or you can cut a server farm down from 20 to 5 – saving on licensing, management and maintenance,” James says.
Caching is another big win. Intelligent new tools that let users automatically run reports can tell whether information is dynamic or static – letting the static information be cached. “This reduces the pressure on data center resources to churn out those reports,” he says.
Back to the fun stuff
In addition to reducing operating expenses, the goal of building intelligence into the infrastructure is to let IT shops refocus their energies on changes to the network that will add to the bottom line. He says today, 90% of an IT shop’s resources are spent fine-tuning apps.
Doug Wilson knows this dilemma well. As IS operations manager for Bay Area Credit Service in San Jose, Wilson manages a busy payment collection call center. He’s implemented VoIP and automation tools that let his PBX servers tackle hundreds of thousands of collection calls each week, reporting back into databases on the varying degrees of success.
He also has rolled out a network of automated tools that manage and monitor live calls – notifying nearby supervisors if calls are taking too long or a collector is experiencing trouble. “Our supervisors used to have to walk around and look over people’s shoulders,” he says.
But Wilson has bigger plans. Today, if a client needs to establish what he calls a massive campaign, Wilson has to call other collection companies to see if they can pool resources to take on the business. If he finds enough takers, then he has to connect to all those resources – networks and servers – and create a temporary network. All this is painstaking, he says.
Ultimately he hopes he will be able to plug in the needs of the campaign, run a tool that is already attached to willing participants’ networks to see what resources they have available, and establish a network on the fly with security, regulatory and other considerations in place. “We’d be creating a virtual company that could win that business,” he says.
Today, when you add another layer of sophistication you hide a lot of complexity, Wilson says. “We want tools that leverage smarts over time and people.”
For Jason Pape, pooling resources and managing them as one unit would be terrific.
Pape, corporate network administrator for International Hospital Corp. in Dallas, acts as virtual IT director for the nine hospitals his company manages and owns. But the hospitals are spread across multiple countries, including Mexico and Brazil.
He says he’d like to see automation tools that let him easily roll out, manage, update and patch applications on servers across continents. With “different countries, different languages, different standards,” there are a lot of hurdles, Pape says.
“Everything in the U.S. medical world is dictated by insurance, but not overseas,” he says. “We don’t have the same infrastructure needs, and that makes all of this difficult.” Pape depends on vendors in each hospital location to manage their software packages. He says this helps keep costs down.
“Most of the responsibilities of the IT staff at the hospitals is geared toward improving the systems and supporting the physical parts of the networks,” he says. “We leave the logistical part of the software to the vendors and make sure we keep a good relationship with them,”
But Pape says that as he tries to standardize applications and tools for the hospitals, the vendor contracts will have to go away. He says he hopes to automate the task of configuring, deploying and managing the new applications.
“Every automation tool out there – I don’t care if it’s a simple script or a whole program – they say it can work miracles, but you still have to make it talk to your database and recognize variables and work with the parts you have,” he says. “That’s just the reality of it.”
New York state’s Kelly says it all goes back to the complexity issue. “There are so many different parts to the network and there are new tools for everything. Take security, you’ve got firewalls, anti-spam, anti-virus. . . . How do we manage all that when we have limited resources?”
Already his team has begun rolling out tools to get a better handle on operations. For instance, he uses a dashboard tool to view how his servers in all locations are performing. But he says automation tools are still lacking. “If something goes wrong on those servers, you still need someone on-site to handle the problem.” He adds that policies for alarms are still nascent.
“The critical alarm notifications need to be done better,” he says. “It gets to the point where you’ve got all these alarms going off but how do you know which is important? Our trained staff learns to deal with this over time – but that’s just it, over time.”
Kelly has a bellwether he says will let him know when the complexity crisis is easing and automation tools are ripe. When he can be alerted to rising temperatures in a remote data center and instantly resolve the problem by switching to battery backup or completely shutting down the servers because the HVAC, battery backup and server systems are all connected, he’ll know he’s reached nirvana.
Gittlen is a freelance writer in Northboro, Mass., and former events editor at Network World. She can be reached at sgittlen@charter.net .
