Q&A: How Indiana cleaned up its big IT mess

CTO says: "The vendors were running wild."

In 18 months, the state of Indiana has gone from disarray to discipline in its IT infrastructure. Brought in by Governor Mitch Daniels, an experienced IT management team has cut 230 jobs, consolidated hardware and services, renegotiated contracts and saved the state $25 million in annual costs. Senior Editor Carolyn Duffy Marsan interviewed Gerry Weaver, CTO of Indiana’s Office of Technology, about how he pulled off this remarkable turnaround.

In 18 months, the state of Indiana has gone from disarray to discipline in its IT infrastructure. Brought in by Governor Mitch Daniels, an experienced IT management team has cut 230 jobs, consolidated hardware and services, renegotiated contracts and saved the state $25 million in annual costs. Network World Senior Editor Carolyn Duffy Marsan interviewed Gerry Weaver, CTO of Indiana’s Office of Technology, about how he pulled off this remarkable turnaround.

What was the state of Indiana’s IT operations and infrastructure prior to the election of Governor Daniels?

I had just taken early retirement from [Electronic Data Systems]. He brought in a new management team as you normally do when you change administrations. There was Karl Browing, who is the CIO, and myself. The first thing we did was assess what we had. We found a pretty dismal situation. In one of my past lives at EDS, my job was to go around and fix major problems we had with our biggest clients. Quite frankly, I have never seen anything in quite as bad of shape as what we had here.

Our service to other government agencies was very poor but at a high price. The management team was weak. They used the build-it-and-they-will-come mentality. There were no metrics. There were no real business processes in place. They had no strategic plan. The data center was in complete disarray. There were cables all over the place. It was a mess.

Each agency had its own infrastructure groups supporting PCs, doing LAN management and server management. The vendors were running wild. We found some terrible contracts that had been negotiated. There was no disaster recovery. The statewide network was a huge problem. I don’t want to call it abuse, but there was definitely misuse of state money.

We were asked by the governor to do a couple of things. He wanted to develop a statewide infrastructure services team. He wanted service excellence. He wanted reduced costs. And he gave us an edict to be successful. One of the critical success factors of this effort was the governor. He gave the power to Karl and myself to dictate the way things were going to go.

What progress have you made so far? 

Before we arrived, IT infrastructure services were provided by 400 people across the state. We had close to 100 people at the time, and we were supporting around 900 PCs. We have taken that 400 people and reduced it down to 170 people, and we’re supporting 24,000 PCs.

The first thing we did was establish a service excellence plan. We looked at what services we needed to provide. We established the key metrics that you’d measure an IT organization on, and we started measuring our performance. We used standard red, yellow and green indicators to measure our performance.

We started with 20 primary service areas. When we first started measuring, they were all red. Now we only have one red. That measures the call-abandonment rate on our help desk. We wanted to get to 2%, but it may be that we can’t afford to get to 2%. We have two metrics that are yellow. Our goal is have those be green by the first of the year.

We consolidated all of the infrastructure services including help desk, server administration, mainframes, desktop services and networks. We consolidated all of these functions in 14 months without spending any money on contractors.

We went through agency by agency for the 70 agencies we support. We came up with agreements about the services we would provide and the cost savings they would receive. We’ve finished every agency except the Bureau of Motor Vehicles. They were in the middle of implementing a new computer system  [purchased before Governor Daniels was elected], and we felt it was inappropriate to migrate them because of all the changes going on there.

So far, we’ve generated around $14 million in annual savings from these steps. The savings came from eliminating contractors and people.

What did you do with the existing IT contracts?

We went through them all. There were some major companies we were dealing with. It was very obvious that we didn’t get a great deal. So we went back to those companies and said if they wouldn’t renegotiate with us, we’d let the contracts run out and we wouldn’t do business with them again.

We had purchasing agreements to buy PCs from four or five different computer hardware vendors. We got the same price whether we bought one PC or 500. Any buys over 10 machines, we put out to bid and we saved millions on that. We decided to have a single, standard PC contract. We put that out for bid to select one PC manufacturer to do business with us for the next four years. Dell won it. We just awarded that contract in May, and we’ve already saved $400,000.

The other thing we did was standardize our configurations. We have a configuration that’s kind of medium to low end, and if you want something above that, you have to give a justification. We did the same thing for servers. Dell won that contract, too.

Just with our contract renegotiations, we saved somewhere in excess of $10 million. That’s in addition to the $14 million we saved in people and contractors.

What about the Indiana Telecommunications Network?

We inherited Intellinet (the state agency that oversaw the Indiana Telecommunications Network), which was in trouble. They set prices for T-1s and managed circuits. When we first got here, a T-1 was $1,100. We worked with the service providers and got that down to around $700. Then we went out for a new bid. AT&T won it. We just awarded them a four-year, $22 million contract. Now T-1s are $400. We saved the state $1.5 million to $2 million on the network, plus we’re giving them redundancy that’s built in.

Was it hard to make so many changes that quickly?

The only way to do it was to change the majority of the management team. Of the 30-some managers from the past administration, maybe four are still here. The rest are gone. It’s not easy to recruit really good people to state government. We found a lot of good young people. We told them they’d get lots of experience, and that we’d help them be successful. As a result, we have a lot of really good young managers who are very intelligent and learning daily. Now they know how to negotiate with suppliers to get the best deal. They’ve brought a lot of enthusiasm and new ideas.

What was the reaction of the rank-and-file IT staff to the new management team?

Most of them welcomed the change and said why the heck didn’t this happen sooner. One of the things we did was establish a strategic plan for the IT operations group. We refresh it every six months. Part of the idea with any strategic plan is to get your organization to understand where you are trying to go.

One of the key success factors for us is communications. With each of the agencies, we have [regular] meetings where we go through the status of everything and we document it. We’re tracking everything. Every week, I get a status report of every project that’s going on and when it will be done. Our goal is to be the best state-run IT organization in the United States.

What’s next on your agenda?

We’re consolidating data centers. We just finished Workforce Development, and we’ve got five more planned out over the next six months. We had seven data centers, and now we will have one with a remote disaster recovery site. We’ll save around $1.5 million to $2 million more by consolidating all the data centers.

What new technologies are you interested in applying?

We have so many data warehouses. Now we’re looking at how to make those useful from the standpoint of detecting fraud. For example, when a person dies the Indiana Department of Health issues a death certificate. Who’d like to know about that? Workforce Development, Family and Social Security Administration, Voter Records, Bureau of Motor Vehicles and Department of Revenue. We can automatically generate transactions or subscription notices to each of these systems using Web services technology.

Other states such as Minnesota are embracing VoIP to cut the costs of its regional and long-distance telephone bills. What about Indiana?

We cannot go VoIP across the state in one fell swoop. It has to be built over time. Part of our problem is that we still have Category 3 cabling in a lot of these buildings. There are certain agencies like Bureau of Motor Vehicles and Department of Transportation that want to start using VoIP in their remote locations. We put an RFP out for VoIP. Cisco was selected for that contract with two channel partners. We will use that contract as building blocks. Then two or three years from now, when we’re all on VoIP, we won’t have five or six different VoIP systems that we have to integrate.

Indiana lags other states in security, with nearly one-third of its servers considered highly vulnerable to comprise in a report released in January. How well are you doing in terms of shoring up the security of Indiana’s systems and networks?

When we inherited security, it was in the same shape as everything else. There were no security policies. One of the first things we did was write a very comprehensive security policy. Then we brought in a company to do an assessment of potential vulnerabilities. Since then, we have significantly reduced the number of vulnerabilities they found. The vulnerabilities that were found were mostly in servers in other agencies that weren’t managed by us. We’ve consolidated those servers and put in the appropriate security patches. There might be 1% or 2% of our servers considered vulnerable now. As we’ve consolidated, we’ve significantly reduced the number of servers. We’ve gone from 2,500 in the state to 1,800. Our goal is to reduce down to 1,200. That makes it easier to manage, and we’ve standardized the configurations on them.

What grade would you give your office in terms of delivering services to its state agency customers?

When we first got here, I’d give us an F+ or a D-. I think we’re at a B+ right now. We’re shooting for an A-. I don’t think we can afford an A. I hope to be at an A- in January.

Of all the steps you’ve taken to improve Indiana’s IT infrastructure and management, which ones provided the biggest bang for the buck?

Our process for consolidation and the service excellence plan we developed, which includes the metrics we measure and how we communicate with people.

What were the biggest challenges you faced in terms of changing the way Indiana acquires and manages its IT infrastructure?

Getting all the other agencies to understand what was going to happen. We had a few hard meetings with agency IT directors. There were a couple that didn’t buy into our plan and, quite frankly, they’re gone.

You came to this job from serving as vice president of EDS’ Midwest region. What are the key differences you see in managing IT infrastructures in the private sector vs. state government?

Procurements take much longer because you have to do the RFP process in certain ways. Recruiting is harder. Even though the pay is lower, we are able to give our managers a lot of power to make things happen and that’s a positive for them.

Getting personal: Gerry Weaver
Organization:State of Indiana
Title:CTO, Office of Technology (tenure: 18 months)
Responsibilities:Oversees IT infrastructure, including voice, data and video networks, and services including helpdesk, desktop services and services administration.
Staff size:170
Annual IT budget:$60 million
Network budget:$5.5 million
Previous jobs:A variety of positions with EDS, including vice president for the Midwest and vice president for Global Accounts. Also held management positions with Northern Telecom and General Motors.
Education:Bachelor's degree in Economics and Mathematics from the University of Wisconsin
Last good management or IT book read:"I've been reading a lot about service-oriented architectures, but I can't say that any of the books have been good."

Learn more about this topic

UPDATE: Weaver responds to criticisms of his comments in this article.

Business execs share IT lessons

06/19/06

Spotlight to shine on network, application management at Interop

09/18/06

Indiana pays E-Rate program fines

06/28/06

Join the Network World communities on Facebook and LinkedIn to comment on topics that are top of mind.

Copyright © 2006 IDG Communications, Inc.

IT Salary Survey 2021: The results are in