What CIO Bob Galey’s learned from his $330 million deal with IBM.Under tight financial constraints, Amtrak last year signed a seven-year, $330 million deal with IBM that involves outsourcing most of the passenger railroad’s IT services, including operation of its voice and data networks. Amtrak CIO Bob Galey recently spoke with Network World Senior Editor Carolyn Duffy Marsan about how best to structure and manage a network outsourcing arrangement. Here are excerpts from their conversation:Defining responsibility with your outsourcerTop voice and data priorities Aging technologyAdvice to those considering outsourcing Online-exclusive questionsWhat do you oversee?My office handles all of the IT except for the train control systems. We have a very large reservation system and associated Web sites. We also have an awful lot of financial systems. We keep track of who the crews are on the trains and where they are, the customers on the trains, and all the incidents that happen on the trains. We have 530 stations where we pick up people. We have a network connecting half of those stations, and the other half are through dial-up. It’s a frame relay network. We have about 7,000 desktops and two fairly large mainframes. We have 450 servers that we support. We also support the voice network.Given that you’ve outsourced most of your IT services to IBM, what IT functions have you retained?I have a staff of 296 people. The vast majority are software developers. That [function] is not outsourced. And we don’t outsource [operation and maintenance of] our servers. We oversee the outsourcing agreement in two ways: the administration of the contract and the operation of the contract. We take the lead on planning for our LANs but not for the WAN. IBM works with AT&T on our WAN. IBM has about 150 full-time equivalents that support us. How do you and IBM divide responsibility for Amtrak’s voice and data networks?We have an internal voice network that connects all of our sites so you don’t have to dial out to place a call. That’s probably a quarter of our budget for voice. The vast majority of our budget in voice has to do with our call centers, which use inbound 800 services. The call centers cost about $12 million to $13 million a year. IBM has responsibility for the switches and desktops in the call centers, but we man them. We plan the technology upgrades for the call centers along with IBM.The data network is all the responsibility of IBM. The contract is written so that they have to provide us with the services we need. They also manage the network. That’s something we would have a hard time doing ourselves on this scale. It’s their responsibility to keep us in the right range [on bandwidth] so that we’re not overpaying or underpaying or overutilized. We’ve retained responsibility for our overall IT architecture.When you renegotiated your outsourcing deal with IBM six months ago, how did you define the voice and data services?The old contract was written in 1994 and patched up a few times, but it was not very specific. Unlike a good wine, it didn’t age well. The new contract is very specific. We’ve broken the contract into 10 towers of service, [such as] voice or data or desktop support. The term sheets for each of those towers of service lays out in gory detail who is responsible for what. Attached to that is a procedure manual that describes not only what needs to be done but how. And tied to that are service-level agreements. IBM’s been working on a score card that is tied to our incident management system so that every day we’ll be able to see where we are compared with the [service-level agreements]. What are your top priorities with regard to your voice and data networks? We have one basic goal this year, which is to meet the availability and reliability criteria with our networks. There’s a big difference between 98.5%, which is what we had on the old contract, and 99.9%, which is where we’re headed with our new contract. That gives us a half-hour more of uptime per month. So far, we’re doing pretty well. You’re always going to have problems with a large frame relay network. What we were looking for on the WAN side was better response times, so the SLAs reflect that. We don’t want outages to last for long periods of time. In major locations, we wanted better back-up facilities.Do you have any plans to upgrade the network infrastructure in the next year?We have a lot of end-of-life servers, and we’re upgrading those. We’ve already purchased everything, so no calls, please. We have about 450 servers. We’re upgrading 30% of them. We have both NT and Unix servers. Anything that’s facing the Web uses Unix, primarily [Sun] Solaris. Our Web site, Amtrak.com, is based on Unix. We recently upgraded that Web site. Three years ago, we were handling about 4,000 transactions per day in terms of booking tickets. We just had our highest day ever of 9,400 transactions. Now 23.4% of our tickets are sold over the Web. A good number of the NT servers we have are used as an interface to the reservation system [which runs on an older IBM architecture].Where does the Internet fit in your overall network architecture? We provide Internet access to most of our employees with workstations. We have 25,000 employees and only 7,000 workstations. Most of our employees are on trains moving around, and they don’t have Internet access. We looked into what it would take to provide connectivity to our trains, and two things jumped out very quickly. One is that it’s very hard. The second is that it’s very expensive. We also haven’t really come up with a killer application that we need it for. A year and a half ago, we brought in a VPN from V-One, and we implemented it so the other 18,000 employees that don’t have Internet access from work can dial in from home and see our intranet. We have about 1,200 registered users. Now we’re embarking on a program where we’ll be able to use the VPN to get to other systems within the company. There’s some movement in the company – albeit small – for people to be able to work from home. We have a number of people testing it out now, and it seems to be working very well. How have you ensured that your voice and data networks will be competitive from a technology point of view throughout the course of the outsourcing contract?After two years, we can benchmark any tower of service. If we benchmark cost and quality and service levels, and find that it changed dramatically enough to make a difference, then we have the right to sit down and work through the negotiations and get the price down. Secondly, the contract is broken up. It’s a seven-year contract, but it’s a four-year base with three one-year renewables. And the renewables are based upon quality. What advice would you offer to CIOs who are thinking about outsourcing their voice and data networks?The biggest problem they will end up having is personnel issues. They’re going to either be replacing a number of their employees, or their employees are going to be moving over to the new company. How many people are still working for IBM that worked for Amtrak [back in 1994]? I only know of one.In setting up the deal, make sure you have a really good set of lawyers. Shakespeare said: “Kill all the lawyers.” That’s exactly what you don’t want to do in this situation.When we sat down with IBM’s management, the first thing I told them is I don’t want any lawyers in the room when we negotiate what the business deal is. I didn’t want the lawyers to start bringing up all the detail that they have to deal with. But you’ve got to get to that detail. After you agree on the business deal, you need a really good set of lawyers [to finalize it]. These are generally outside counsel who specialize in this [area].If it’s a big deal, you want to get a consultant in to help you with what’s the best practice in the industry. You have to have a good idea of what kind of telecom rates you should be expecting, what kind of management fees, how much things should be costing you, and what are prevalent SLAs. You may be satisfied with 98.5% [reliability], but it’s good to know where the industry is. And this information isn’t readily available. We used the Outsourcing Institute. We were really satisfied with them.What concerns about your voice and data networks keep you up at night?There aren’t any, actually. We have a lot of old systems, but they run pretty well. From a reliability perspective with our network, I think we’re right up with the best. It doesn’t keep me up at night at all. I don’t carry a pager. I don’t carry a cell phone. I’m not worried about those kinds of things. I also have really good people managing the vendors, but I think the vendors put very good people in those slots. Amtrak has had an IT outsourcing deal with IBM since 1994. Six months ago, you renegotiated, expanded and extended that deal for seven more years. What were the main reasons that Amtrak wanted to renegotiate this deal?I wasn’t the one who outsourced [our network infrastructure] in the beginning. We had the Outsourcing Institute help us do an analysis of what it would cost to take it in house. We weren’t financially in a position to do that because we don’t own a lot of the assets. We would have to go out and buy or lease a tremendous amount of assets. That wasn’t an option.There were four things I wanted to get accomplished with renegotiating the contract:I wanted a better contract because we were spending way too much energy — as was IBM — in disputes. That very high noise level went down to where it’s almost nonexistent because things are so well-defined.Second, our relationship with IBM wasn’t a good one. I needed to abruptly change things. Our relationship was with our project executive; it wasn’t with IBM; and this was a big issue. We weren’t getting the full value that we could have from IBM. [The renegotiation] got a lot of attention in IBM. The relationship between myself and my staff and senior executives at IBM has really blossomed into a good relationship. We’re getting people who we wouldn’t normally see every day on our contract come in and help us with [architectural] issues.Third was cost. The old contract was set up so that the more services we used, the more it cost us per unit, which is just the opposite of what everyone else has. The new contract brought [volume discounts] into place. We got a lot of cost savings, especially in the data and voice area.The last thing that we wanted to increase was the quality. We see indicators that we are much better now.You are six months into the new contract. What’s your overall analysis of how it’s going so far?Our overall analysis is that it’s going well. There are certainly always areas where it has to go better, but the trend is definitely up. A lot of it again was that people needed a clear understanding of what it is they need to be doing both from our side and their side.In the press release announcing the IBM deal, you said Amtrak hoped to save $85 million over the life of the IBM contract. What progress have you made in reaping these cost savings?We are absolutely on target with the savings. It comes out to about $1 million a month.What metrics do you use to measure how well IBM is doing?We’re using the service-level agreements almost solely because that really tells us what they’re delivering and what they’re not delivering. It’s more stick than carrot. There are penalties if they don’t meet service-level agreements, and we look at the service-level agreements from a prioritized basis. There’s a pool of their fee that’s at risk every month. We weight the SLAs because some things are much more important than others. If the reservation system is up 99.9% of the time, that’s much more important to me than whether some router goes out someplace for a small segment of the network. So we prioritize the things that are important, and there’s a pool of money at risk.Which three SLAs are the most important?The reservation system is first. Our other mainframe that our business systems are on is second. And the network — WAN and LANs — is third. Related content how-to Doing tricks on the Linux command line Linux tricks can make even the more complicated Linux commands easier, more fun and more rewarding. By Sandra Henry-Stocker Dec 08, 2023 5 mins Linux news TSMC bets on AI chips for revival of growth in semiconductor demand Executives at the chip manufacturer are still optimistic about the revenue potential of AI, as Nvidia and its partners say new GPUs have a lead time of up to 52 weeks. By Sam Reynolds Dec 08, 2023 3 mins CPUs and Processors Technology Industry news End of road for VMware’s end-user computing and security units: Broadcom Broadcom is refocusing VMWare on creating private and hybrid cloud environments for large enterprises and divesting its non-core assets. By Sam Reynolds Dec 08, 2023 3 mins Mergers and Acquisitions news analysis IBM cloud service aims to deliver secure, multicloud connectivity IBM Hybrid Cloud Mesh is a multicloud networking service that includes IT discovery, security, monitoring and traffic-engineering capabilities. By Michael Cooney Dec 07, 2023 3 mins Network Security Cloud Computing Networking Podcasts Videos Resources Events NEWSLETTERS Newsletter Promo Module Test Description for newsletter promo module. Please enter a valid email address Subscribe