Danone Group cuts costs, improves efficiencies by centralizing IT, adopting ITIL processes
IT infrastructure and services are not the first things to come to mind when you think of Danone Group, the $3.5 billion company known for its Evian water and Dannon and Stonyfield yogurt brands. But when it comes to packaging and delivering water and yogurt, IT services and the automation they provide are indispensable.
“We can’t deliver one ounce of yogurt or water unless there’s some kind of technology involved, network or server or what have you,” says Mike Close, North American CTO at the company. “It’s really that automated.”
In a presentation at the recent Network World IT Roadmap event in Boston, and in follow-up interviews, Close explained that IT was becoming a bottleneck at Danone, especially because its various subsidiaries all had their own IT departments, with separate staffing and infrastructure.
“All of the sites had different systems,” he says. “They used different networks, they didn’t have the same antivirus software, the same monitoring tools, not even the same operating systems in some cases.”
Homogenizing IT
In an effort to cut costs and increase efficiencies, Danone decided to standardize on Cisco networking gear and centralize its IT services, closing data centers in Montreal and White Plains, N.Y., and building one main data center in Manchester, N.H. Close says the idea was to reduce staffing while improving the overall level of service.
For example, five sites that previously had their own IT groups each needed a database expert. “Our idea was that if we centralized IT, we’d have a primary database engineer and then a backup engineer handling all five sites,” Close says. “We’d have better service, since everyone had a backup, and we’d still have a net savings of three engineers.”
While the company retained some of its original staff, it ended up hiring most from scratch. Close says this was primarily because the current staff either did not have the requisite skill set — for example, they were Novell engineers, but Danone no longer had Novell in place — or were unwilling to relocate to New Hampshire. “We probably kept three or four people who decided to move up here,” he says. “The rest transitioned off in about a 12-month period.”
And that left Close with a daunting hiring task. He decided to concentrate on hiring what he terms “business technologists,” or staffers with not only the requisite technical skills, but also strong customer service and business acumen. Working together with a full-time human resources staffer, Close was able to staff up successfully. “We’re about a year and two months into this, and we have everyone still on board, except one administrative assistant,” he says. “It worked out well.”
With staff in place, the next step was standardizing and stabilizing IT operations. The idea was to reduce the number of diverse systems in place by consolidating data centers, employing virtualization technology and standardizing services. That, in turn, would reduce support requirements. “There’s no way to support six different sites if they all have different types of equipment in them,” Close says. “You’d need the same number of people as you had before because of all the individual expertise you’d need.”
The plan also stipulated replacing risky infrastructure. “We focused on making sure the hardware in the network was stable, and we got rid of things that had chronic problems,” Close says. “If we had an organization where the file server was down every day, we’d say hey, that’s it. We’re putting a new file server in here and just drop one in.”
A lump in the mix
If Close was unsure he was on the right road map, what happened next made the wisdom of the whole project crystal clear. Not long into the whole staffing and stabilization process, a staffer suddenly decided to quit.
“He worked at one of our smaller facilities, a water business, and he was the only IT resource they had,” Close says. “He decided to depart on a Friday afternoon, dropped off his keys and his pager, and said, ‘I’m off to do something else,’ no two weeks notice, no nothing.”
The facility’s ERP system ran for just 24 hours before grinding to a halt. “Not only was he the only local IT guy, but he was doing a lot of things manually, rather than having automated cleanup processes and backup processes and what not,” he says. “It was a pretty big problem.”
Fortunately, Close’s new staff was up to the job. They quickly figured out what was wrong with the ERP system and began fixing it and getting it back online. It was at that point that the primary flaw in Danone’s plan came to the fore: the IT services staff lacked a standard process for handling incidents and communicating with one another and the business.
“We had great expertise in place, but no common process to fall back on,” Close says. “The staff all had different ideas of how to handle incidents, how to notify people, how to communicate amongst themselves, and that confusion added to the time it took for us to recover the system. We fixed the problem fast, and it really should have been a shining moment for us. But it didn’t come through as a shining moment because of the lack of process.”
ITIL smoothes the process
Once the ERP system was back up and running, Close regrouped and decided to adopt ITIL processes for handling incident and change management. A division of Danone in Europe had successfully implemented ITIL, so Close thought it made sense, although he underscores that the important thing is the process itself, not the specific standard.
“It’s all about having a common vocabulary,” he says. “When something goes wrong, do you call it an outage or an incident? What does the business call it? How do you communicate the problem to the business? You can be doing a great job in the back room, the tech guys can be acing everything, but if you’re not communicating properly, it really looks disjointed. And it makes the business question whether the right things are being done.”
In a matter of weeks, using an iterative, workflow-based process, Close was able to put communications and incident management processes in place. Today, all problems are tackled and communicated in a predictable, effective way.
“Now, it doesn’t matter where I’m at, if a problem happens, I know the process that everybody on my team is following to solve that problem,” Close says. “I may not know what the problem is, I may not know when it’s going to be solved, but I know the exact steps they’re going to take to solve it for the most part. And so does the business side. It’s predictable.”
Steps to ITIL success
Danone CTO Close offers five steps for implementing standard processes:
1. Get senior-level buy-in. Danone had senior management’s attention because it had already suffered through a major outage. “But it’s really better if you can get senior management buy-in before an incident happens,” Close says.
2. Tailor the process. While ITIL gave Danone a good starting point, Close says his group was successful because it tailored the ITIL process to the business. “I don’t suspect that even another yogurt company’s incident management process looks like ours,” he says. “Our company culture is different, and the way we like to communicate is different. But we still follow the ITIL framework.”
3. Leverage third parties. Implementing the entire ITIL stack takes time, but Close says organizations can speed the process by turning to outside vendors. “We’re looking into companies like BMC, which sells help desk and change management products with the ITIL framework behind them. It jumpstarts the whole thing.”
4. Use an iterative workflow. Danone built its IT processes based on multiple rounds of feedback from all concerned. “The team working on communications built a process, and revised it as a team,” he says. “Then they brought in the remainder of the IT group and revised it again. And then, they talked to the business units, and added in their input. So we kept modifying as we went.”
5. Test. Once the process was in place, Danone did real-world tests to ensure things ran smoothly. “We gave our support people a specific situation and let them walk us through the process, sort of practicing what kind of communications they’d send out and how they’d send it,” he says. “And then we’d coach them as they went along.”
Cummings is a freelance writer in North Andover, Mass. She can be reached at jocummings@comcast.net.
