Are you content with the way IT is viewed in your organization? If not, you need to study the work of Dr. George Westerman, a Research Scientist in MIT Sloan’s Center for Digital Business (CDB), who suggests you rethink the metrics you measure and how you communicate IT’s value.
Also read: Revitalizing IT’s worth
Westerman recently teamed with Saby Mitra, an associate professor in Georgia Tech’s College of Management, and Vallabh Sambamurthy, professor of Information Technology in Michigan State’s Eli Broad College of Business, to publish a study with the Society for Information Management’s (SIM) Advanced Practices Council titled “How do CIOs measure and communicate IT performance?”
Network World Editor in Chief John Dix caught up with Westerman at his MIT office to hear what the team learned from the 23 CIOs they examined for this SIM study.
In your research with Mitra and Sambamurthy you explore how IT executives can achieve more success by focusing their efforts on driving business value. Give us the elevator pitch.
GW: One of the biggest failings in IT’s ability to deliver value is not being able to talk about value, not being able to get beyond classic IT metrics that nobody else cares about. So what we’ve done in this research is show the right way to talk about value and how you move up the value hierarchy.
We asked the question, how do effective CIOs talk about value and what’s the process they use to become more effective? We find that, first, there’s a translation to take the IT metrics and make them meaningful for people. But that’s just the start. Once you’ve done that and you’ve got some credibility on delivering the IT work, then you want to move to business metrics for your operations and project benefits delivery. Then CIOs are able to move into the innovation and strategic IT value space from there.
What metrics do successful organizations collect?
GW: Many people publish the metrics that are easy to get: availability statistics, project on-time measures, these kinds of things. But in our case studies we find that, as you get beyond a basic level of goodness in IT, nobody cares about those measures. They really care about what we are doing to improve the business. And most IT people aren’t providing those numbers. They’re critical, but harder to get.
In our study with SIM APC and 23 CIOs, we found there are three categories of metrics to measure. One is operations metrics, how well is the company running. No. 2 is project metrics, how well are we changing the company. And three is innovation metrics, how ready are we for the future [picture a matrix with these being three categories along the X axis].
Within each of these categories there are three levels of metrics [the Y axis], with the bottom one being IT. So typically the value conversation, if it’s happening at all, involves very technology-specific numbers – how many nines, how high is user satisfaction with the help desk. But CIOs succeed better when they’re able to move up to the two higher level business metrics in each category.
At the top level is business revenue, product performance, those kinds of things. Making the link between IT and these top level measures can be a little tough. But we also found a powerful middle level, which is business process performance — what’s the quality, the efficiency of business processes? There’s a direct link to IT there, and there’s also a business process link with most executives in the company. So, what you want to do as a CIO is get out of your IT-centric metrics and talk about those business metrics.
JM Family Enterprises, for example, put together a dashboard that showed not just whether the servers were running, but whether business processes were running. And it was tremendously useful because business executives could see processes were running pretty well most of the time, and when things went down, the executives could see they came up reasonably fast. But boy, it was also tremendously helpful for IT because the people in the data center suddenly saw, “Hey, it’s not me down, it’s warranty sales that are down.” And the level of quality went up really fast.
Once they had some decent measures of business process performance in terms of failure and reliability, they could then get to the unit cost of the business processes because now they had that kind of information. That helped them make better decisions on what processes to invest in.
What metrics are used to gauge innovation?
GW: At that bottom level we are just talking about making the platform better, bringing new technologies in at the right rate. The upper two levels are more about business innovation, and it depends what you’re trying to do. Some people are measuring innovation in terms of the dollar impact; “Can we see how much money we’re getting from new products, new services?” Others are using innovation as a way to change their culture, so they’re measuring not only the dollar impact but also involvement of people, the number of ideas they look at, these kinds of things.
How do you get to the business metrics. Do you jump in with both feet or is there a sensible progression?
GW: First, CIOs should ask their business colleagues: What are the key measures you use to manage your business? And if they don’t know, help to think through how you can help them measure. Then, find ways to measure how those things change because of projects. Then you get the chance to suggest new ways to improve those measures. But it takes focus.
What we found is the CIOs who are most effective in communicating tend to focus in certain areas. If you talk about too many things you lose your focus and people following you lose their focus. What’s critical is to focus in an area that needs help, and as that gets good, hand it off to one of your people and then move on to the next piece.
For example, we see a regular pattern of people that move up from IT specific metrics to business operations kinds of metrics. Obviously they then need to delegate who is looking at the IT specific metrics. Similarly, we see people moving from an operations focus to a project focus, and they need to delegate that operations stuff so it keeps being managed well. If you can’t delegate it, you can’t let go of it. And then you can’t focus on changing what needs to be changed next.
Can it be hard to realize when you have to change focus?
GW: It’s a leadership challenge. As the leader of the IT organization you should be going out and trying to find the place where your intervention is most useful. And so if you’re spending a lot of time focusing on something that’s already pretty good, it’s time to move onto something else.
One other thing though is to remember that CIOs influence a lot of metrics that they don’t own. And this is especially true in the project and innovation space. In the project space, for example, CIOs usually can’t take ownership of business benefits, but they can certainly facilitate a process to help the businesspeople take ownership.
At times do they have to fight to be heard?
GW: It’s interesting because we heard a lot of people talking about not being as strategic as they would like and wishing they’d get the chance. But the most effective CIOs didn’t wait. They stood up and started taking charge of the IT value conversation. They laid the groundwork by showing they were good at their area, then made a case they needed to change projects, and made a case that innovation mattered.
There may be a limit to how strategic you can be just because of the nature of the company. At one biotech company we studied, for example, IT will always be back office. They’ll never really get into a lot of innovation so they’ve done a lot with operations and projects.
But many times the culture of the company is used as an excuse. We’re seeing CIOs getting a full seat at the table over the course of three or four years by just standing up and taking charge. Nobody will listen at first, but you just slowly build your credibility.
And your 2009 book with Richard Hunter, “The Real Business of IT: How CIOs Create and Communicate Value”, is supposed to show people how to increase their value?
GW: That’s the whole idea. If the best way to increase the value of IT is to talk about value, then there are patterns you use in talking about value. That book is tremendously useful for showing IT leaders a path they can take to increase the strategic role of their units. Our new research aligns with the book and adds new insights. You start by focusing towards the lower left of our matrix, on IT Operations and IT Project Performance. As you get good at that, you move up into the business operations. You move up into changing the way you prioritize and assess benefits of projects. And that gives you the opportunity to say more about what projects and what innovations ought to be done later.
Do your efforts dovetail with earlier work on metrics?
GW: A lot of work has been done on benchmarking, how well does your performance match the metrics from others. It’s very useful stuff, but we’re taking a different focus. We’re focusing less on what the specific level of a certain measure should be — what should your percentage of successful projects be, how much return should you get on a project, etc. – and focusing on how you use metrics to steer the conversation. The metrics are important only to the extent that you use them to have a better conversation and to make better decisions.
Does the movement to cloud, presuming it happens, change what folks measure?
GW: It’s interesting because cloud is a technology, but it’s basically doing what we already do, which is delivering some kind of operations. Eventually, the cloud may move to business processes, too. But, at the moment, what we talk about is technology. So I think cloud is going to just be more of the same in terms of how we communicate value. It may, however, drive more scrutiny over whether internal performance is, or can be, as good as vendors provide. That means we may start to see some tough conversations on what the IT unit should be in the business of doing internally.
Does your work dovetail with ITIL or COBIT or any of those efforts?
GW: One of the biggest issues we have in measuring is not knowing what the numbers are. If you’re not systematic in your management, it’s hard to be systematic in your measurement. And what you get out of doing things like ITIL is a very systematic way of managing your IT processes. The metrics can emerge from that.
Having said that, do you have to put ITIL in to get decent metrics? No. And, by the way, the metrics you get from internal IT process improvements will typically be only one part of our matrix. You still need to drive a conversation about business metrics and how IT can help improve those.
When you add it all up, how would you recommend that people get started if they’re not doing some of it already?
GW: The main takeaway from both the 2009 book and our new study is that, if you want to be more strategic, you can’t just wait to be asked. You need to stand up and take charge of the value conversation. The way to do that is to start from wherever you are and gradually build up credibility. Focus on a certain area, and get really good at that. Usually that’s the area that you manage, IT. Then take the next step. Figure out where you can move — Is it operations? Is it projects? — to really improve capability and metrics in those areas. Then, you’ll get the opportunity to make a bigger and bigger strategic contribution. The company benefits, and so do you. So get started!
