We are often asked to research and document case studies showing the return on investment for an application performance management (APM) technology that improves performance and leads to financial benefits. This is classic ROI thinking. You spend money to gain money. The benefits exist, but classic ROI case studies for APM technologies simply cannot capture them. Let us explain why.
Calculating an ROI involves documenting cost savings, and/or revenue increases for a given investment. In both cases you need to compare the financial situation before and after implementing the APM technology.
Cost Savings
The savings side in the classic ROI usually involves spending less on people and/or infrastructure than you would without the APM technology in place.
You probably know how much time it takes to resolve an incident. An APM technology that provides insight can enable your staff to work faster. The technology may even automate steps so your staff doesn’t need to do some tasks at all any more. These improvements can reduce staffing needs. Staff savings are fairly easy to quantify if they are significant enough to save a full time equivalent (FTE)–i.e., a person. You have to actually reassign or terminate the person to realize the benefit.
But what about other cost savings from faster problem resolution? For example, do you know the cost of downtime for each critical application? How can you capture that in an ROI analysis?
Many ROI analyses include a cost associated with poor capacity planning–i.e., spending too much on unnecessary resources like the expensive high speed circuit that is only 5% utilized. But these costs are often minor and rarely realized since growth fills the void. At best the rate of capacity cost increases is slowed.
Regardless of the scenario, the cost reduction formula is ROI = (Old cost – New cost)/APM cost. The results can be expressed as a percentage gain if amortized over time or as a payback period if calculated on a cash basis.
These cost savings fail to reflect improved application performance, merely the ability to maintain the same level of service more cheaply.
Increase Revenue
Many APM technologies, processes and best practices significantly improve application performance. In our enterprise work we find the most often cited APM metric is application response time as seen by the user. This is confirmed in a recent report by Enterprise Management Associates entitled “Quality of Experience (QoE): the Ultimate Collaboration – How Real Deployments are Succeeding and Why”, which shows that the top metric applied to QoE is “response time/ responsiveness per application” (page 15). In fact, 3 out of the top 4 metrics dealt with response time (the fourth was availability).
Now comes the interesting part. You need a response time baseline before deploying an APM approach in order to compare the before and after situation. But we find most companies are very poor at measuring and documenting response time. Enterprises often begin measuring response time after a big upgrade. We ask them on the eve of the upgrade: “So what are you going to compare your results to since you don’t have data on today’s system and after the upgrade today’s system will be history?” The reply is often: “You are right. But we don’t have budget to measure today, we will buy the measurement system as a part of the upgrade budget.” By the way, guess what we’ve seen happen in the end? The upgrade got delayed and ran over budget. Something had to be cut along the way, and the anticipated measurement system was never installed. So they will never know the true value of their investment
But your enterprise is smarter and you do know response time before and after the upgrade. Good! Now you must overcome the second hurdle–quantifying the monetary value of the better response time. What did it mean to the business? Here are some of the benefits we have seen.
- More users served. Because the application is now more useful to more users in more locations under more access conditions, we often see application “uptake” improve.
- Greater employee productivity.
- Extended geographic reach. This permits enterprises to extend their business into previously out-of-reach regions of the world.
- Greater agility in designing products and delivering products and services.
- Improved collaboration enabling project teams from many locations to work on projects without the need to travel.
- Better online “conversions”, i.e., more online shoppers buy.
- Higher sales per customer.
- Greater uptake for new applications. This enables enterprises to retire legacy applications faster.
These benefits can make you more competitive. They make it possible for you to differentiate yourself from the pack and grow your revenue. But monetizing the benefits is complicated because many other factors also influence revenue, making it hard to show: old response time contributed X to revenue, while new response time contributed Y. But if you can, then the ROI formula is really simple (Y – X)/APM cost.
Conclusion
If your enterprise is just starting on its APM journey and you have just a few labor-intensive tools, you can easily cost justify your purchase with a cost reduction ROI. But if your company has already invested in automation, and troubleshooting is fast–then adding APM technology will have a marginal classic cost savings ROI.
Our advice is for you to invest in solutions that you are confident will improve application performance for your users and to tell your management that what matters most to the business is making your company more competitive and financially healthy. Let’s get away from classic ROI’s for APM solutions because they just don’t apply any more.
