Apdex Users Report Application Performance Benefits

The third annual Apdex Alliance symposium was held this week in conjunction with the CMG (Computer Measurement Group) international conference in Las Vegas.  There were twice as many attendees as last year, which is an accomplishment in today's economy.  The event was a great opportunity for users, vendors, and Apdex practitioners to learn from each other.  The most compelling sessions were case studies presenting ten very different examples of how Apdex is being applied.  The Apdex report examples showed a lot of innovation in easy-to-understand visualization.

The end-user cases included an HR software-as-a-service case, Internet access from inside the corporate network of a global manufacturer, SAP at a food company, SAP on the Internet, ERP at a manufacturer, and a county e-government portal.  Have a look at the presentations at the Apdex website. Apdex's most common use is to track SLAs, discover hidden performance issues, and capture effects of infrastructure changes.

Bruno Savard from on-demand talent management company Taleo gave an excellent case study. Taleo helps companies recruit highly talented candidates, develop them into top performers, and manage their performance for outstanding business results.  They deliver their service as an impressive SaaS with 3,800 customers in 200 countries managing 107,000,000 candidates, with a traffic load of 36M transactions per day.  Savard implemented Apdex after hearing about it at a previous Apdex symposium.

The Taleo Apdex implementation uses synthetic transactions sent every minute from just inside the firewall through all of the application tiers.  The response time for each full task (page load time) is measured by a Coradiant TrueSight device.  Since this is a measure of performance at the data center, the Apdex T value is a very fast 100 milliseconds. Savard determined T by an imperial method--he tested for a week during which the service was not experiencing performance issues.  He then selected the response time at the 98^th percentile of all samples.  This turned out to be a very useful Apdex T which has been producing Apdex scores that uncover true performance issues.

Taleo prominently displays the Apdex score timeline in both the data center operation room and the customer service call center.  This Apdex reporting system has been in place since April 2007. During that time the overall Apdex score has improved and Apdex has uncovered problems such as a database issue, server memory constraints during peak hours, and VMware configuration errors.  Taleo has also recently integrated Apdex into its capacity management process.  Apdex helps Taleo detect live issues affecting the user experience and provides meaningful and proactive alerts.  The customer service center uses the real-time Apdex graph to quickly validate customer incidents.

Savard concluded with several testimonials including one from the production director who said, "The APDEX project is one of the best innovations I have seen at Taleo."

The last symposium session included presentations on improving Apdex.  Alan Ackers from Macro 4 made a case for changing the relationship between the T (tolerating) and F (frustrated) thresholds, while OPNET's Yiping Ding proposed that the currently specified relationship of F=4T is valid for Apdex in the Excellent and Good ranges of Apdex scores.  These presentations sparked animated discussion, which the Alliance will take under advisement.

Peter Sevcik presented the results of a recent Apdex Alliance member poll that shows Apdex is being applied in unforeseen situations that will require some changes to the specification.  These changes along with the topic the Apdex thresholds will be on the Apdex Technical Working Group's agenda in 2009.

We all look forward to interesting work in 2009 and the fourth annual Apdex symposium.

• Network Management
• Apdex
• Application Performance Management