Apdex, the open standard we introduced last week, describes how to report application response time measurements so even business managers can understand. The methodology is measurement-tool agnostic--you can use synthetic agents or measure the real user experience. You can measure in the data center, out on the net, or on the user's desktop. However you gather the data, after an hour or a day you will have thousands of values to work with. Apdex converts all those values into a simple index that allows you to make apples-to-apples performance comparisons across applications.
The process starts with defining a ‘report group' that the index value will represent. This is the first step in reducing vast numbers of measurement samples into a meaningful data subset. Report group parameters may include such things as user group and time of day.
The index is based on three application responsiveness zones:
Satisfied: The user is fully productive. This represents the time value (T seconds) below which users are not impeded by application response time.
Tolerating: The user notices performance lagging within responses greater than T, but continues the process.
Frustrated: Performance with a response time greater than F seconds is unacceptable, and users may abandon the process.
The two thresholds of T (the tolerating threshold, also known as the target time) and F (the frustration threshold) define three performance buckets into which all the samples of a report group can be placed: zero to T, T to F, and >F. The index calculation is a weighted sum of the percentages of samples that fall into each performance zone.
The Apdex formula is the number of satisfied samples plus half of the tolerating samples divided by all the samples. Samples from frustrated users do not count.
Apdex[T] = (Satisfied + Tolerating/2)/Total
This ratio is always directly related to the users' perception of satisfactory application responsiveness. To understand the full meaning of the ratio, it is always presented as a decimal value with the target time T in brackets (or subscript without brackets). For example, if there are 100 samples with a target time of 3 seconds where 60 samples are below 3 seconds, 30 are between 3 and 12 seconds, and the remaining 10 are above 12 seconds, the Apdex formula result is:
0.75[3] = (60 + 30/2)/100
The Apdex score is always between zero and one, with one the best score possible. Like school grades, a unit-less index makes intuitive sense and reflects the way most managers think--i.e., bigger is better. An application that performs better today compared to yesterday has smaller response time numbers but its Apdex score is higher. It may sound simplistic, but trust us--you are more likely to get budget approval to "get an A" than to lower a bunch of measurement values.
Also the Apdex score operates like availability which is well understood by non-IT managers. It is expensive to improve availability from 0.96 to 0.98 and it is even more expensive to reach 0.99 availability. An Apdex score slips seamlessly into the same management discussion.
So those are the Apdex basics. We encourage you to export some response time measurements from your favorite tool into a spreadsheet and apply the Apdex formula. Let us know how it goes. We will be explaining more of the methodology in upcoming blogs. Or you can get a full immersion at the Apdex website.