From a user’s perspective, application performance is response time. So you need to find out what your users' performance expectations are and identify a response time threshold that satisfies most of them—then you need to measure continuously so you can meet and improve on it.
Response time needs to be translated into something meaningful for users—simply measuring and reporting on response times is not enough. It just gives you a sea of meaningless numbers. The Application Performance Index, also known as Apdex—an open standard that defines a method for reporting and comparing the performance of software applications—is a good way to translate response times from mere numbers into something that reflects user satisfaction. It does this by specifying a uniform way to analyze and report on the degree to which measured performance satisfies users.
The Apdex methodology converts many measurements into one number on a uniform scale of 0 to 1 (0 = no users satisfied; 1 = all users satisfied). The resulting Apdex score is a numerical measure of user satisfaction with the performance of enterprise applications. The Apdex formula is the number of satisfied users plus half of the number of tolerating users plus none of the frustrated users, divided by all the total number of users:
Apdex(t) = (satisfied count + tolerating count/2) / total samples
The subscript ”t" is the target response time. The frustrated time is four times the target response time. This multiple is directly related to users' perceptions of satisfactory application responsiveness.
For example: If you have measurements for 100 users with a 3 second application response goal , and 60 users experience a response time under three seconds, 30 are between three and 12 seconds, and the remaining 10 are above 12 seconds, then the Apdex score is calculation is:
(60 + 30 / 2) / 100 = 0.75
Although some may feel that picking a target response time is too difficult or subjective, because some users may be more patient than others, research shows that people react consistently to information displayed on a screen. The greater variable is how the application is designed and what it displays, and this variable is most important to consider when choosing a value for “t".
You can learn more at the Apdex Alliance website www.apdex.org.