Last time, I began the discussion around how network managers need to capture the end-user experience to fully understand how well the network and application infrastructure is performing. This time I will get more in-depth as to how to measure the performance of optimized network and application environments to ensure the end-user experience meets preset expectations.

Industry watchers from Burton Group, Enterprise Management Associates, NetForecast and Yankee Group have been discussing recently the means by which network managers could get a better read on application performance and establish a measure against which to compare performance of various applications. For instance, Peter Sevcik, president at market research firm NetForecast, last year launched the Apdex Index along with several vendors in an effort to create a formula to measure performance for various applications. The index "converts application response time measurements from many tools and services into a single number using a scale," he says.

"Apdex also benefits users because the metric, generated by multiple systems, will produce the same result as long as the service being evaluated and the definitions of 'tolerable' performance are consistent," Sevcik says. "In that respect, some of its simplicity comes from the fact that it can be compared across services and tools."

While the Apdex Index offers a way to measure performance, performance management gurus say measuring application performance relies upon understanding the underlying data sources before plugging them into a formula.

"We need simple metrics, with complex underlying design. We don't want the measurements to be misleading and to impel an erroneous change," says Eric Siegel, a senior analyst with the Burton Group. "The design should correctly weigh the factors important to the enterprise [company] and use appropriate statistical treatments."

Dennis Drogseth, a vice president of research at Enterprise Management Associates, says enterprise companies must also define what application performance success is to their organization, as the definition would likely change across IT shops and business units at varying companies.

"We didn't agree that there can be a single metric for IT success. ... Instead we agreed that there can be a single metric for effective application performance," he says. We also agreed that performance in the sense of response is just one of multiple metrics for measuring [quality of experience]."

Ann Bednarz is associate news editor at Network World.