Ensuring quality of experience

For the past couple of years, quality of service has been a source of great contention and confusion among IT professionals.

How does one ensure quality of service as experienced by the user? How do you even measure such an idea? Quality from a user perspective is almost never the same as it is from the perspective of the people delivering the service. In fact, one might argue QoS doesn't measure quality at all; in most cases, QoS is merely a set of metrics that both parties agree might correspond to quality in some way. These metrics, however, never address the user perception of the service, a notion that has been the center of such services as voice telephony for many years.

QoS, of course, has become of paramount importance when looking at converged networks. Of most immediate importance has been how to determine QoS for services such as voice telephony. The circuit-switched world has Mean Opinion Scores to measure user tolerance for noise, signal level, echo and so forth, and to rate the overall perceived quality of speech.

In voice over IP (VoIP), however, many more factors can impact perceived quality than just the amount of noise present. VoIP can have wildly different quality levels depending on the data transfer rate, the latency of packets in the network, the degree to which packets arrive out of sequence and a multitude of other factors.

Perhaps most important is the idea of "bursty packet loss," where under conditions of network congestion, packets are lost not individually, but in bursts. These bursts tend to evade technologies that might otherwise correct or repair such losses.

The mission of IT is to manage all of these issues so that the user can experience, on a very subjective level, a high level of service quality.

While it is fairly straightforward to monitor QoS elements, it is much more important to monitor those QoS elements in real time and to correlate them with a perceived service quality. To illustrate, if QoS is measured as an average over time, then it is highly likely that measurements such as jitter can be within five nines of adherence to agreed levels - yet the user experience can be way below expectations. This is because when the user really needs high service quality, during busy periods, there can be significant and detectable degradation, probably caused by bursty packet loss. Overall, the service is great, but the user experience is unacceptable.

This is the reason the current form of service-level agreement is hard to apply to VoIP. While it is easy to specify service in terms of QoS metrics, it is very hard to choose metrics that accurately reflect call quality. It is literally a question of users knowing good service when they hear it, but not when the service provider guarantees low jitter and latency.

To date there have been few vendors that have understood this. One that does is Telchemy, a start-up in Atlanta. Telchemy offers embedded software for VoIP devices such as gateways, IP phones, probes and analyzers. It uses a real-time call modeling approach to determine the users' experience, drawing from QoS metrics such as jitter, latency and, most importantly, burst and random packet loss. This makes possible accurate call quality measurements and service-level agreements that are meaningful from a user perspective.

Enterprise Management Associates recently completed a study on VoIP that indicated that two of the hurdles to its adoption are perceived low service quality and the lack of management tools. EMA has advocated a user-centric approach to SLAs for some time now. It appears that tools are now becoming available for converged service that will allow this to occur.

Usage-tracking software on tap from Tally Systems
Network World, 03/11/02

Next-generation home nets could spell IT trouble
Network World, 03/11/02

NetQoS tracks performance
Network World, 03/11/02

Keynote upgrades Web site benchmarking
Network World, 03/11/02

OpenView users lament HP's integration delays
Network World, 03/11/02

Dennis Drogseth is a director with Enterprise Management Associates, a leading analyst and market research firm based in Boulder, Colorado, focusing exclusively on all aspects of enterprise management. Dennis has extensive experience in network management platforms and products and is researching trends in management software and changing IT roles internationally. His 18-plus years of experience in high-tech includes positions at IBM and Cabletron. He has been quoted in the press and is a speaker at industry events. He can be reached via e-mail.

Audrey Rasmussen is a research director with Enterprise Management Associates in Boulder, Colorado, a leading analyst and market research firm focusing exclusively on all aspects of enterprise management. Audrey has more than 20 years of experience working with distributed systems, applications and networks. Her current focus at EMA is e-business, SMB/SME and MSPs. She can be reached via e-mail.

Enterprise Management Associates in Boulder, Colorado, is a leading analyst and market research firm focusing exclusively on all aspects of enterprise management software and services.