Measure Your User's Experience for Good VoIP Quality

Good VoIP quality requires measuring your user's experience. Why? Because we're talking about talking, not e-mail. A VoIP application's job is to recreate a real event-a conversation between live people who expect a good quality experience. Voice-related traffic needs to be delivered quickly and consistently to achieve a smooth and natural conversation.

First let's distinguish between quality of experience (QoE) and quality of service (QoS). QoS deals with how well the network delivers packets. QoS techniques like classification and DiffServ differentiate network packet streams and give preference to streams sensitive to factors like packet loss and jitter. QoE in contrast involves measuring and making sense of the actual user experience.

Unlike data, real-time voice streams are carried by UDP not TCP--and UDP doesn't recover lost packets whereas TCP does. For data applications small packet losses causes minor responsiveness slowdowns, but for voice a little packet loss dramatically undermines the user experience.

Run-of-the-mill data networking test tools aren't designed to track down problems causing packet loss in real-time streams. The only means to detect when a network path is losing packets is to test the stream itself or to simulate the stream. The measurements must provide visibility into the whole network from end to end.

At this point we can hear you say: "But my tools monitor packet loss so I am all set." You may think you are because you have a tool that looks for packet drops on network routers. The tool probably monitors for output queue drops on critical links--like where large numbers of links converge or at boundaries where traffic flows from a high speed link (say a LAN link) to a lower speed link (like a WAN link). Although this is useful information, it is not enough.

The reason is that packet loss can occur anywhere along the path. Loss often happens because a half/full duplex mismatch prompts a layer 2 error that remains undetected by the Ethernet collision mechanism because of the mismatch. The packet is dropped because it's incomplete or has a bad checksum so no router queue ever sees it. The router reports that there were no packet drops, but this packet never even got there! A noisy copper line, bad Ethernet cables or even a router interface that is unmonitored can cause similar results. The fact is that even though your tools tell you all is well, the application is still missing data and the voice quality will plummet.

But just delivering packets to their destination is insufficient to ensure a quality experience. For example, suppose that the voice reproduced by your VoIP system has a poor signal-to-noise ratio. Even if every packet is delivered, noise can garble voice.

Your voice stream may start in the PSTN, pass through a gateway into a VoIP realm, continue across a SIP trunk, have its compression type recoded for compatibility with another VoIP vendor's equipment, and finally be reconstructed on a VoIP handset. More transitions than this are not just possible, but likely. Such a scenario can introduce problems including noise, bad recoding, echo from long end-to-end delay, and quality loss due to the compression type used. The end result can be bad voice quality despite the fact that the network delivers all packets on time.

Thus, quality of experience (QoE) measurement tools are essential get to the bottom all of these problems. These tools reconstruct the voice signal and run signal processing algorithms to assess call quality. QoE measurement tools usually deliver a score that more accurately reflects the true user experience than simply examining network packet characteristics. QoE standards have been defined by the ITU to provide a consistent way of measuring expected voice call quality. This helps compare measurements between vendors, service providers, and different implementations.

QoE measurement tools are new, but they are maturing and getting better fast. If VoIP is important to you, we recommend you get one of these tools to help you establish and maintain VoIP quality.

• Network Management
• VoIP