Vendors were asked to supply their VoIP analysis and management product on the hardware platform of their choice for hands-on analysis in Miercom's testing lab in New Jersey.

The test network consisted of two LAN segments with Cisco Catalyst 3750 10/100/1000 switches. The LANs were interconnected via simulated WAN link with T-1 and DS-3 links. The WAN simulation was provided by PacketStorm Communications' 1800E WAN emulation product.

VoIP calls were generated between the two LAN segments using a WinSIP call generator and snom SIP phones. Our test bed was calibrated using third-party Perceptual Evaluation of Speech Quality instrumentation.

During testing, a mix of calls with G.711 and G.729 codecs was placed between LAN segments. The products were expected to identify the number of calls and discern as much detail as they could about each individual call via a port configured for monitoring the uplink connection to the WAN. Latency, packet loss and jitter were then introduced, and the products again were given the task of properly measuring the impairments. We expected that the products would be able to discern the one voice call that was impaired and identify any other problems with the VoIP communications.

We tested the tools' ability to detect and troubleshoot two dozen converged network problems. The tests were detecting overall network impairments causing jitter, latency and packet loss both at call-setup time and while calls were in progress; slow SIP server response; duplicate addresses for registration and proxy servers; unresponsive SIP registrar and proxy; Real-time Transport Control Protocol misconfiguration; excessive echo and malformed SIP call-setup messages.

When testing was complete, the products were required to produce automated reports based on the calls and other activities that transpired during the testing. We evaluated these reports' accuracy, clarity and completeness. Real-time monitoring and alerts about the specific anomalies induced during testing were noted while testing was in progress.

< Return to main test