VoIP monitoring tools move toward maturity with new levels of accuracy and troubleshooting techniques

In a very close race, Touchstone edges out the competition

In our third review of VoIP monitoring tools, products from ClearSight Networks, Codima Technologies, JDS Uniphase, WildPackets and Touchstone Technologies, all products were able to monitor the test network. They differed primarily in how accurately they assessed the voice quality of degraded calls, how well they could provide expert advice -- what the error or logged event meant and suggestions on how to fix and what else to look for in the network relating to this problem -- and, lastly, how many VoIP-specific alerts and other criteria they could monitor.

The five tools we reviewed dig deep into VoIP networks to root out the causes of problems and provide network administrators with easy-to-understand, graphical displays of VoIP activity.

In past reviews, we focused on the ability of these products to give network administrators insight into their company's VoIP networks. This time around, we wanted these products to help diagnose and repair the VoIP problems they see. Specifically, we looked for proof that they accurately isolate the problematic VoIP calls and facilitate for proactive management of VoIP streams.


How we tested these VoIP tools

Archive of Network World tests

Subscribe to the Network Product Test Results newsletter


Products from ClearSight Networks, Codima Technologies, JDS Uniphase (JDSU), WildPackets and Touchstone Technologies were tested by Miercom engineers in their central New Jersey lab. (For a full listing of available VoIP monitoring tools, see our Buyer's Guide.) 

Testing focused on six categories: setup, configuration and deployment; display and interface; real-time monitoring; diagnostics and troubleshooting; reporting, alerts and triggers; and advanced features. Some of those advanced features include extensive codec support beyond the G.711 or G.729/.729a that most analyzers support; Perceptual Evaluation of Speech Quality analysis for testing networks; R-factor correlation; the ability to measure video quality; special provisions for analyzing voice over Wi-Fi; and simultaneous multisegment analysis. 

All the products tested support Session Initiation Protocol (SIP) and can be used in multivendor VoIP environments. Codima's product did have a tighter integration for monitoring Avaya's VoIP line because Codima's first deployments were rooted in Avaya deployments. 

They differed primarily in how accurately they assessed the voice quality of degraded calls, how well they provided expert advice — the meaning of the error or logged event, suggestions on how to fix it, and what else to look for in the network related to the problem — and how many VoIP-specific alerts and other criteria they monitored. 

Most products could inject or replay traffic as well. Being able to replay the audio stream and hear for yourself what the suspected bad VoIP call sounded like offers a reality check to troubleshooting when it's coupled with mean opinion score (MOS) and R-factor scores. (MOS is a voice-quality rating that ranges from 1.0 [worst] to 5.0 [best].) As a side benefit, this feature allows you to apply traffic back on the network for limited stress testing, predeployment-site-survey purposes or other, more involved troubleshooting tests of the underlying network hardware in the converged network. 

In general, the products we tested offered more accurate voice-quality measurement than past versions did, made it more possible to drill down and see specific call detail and measurement data while allowing managers to monitor a greater aggregate of trunks or call groups, and included more-advanced troubleshooting abilities and better expert advice on where to look for a fault and how to interpret the alerts or warnings. 

However, the products all seemed unable to provide trend-analysis data beyond a week, and generally could not do so for periods beyond a month. Most vendors seem to be moving toward integration with third-party trend analysis tools to digest the huge amount of data these VoIP monitoring tolls generate in real time over time. 

This one is really almost too close to call, because there is only a .32 difference (on a five-point scale) between the highest- and the lowest-scoring product in this test. Touchstone's WinEyeQ provides the most accurate VoIP statistics and voice-quality assessment in real time and very narrowly wins our Clear Choice Award. It served up details on just about every converged network problem we threw at it. From SIP proxy-setup problems to underlying network problems, we could easily isolate and see to a gnat's-eye-level of engineering detail what the problem was. 

ClearSight Analyzer Distributed offered a very complete, rich set of features for managing networks, VoIP and beyond. It was best at isolating faults in our VoIP network and troubleshooting more than a dozen common converged-network problems. 

JDSU's PVA-1000 is a very cost-effective, distributable VoIP analysis and management tool. Strength in protocol analysis and decoding ability, which are where JDSU's roots lie, were evident. The low-cost, $8,000 complete package was impressive, with its unlimited-distribution client and ability to put a "click here" interface for desktop users to track bad VoIP calls. 

WildPackets' OmniPeek Enterprise went beyond VoIP as well, and included e-mail and instant-messaging-rebuild replay capabilities. It also featured an expert commentator to help troubleshoot problems detected on the VoIP network. Codima's Toolbox's autopopulating graphical interface was the most intuitive. Endpoint IP phones appear as individual icons that allow for easy drill-down for call statistics. We give it outstanding marks for ease of use and its network-autodiscovery capabilities. 

Although we encountered an occasional hiccup, a few minor glitches and some petty shortcomings, we by and large were impressed with all five VoIP-analysis tools, and would have no reservations recommending any of them.

Smithers is CEO of Miercom Consulting and Integration  a technology integration and testing firm. He can be reached at rsmithers@miercom.com

NW Lab Alliance

Smithers is also a member of the Network World Lab Alliance, a cooperative of the premier reviewers in the network industry each bringing to bear years of practical experience on every review. For more Lab Alliance information, including what it takes to become a member, go to www.networkworld.com/alliance.

Learn more about this topic

Digging deep into your net with VoIP analysis tools

09/18/06

VoIP analysis tools

03/28/05

Join the discussion
Be the first to comment on this article. Our Commenting Policies