The first fundamental requirement of any VoIP analysis and troubleshooting tool is that it report accurate information while monitoring a network. The four basic measurements -- voice quality, latency, packet loss and jitter -- are key metrics that must be assessed accurately and in real time. The key differentiator in this round of testing was the accuracy of the tools in measuring these four key metrics.
The first fundamental requirement of any VoIP analysis and troubleshooting tool is that it report accurate information while monitoring a network. The four basic measurements — voice quality, latency, packet loss and jitter — are key metrics that must be assessed accurately and in real time. The key differentiator in this round of testing was the accuracy of the tools in measuring these four metrics.
The second fundamental requirement of any VoIP analysis and troubleshooting tool is usability. If it is difficult to set up, if the configuration challenges even experienced VoIP engineers, if it cannot be configured to scale with the environment it is designed to monitor, the requirement for accuracy doesn’t really matter.
Touchstone's WinEyeQ came out on top in our testing based on how well it nailed these two fundamentals. WinEyeQ has a clean, intuitive interface that is very effective whether it's watching one call or thousands. There are no worries about IT administrators or other computer-savvy operators not being able to deploy and use this product.
As for accuracy, WinEyeQ also provided the most precise voice-quality and network-operations statistics of all the products tested; it came closest in pinpointing actual MOS, R-factor and jitter conditions. WinEyeQ was always within 0.1 of the "actual" MOS rating for voice quality, and within 10% for jitter and latency measurements. It did so on the first attempt, whereas most of the other products required extensive tuning to detect our network's problems accurately. WildPackets' OmniPeek Enterprise was the closest to WinEyeQ's benchmark, coming within 0.2 of the rated MOS expected, but it was not consistently accurate.
Ten of the 24 network-troubleshooting tasks we challenged the vendors to detect and react to depended on measuring voice quality. All of the products tested could detect these problems and were capable of assessing voice quality, but WinEyeQ's greater accuracy in measuring made for more accurate reporting of events, and alarms and fewer false-positive notifications when it assessed the network. This is important not only when it comes to finding problems but also in terms of preventing unnecessary work on the part of a VoIP administrator. If a VoIP analysis tool inaccurately senses a problem — perhaps too much jitter — it can trigger a domino effect of other false alarms.
We believe WinEyeQ's spot-on performance has much to do with the fact that it was designed from the ground up as a VoIP analysis and monitoring tool.
Despite its ability to hit even the tiniest VoIP-network detail squarely on the head, when it comes to identifying the problems to which those details correlate, WinEyeQ's interface design is clean and simple. There's some unnecessary glitz, but it provides everything you need to dig down in to do packet-level analysis for VoIP and video transmissions in order to isolate, verify and troubleshoot a problem. The interface's efficiency, organizational structure and consistency are a notable improvement from the last time we tested this product.
The product provides significant drill-down capability, letting network administrators precisely locate the sources of VoIP problems and make the adjustments needed to improve or preserve quality. It also provides a call-by-call report card and stream-quality indexes. Together these elements let administrators quickly find poor-quality calls and help explain the reasons behind performance problems. For example, we were able to isolate the one bad call out of a hundred calls placed during one test scenario.
WinEyeQ proved capable of analyzing and providing excellent reporting on as many as 1,000 simultaneous calls, and its alarms were not only accurate but also issued in real time and often more quickly than the alarms provided by some of the other products we tested. It was quickest — its response was almost immediate, for example, in our network-troubleshooting tasks — to detect an unresponsive SIP registration server.
WinEyeQ is available in configurations ranging from carrier-grade, customer-premises distributed systems to stand-alone analysis tools. In this test, we worked with the WinEyeQ Professional version, which can function as a stand-alone analyzer or a distributed probe within an entire VoIP management system. We installed it on a span port on a managed switch, a typical installation scenario for a small organization. For larger organizations, Touchstone recommends plugging WinEyeQ into a mirrored port or using a line tap to guarantee perfect measurements and complete fault-tolerance. WinEyeQ also can be deployed in a distributed fashion, and for that the company offers a Web-based GUI and management framework that lets you poll those distributed probes, as well as push out software updates to them.
WinEyeQ provides analysis of standards-based voice and video protocols, including SIP, H.323, Media Gateway Control Protocol (MGCP) and Megaco. It analyzes Real-time Transport Protocol (RTP) audio and video streams, and Real-time Transport Control (RTCP) and RTCP XR calls, regardless of the protocol used to establish the calls. Because the product focuses on VoIP, there is no in-depth analysis of data from other data protocols, unlike most other vendors' wares. It handles about 650 metrics per call, while many other products provide significantly less coverage.
WinEyeQ offers a wide range of preformatted or boilerplate reports, including call, error, watch, session, alert, alarm and endpoint summaries. All the vendors tested provided the same, some perhaps too much, reporting — if there could be such a thing as too much information.
WinEyeQ came up short in a couple of areas. It lacked the expert commentator ability that other vendors, such as WildPackets and Codima, provided for their tools. The network conditions for which we were seeking assistance in troubleshooting were detected, but no advice was provided on what we should do or otherwise check to further troubleshoot the problem.
Also, context-sensitive help was lacking, and certain items — such as those in the file menu — were somewhat context sensitive but not in the sense that we expected. However, using the F1 key in some areas sends the user to the correct area in the Help menu.
Learn more about this topicDigging deep into your net with VoIP analysis tools
09/18/06VoIP analysis tools
“Wake up!” the good folks a Merriam-Webster just tweeted. “Sheeple is in the dictionary now.”
Wireless LAN users can’t just stay comfortable in the 5GHz realm – the older 2.4GHz frequency bands are...
A review of 18 companies that offer free cloud storage
Sponsored by Silver Peak
Sponsored by Aquantia
The legal fight between Apple and Qualcomm on licensing modem technology is turning uglier every day.
F5’s new President and CEO, Francois Locoh-Donou talks about his vision for the app delivery...
Companies of all sizes and types are rethinking not only how their employees work, but also where that...
Use cases in industries such as aeronautics and chemicals are a proving ground, and a roadmap to...