Stepping up to the IP telephony management table

We determine how well IP PBX vendors are making extra management tasks easier to bear

A converged IP PBX system brings with it new and expanded management responsibilities for things such as voice quality, quality of service, bandwidth consumption, security, voice-over-IP parameter settings, IP call routing and softphone deployment.

In this round of testing, we set out to determine how well IP PBX vendors are making those extra management tasks easier to bear.

We tested products from four vendors and found that in some areas – configuration, for example – the tools were adequate across the board (for details on the IP PBX configurations submitted and pricing click here). However, for real-time monitoring, generating user-settable alarms and traps, and long-term trend reporting of IP-telephony activity – clear winners and losers are emerging.

All the leading IP PBX vendors were invited to participate in our test. Alcatel, Avaya, Siemens and Vertical Networks accepted. Shoreline Communications and Nortel declined without explanation. 3Com and Mitel Networks said they were in the middle of product upgrades. Cisco originally had agreed to be in the test but then declined saying new pieces of its IP-telephony management toolbox weren’t ready for testing.

Vendors could submit any of the management tools they offer to monitor or control an IP-telephony-only environment, and third-party packages that augmented their own products. Except for Vertical, all vendors provided a mixed bag of tools. Vertical based all its management on the native Java-based browser interface that comes free with its system.

Because vendors submitted a mix of their own plus third-party tools, we decided not to render a scorecard for this review. However, there are several areas in which vendors differentiate themselves and which could be factors in your buying decision.

Different strokes

Real-time monitoring : Only Avaya and Alcatel offer users a straightforward way to see what QoS conditions – in terms of jitter and packet loss – current VoIP calls are experiencing (see graphic). Other factors include:

Phone configuration: Siemens embraces phone-side configuration of IP-telephony parameters. For example, setting the jitter-buffer size, a key parameter affecting VoIP voice quality, can be modified only on each individual IP phone; there is no centralized, systemwide jitter-buffer control in the Siemens environment.

Vertical’s approach, by comparison, is weighted more heavily with the system administrator. IP PBX management should be centralized when possible. Implementing settings on a per-phone basis breeds inefficiency. Also with access to their own VoIP parameters, some users inevitably will experiment with their phone configuration. Also, centrally monitoring a whole network or subnet of IP phones yields a much better view of what’s happening – in terms of bandwidth consumption or QoS performance, for example – than you get by querying individual phones.

Reboot time: Another key issue is how long it takes an IP phone to reboot. Siemens was best at 20 seconds. At the other end of the scale, Vertical’s IP phones didn’t come back up for 2 minutes, 15 seconds. Almost all IP-phone configuration changes require a reboot. The difference between 20 seconds and two-plus minutes certainly affects end-user disruption and downtime, and whether changes best are applied in real time or after-hours.

Configuring QoS: Avaya and Vertical have very reasonable type of service (TOS) and Differentiated Services values implemented and enabled by default (see graphic). But neither Alcatel nor Siemens have any default QoS set. Unless the user inputs one, all VoIP traffic streams are sent on a best-effort basis, riding along with all other data traffic. Alcatel’s interface for setting or changing VoIP QoS is especially cryptic.

Events and alarms : IP PBX vendors have not done much in updating their event and alarm notifications to address IP telephony concerns (see graphic). For example, only Avaya lets users set thresholds and get alarms when bad QoS conditions are encountered. Alcatel offers some minimal capability in this regard, but Siemens and Vertical offer none.

Trend reporting : The effectiveness of long-term and trend reporting depends heavily on whether the IP PBX includes useful IP-specific data elements in its call-detail records (CDR) (see graphic). Vertical provides the richest set of IP call details, followed closely by Alcatel.

Avaya and Siemens provide very little useful, IP-specific call information in their call records. However, Avaya compensates for this with a new specialty management tool, called VoIP Monitoring Manager that closely tracks only VoIP traffic.

Management interface: Vertical’s was the most efficient and navigable, followed by Siemens. Alcatel had the most tedious and complex interfaces to use.

Product particulars

Alcatel’s OmniPCX Enterprise: It don’t come easy

Managing Alcatel’s IP PBX in most cases requires OmniVista 4760, Alcatel’s modular management software package. The server portion, with integral Sybase SQL database, runs on a beefy Windows 2000 platform with a minimum 512M bytes of RAM and at least 1-GHz CPU.

Administrators access the server via a Java-based browser interface. Besides OmniVista, we also made heavy use of Telnet directly into the call server.

The OmniVista modules each present a different interface to the user. And while they all employ a Windows-type GUI a few of the key modules – such as the configuration module – are laid out strangely and are cluttered with unnecessary objects. The result is an interface that’s difficult to navigate. This complexity was compounded because we could not get OmniVista’s online Help working properly.

Another drawback is that softphone call control and management require a totally separate Windows-based server application, which is not even accessible via OmniVista.

On the bright side, we accomplished almost every task we attempted via one or the other management applications. However, completing many of the tasks, such as resetting IP phones or gateways, involves the cryptic Telnet command-line interface. Alcatel says it plans to move some of these features onto interactive Web pages in its next major system release, Version 5.1, scheduled for September.

A very impressive and valuable feature is that the OmniVista alarm module maintains a stateful watch over events in the log and applies some degree of event correlation. When a severe condition that resulted in an alarm message is resolved or rectified, the software automatically will clear the event.

A separate useful tool is the Alcatel VoIP Assessment Tool, which comprises a Windows application and software that loads on Alcatel IP phones. From the Windows station the user directs any two Alcatel IP hard phones to conduct a user defined VoIP test with each other. The results are sent back to the Windows console, which processes the data and presents accurate QoS measurements of packet loss, jitter and delay, and relative VoIP-quality assessments.

Alcatel turned in its best performance in real-time monitoring, in part because of the added management capabilities the VoIP Assessment Tool provided. Alcatel came in just behind Avaya in the area of alarms, events and traps. Many performance metrics are accessible for use in defining events and alarms, but actually setting thresholds and event notifications is a complex and tedious process.

The overall difficulty of managing the OmniVista is high. Expect several months of experience and training to achieve management proficiency.

Bottom line: Alcatel

Management tool kit: OmniVista 4760, with configuration, alarms, accounting/metering, performance, directory, topology, and monitoring modules. Telnet command line. Alcatel Voice Assessment Tool.

Price as tested: Pricing is based on number of administrator seats (up to 10), call servers (up to 100) and total stations. For the 250- to 1,500-station range, list price for the OmniVista package ranges from $18,000 to $40,000. The Alcatel VoIP Assessment Tool lists at $2,500.

Strengths:

• Event log automatically clears events when they are resolved.
• Can accurately measure and report full range of VoIP QoS parameters.
• Issues special call records for IP calls, which provide much useful IP-telephony data.

Shortcomings:

• Soft phones require a separate call-control and management software structure.
• Online Help was not available, and we couldn’t get it working.
• Some of the key OmniVista interfaces are unnecessarily tedious.

Managing Avaya Communication Manager: A little help from my friends

The cornerstone of Avaya IP-PBX management is the Windows-based, though command-line-like, interface called Avaya Site Administration (ASA), which is just one component of the management package Avaya sent for evaluation. We tested Avaya’s Integrated Management, Enhanced-level package, the third-highest of four management-software bundles.

The package includes ASA, a Java-based Windows application, plus three other applications:

• Avaya Multi-Service Network Manager (MSNM), a customized, SNMP element manager that runs on Win 2000 and handles trap logging and SNMP polling for real-time status monitoring.
• Voice Announcement Over LAN, a tool that lets you manipulate and import .WAV voice-response messages into the IP PBX.
• VoIP Monitoring Manager (VMON), a tool that does a splendid job of monitoring VoIP activity by collecting and analyzing real-time transport control protocol reports.

Avaya also threw in a third-party call-accounting package, called eCAS from Veramark.

Avaya delivered the best overall performance in the reporting category because of eCAS, combined with the reporting capabilities of ASA and the VMON. The same is true for alarms, events and traps. All the management pieces contributed to Avaya’s top score in this category. One obvious drawback, though, is that you have to constantly shift back and forth between different interfaces.

At least half of our test criteria was accomplished via the ASA interface. Although Avaya is making ASA user-friendlier by adding more useful Java applets, selectable as icons on the main screen of the interface, it still is based on tons of cryptic commands and acronyms that are, by and large, not spelled out anywhere. The on-screen Help is only marginally useful. ASA users likely will keep and maintain a sizeable crib sheet of key commands.

The value of the VMON should not be understated. For keeping a close eye on VoIP activity and especially QoS performance, this application alone is worth the $5,000 tab for the whole management package. Coupled with the MSNM, Avaya’s performance in real-time monitoring was on a par with Alcatel and Vertical.

We ran a late-beta version of new IP hardphone code, which supported VoIP encryption. This turned out to be very easy to administer (via ASA) and performed flawlessly. VoIP media encryption stands to be a major marketplace advantage for Avaya. No one else now offers it, or likely will anytime soon, on such a clean, well-integrated basis. The Avaya encryption does not add perceptibly to latency, although it does have a noticeable – but for the most part acceptable – effect on voice quality.

The overall difficulty of managing the Avaya IP PBX is medium, based on the ASA interface. It will take a skilled user a month or so to become proficient.

Bottom line: Avaya

Management tool kit: Integrated Management, Enhanced-level package, which consists of Avaya Site Administration, Avaya Multi-Service Network Manager, Voice Announcement Over LAN and VoIP Monitoring Manager. We also employed Java-based Web-browser access to the system call controller and a third-party call-accounting package, eCAS, from Veramark.

Price as tested: Integrated Management, Enhanced-level package, costs $5,000. Veramark’s eCAS application lists for $2,275.

Strengths:

• Best overall performance for events, traps and alarms category.

• Best overall performance for trend, activity reports category.

• Very slick and well-managed implementation of VoIP encryption.

Shortcomings:

• ASA interface still largely based on cryptic acronyms and commands.
• Online Help is only moderately useful.

Managing Siemens’ HiPath 3500: Take it to the limit

HiPath Manager E is the Siemens’ main IP PBX management software. This tool was augmented in our evaluation by the company’s Manager I application that separately handles many of the IP-specific aspects of the system, and the Deployment Tool that discovers and status-checks IP hard phones.

These are all clean, clear and consistent Windows applications, with excellent, context-sensitive online Help. Manager E connects directly to the HiPath 3500 call controller and obtains updated system data via a manual database download.

We also tested Siemens’ HiPath Fault Management tool, an SNMP-based application that collects all traps issued by the IP PBX. Siemens also submitted the Traffic Analyst tool with Impact CDR option, a third-party package for call-activity reporting from Impact Technologies.

Siemens’ strongest showing clearly was in configuration. Siemens met or exceeded all the criteria and tasks in that category through its Manager E and Manager I interfaces, coupled with the ability to Web-browse directly into any Siemens IP hard phone and set or change various VoIP parameters. Of the systems tested, only Siemens supported this per-IP-phone Web-browser access.

These tools work together pretty well on a live, IP-oriented HiPath 3500 system as far as configuration, but they are fairly limited from most other perspectives. Siemens brought in an IP PBX configuration with two call controllers – essentially two discrete systems – for our simulated two-site topology.

Managing two discrete systems was disjointed and duplicative. Everything you did on one – defining numbering plan, call routing, IP-phone user templates – had to be redone for the second system. Siemens offers a high-level umbrella software package for managing multiple systems, the 5000 RSM, which synchronizes and consolidates certain management aspects. But with a starting price of $7,400 for a 5000 RSM that handles up to 15 call controllers, it is not economically viable for only two systems, and Siemens didn’t include it as part of its management tool kit for this review.

The HiPath Fault Manager provides an auto-discovered topology map, somewhat akin to what HP’s OpenView does, but Siemens’ is not a full-function SNMP manager. There’s no management information base browser for retrieving specific management values, for example. There’s a clean event log, but there’s no way for a user to set threshold values to control when events, in the form of SNMP traps, are issued. Overall, this tool is of limited practical use.

The third-party Traffic Analyst application works by manipulating the call detail records that the system sends it. However, there are not many useful IP-telephony-specific fields in the Siemens call-record format, so the reporting capabilities from an IP-telephony perspective are limited.

The relative difficulty of managing this IP PBX is medium. Expect proficiency within about a month.

Bottom line: Siemens

Management tool kit: HiPath Manager E, Manager I, Deployment Tool, HiPath Fault Manager and Traffic Analysis with CDR Option.

Price as tested: Initial cost of the Manager application is $400. The Manager I and Deployment Tool applications come free when the user acquires Manager E. The HiPath Fault Manager costs $1,600. Traffic Analyst with Impact CDR costs $1,900.

Strengths:

• Administrator can Web-browse into any IP hard phone.
• Best context-sensitive, on-screen Help of systems tested.

• Some very good diagnostic capabilities.

Shortcomings:

• Call Detail Records lack useful IP-telephony details.
• Fault Manager needs more SNMP NMS features.
• Some key VoIP parameters are set per phone, but should be centralized.

Managing Vertical’s InstantOffice Model 3000: Take it easy

Vertical’s InstantOffice is at the low-end of the IP PBX capacity scale. The Model 3000 is offered in configurations as small as eight phones. In most key regards, however, this system’s management fares as well as, and in a few respects even better than, the IP PBX big boys.

InstantOffice has long supported IP trunking between remote systems, but only picked up IP hardphone station support in January. InstantOffice does not support a softphone, however.

All the system management happens via the free Remote Management Console (RMC) interface, making Vertical’s the lightest and least-expensive management toolbox we tested.

This is a Web browser and Java-based interface installed directly into the Windows NT-based call controller that drives the system.

Vertical also offers a high-end, umbrella management package for multi-system management, called Multi-Site Manager for about $500 per system managed. Its functionality is limited, and it does not replace the RMC, but rather provides a high-level overview of multiple systems. It was not included in this test because Vertical says it generally is employed if you’re managing 25 or more systems.

Because special applications are not needed on your management console/laptop, access is quick and straightforward. All management actions are accomplished via about three dozen icons that are arranged on the main screen in four groups: General Admin, PBX and Application Admin, Data Admin, and Trunk Admin. It is easier and faster to find what you’re looking for via this interface than any of the other systems tested.

The context-sensitive, on-screen help accompanying the RMC interface is very good, and is very close to Siemens’ as far as usefulness, information content and succinctness.

Vertical offers the most efficient screen for setting QoS values in VoIP packets, letting TOS or Diff-Serv values be entered in hex, decimal or binary, and automatically translating between these. The system does not now support Layer 2 virtual LAN (VLAN) tags as a QoS mechanism, however.

Vertical’s tools do an exceptional job in real-time monitoring, but there are few integral features for summarizing data in reports on a long-term basis. However, Vertical’s CDRs contain many IP-telephony details, including such QoS metrics as jitter and packet loss.

And exporting these CDR files is very simple and straightforward. With some work, users can readily produce long-term and trend reports by processing exported CDR files in a third-party application such as Microsoft Excel.

Relative difficulty of managing this IP-PBX is low. Expect proficiency within about two weeks.

Bottom line: Vertical

Management tool kit: Remote Management Console.

Price as tested: The RMC browser- and Java-based interface is free with each system.

Strengths:

• Rich set of VoIP configuration options, especially for WAN bandwidth optimization.
• Very good, context-sensitive, on-screen Help.
• Useful IP-telephony details contained in CDRs.
• Excellent real-time monitoring and diagnostic capabilities.

Shortcomings:

• VLAN tagging not supported for VoIP QoS.
• IP phones not reporting bandwidth statistics.
• No softphone support.

Mier is president of MierLabs, a network research and product test center in Hightstown, N.J., which focuses on IP telephony and its management. He can be reached at emier@mierlabs.com.