City polices voice traffic

A network administrator for a city in North Carolina moves to a voice system and finds he needs management.

Bobby Parish uses third-party tools to troubleshoot voice traffic on the Jacksonville, N.C., network

For two years, Bobby Parish didn’t worry too much about how his new Cisco-based converged voice/data network performed, mostly because it didn’t cause any major problems. But when the wheels started to squeak, he started to worry.

“We have a lot of bandwidth out there and didn’t run into serious issues,” says Parrish, senior IT specialist with the city of Jacksonville, N.C. “But at the same time, I knew we were unable to proactively negate problems or to plan for upgrades, while knowing voice performance wouldn’t degrade.”

Parrish says the need for management software slowly became apparent, as voice-call quality would falter from time to time, and he realized he had no way to troubleshoot the problems.

In 2002, Jacksonville overhauled its network and decided to run voice traffic over the data network, potentially “saving tons of money on the monthly phone bill.”

“We were going to put in an infrastructure to interconnect all our buildings. The need for a new telephone system arose as well, so we decided it was just easier to put voice on top of that,” Parrish says. “Plus our phone bills went from around $12,000 to $14,000 per month to around $3,000 to $4,000 per month, and that includes all of our T1 lines for [Private Rate Interfaces] and Internet access.”

Parrish says the city chose Cisco as the network and voice vendor, installing Cisco Catalyst 6509, 4006, 4507 and 3500 series switches at various locations. The infrastructure supports about 400 users with 240 IP devices and telephones, including fire and police, as well as public service departments and recreational facilities.

He says the city’s network is completely switched with routing out to the Internet, two locations connected via frame relay and 19 locations supported via Gigabit fiber connectivity, on which he installed the VoIP network. He also uses Cisco CallManager software, the call-processing component of the Cisco IP telephony products.

Parrish segregated voice traffic using QoS within his switching infrastructure and virtual LANs for bandwidth and security purposes.

“Voice management is more important because of the cost-savings expectations and the business impact of failure,” says Stephen Elliot, a senior analyst with IDC. “It is also a strategic decision; once made, there is no turning back unless you cut the project, which means IT and its planning has failed.”

Despite his efforts to keep voice separate on network segments, performance wasn’t always up to snuff. Even worse, Parrish says, when performance degraded he couldn’t find the source of a performance issue or prevent it before it affected users.

“We were fortunate enough that we put in such a robust network that we didn’t experience too many problems,” he says. “But every now and then we’d experience jitter, echo, latency and overall lag with the voice traffic, and we didn’t have anything to troubleshoot the cause of the performance issues.”

Cisco tools were too costly

Parrish used CiscoWorks for data management but couldn’t afford to buy the voice-management add-on. Having rolled out voice in 2002, he would have been ahead of the curve of large management vendors such as CA, HP and IBM getting voice capabilities into their software. According to The Yankee Group, a majority of IT managers depend on their equipment provider for voice-management capabilities.

“The added features in the Cisco product didn’t justify the cost,” he says. “And nothing out there seemed to do what we needed at the time. We wanted to be able to monitor live phone calls and troubleshoot voice problems.”

That is, not until Parrish came across a demonstration of Qovia IP Telephony Manager at a 2004 Cisco Networkers Conference. “The Qovia product was approximately 25% of the cost of the Cisco product,” he says.

Soon after Parrish installed the centralized software on a server, which acts “as the brains behind the product,” and distributed hardware probes, he says he learned about the performance of Cisco CallManager processing issues and determined mean opinion scores on call quality.

“The software can tell you if performance is degraded because of a malfunctioning switchboard, or local utilization on particular network segment is high,” he explains. “Now if I see a problem, I can go out there and tweak the QoS settings to try to meet end-user expectations.”

Better yet, he says, he could determine if some of the smaller locations his network supports could handle the voice traffic, considering their bandwidth capacity. “I am able to do more capacity planning for the edge of the network,” he adds. “I can determine the performance impact of adding phones to the network.”

Yet voice-management software isn’t entirely where he’d like to see it. “We can be proactive on the server side, but the phone side is still reactive,” he says.

That could change now that management heavyweights have taken an interest in VoIP management. For example, companies such as Entuity, InfoVista and Visual Networks (now part of Fluke Networks) are increasing their voice-management capabilities, and industry watchers expect bigger vendors to follow suit in 2006.