In last week's Clear Choice Test on VoIP security, in which we set hackers loose on IP telephony configurations from Cisco and Avaya, I got to play referee in this first-of-its-kind product testing. I was privy to how the hackers planned to attack and how the vendors planned to defend against them. Imagine wearing a zebra-striped shirt on Omaha Beach on D-day.

As it turned out, more Cisco security gurus showed up than we had hackers. I figured that was for psychological effect, but I was only partly right. Over the course of the testing I saw the scope and breadth of settings and interfaces involved in configuring and tuning the gamut of Cisco's security stuff. Mind-boggling is an understatement.

The Cisco VoIP system and underlying Layer 2/Layer 3 infrastructure - all Cisco stuff of course - held up so well against our hacker assaults because the security and defense pieces were implemented in every layer of the architecture. There were security pieces in the VoIP CallManager servers, in the Catalyst switches, in the IOS-based routers, in the intrusion-detection system and in the multiple PIX firewalls. That amounts to a half-dozen radically different platforms, each with its own management interface. Watching the Cisco team (which totals an estimated $1 million in combined annual salaries) adjust and configure all its security stuff, I understood why so many of them had shown up.

If IP telephony is going to prevail, there will have to be some better way for normal users to set up and adjust all of the pertinent pieces needed to make their VoIP networks secure. On the Avaya front, there were fewer security pieces to configure. That's the good news. But the overall security effectiveness of the Avaya solution? That's the bad news.

Avaya actually touts that it is switch-agnostic. That means it will do its best, security-wise, running the Avaya IP telephony package over whatever network infrastructure the customer prefers. We tested its VoIP products running over Avaya Layer 2 switches, and then over Extreme Summit and Alpine systems. Avaya had no more than three engineers on-site during the testing. Cisco showed the world that building a secure VoIP network is possible. But it has a long way to go to convince the world that its customers can do it themselves, affordably and effectively.