Four special Clear Choice Tests push the limits of what products can do.
In the majority of Network World Clear Choice Tests, the bottom line is a firm recommendation of which comparable product offers the top performance, the best management wares and the most useful feature set. But in some test cases, the question of which product is best is better answered with "it depends" - it depends on the technology, it depends on your network, it depends on your users' requirements, and it depends on your budget.
In 2004, four tests fell into that category. In these tests, we examined a new breed of intrusion-prevention system (IPS); delved into the security of two VoIP configurations; completed a survey of where wireless gear stands up and falls down on security; and put a record-breaking number of anti-spam software, appliance and service products through the wringer. Because of the nature of these tests, naming absolute winners wasn't possible. But that's not to say that some of these products aren't worthy for this year's Best Products portfolio.
IPS 'In the Wild'
For our first-ever "In the Wild" IPS test last February, we spent five months testing 11 products on a live distributed network. We examined what these products could detect, how powerful and flexible they were in blocking traffic, and how their management systems supported real network topologies.
In our test, conducted in concert by Lab Alliance members David Newman, Joel Snyder and Rodney Thayer, Top Layer Networks' Attack Mitigator IPS 100 and Captus Networks' Captus IPS 4100XT got top ratings among rate-based IPS products, which block traffic based on load. With a clear focus on the problem of denial of service and distributed DoS attacks, Top Layer brings together all the tools needed to protect against the widest variety of intentional and unintentional problems. Captus' product had an astonishing level of detail and control when it comes to managing packet flows.
For content-based IPS products, which block traffic based on attack signatures and protocol anomalies, the short list comprises TippingPoint Technologies' UnityOne-200 Intrusion Prevention Appliance, Internet Security Systems' (ISS) Proventia G200 and NetScreen Technologies' (now Juniper's) NetScreen-IDP 100.
With a clear interest in core-of-the-network implementation, UnityOne offers a good base for a simple IPS. TippingPoint - recently bought by 3Com - didn't stand out with flashy features, but the architecture of the product and the capabilities it did offer make it worth watching.
NetScreen's implementation stood out for its rule-based configuration that makes tuning easy, its well-thought-out policy settings and its honeypot and high-availability features. Likewise, ISS' inclusion of full intrusion-detection system, excellent forensics tools and a nicely designed attack reaction policy shows a serious understanding of what an IPS should do.
Testing the VoIP security waters
Will security issues be the death of VoIP? That's the underlying question we posed in our industry-first test of VoIP security implementations . We had testers inside and outside of these networks trying to get to the packets on the wire.
Only Cisco and Avaya stepped up to the challenge. The objective was to disrupt phone communications. Via the data and IP phone connections, the attack team used scanning tools and other techniques to see and learn what they could of the topology. After discerning and identifying "targets," the hackers then systematically launched dozens of attacks, at times in combinations concurrently.
Cisco came out the big winner in this test, proving it could build a VoIP network - comprising its IP PBX and CallManager Software plus $80,000 worth of Layer 2/3 networking and security gear - that a sophisticated hacker assault team could not break or even noticeably disturb. The elaborate IP telephony package - with underlying Layer 2/3 infrastructure and assorted security add-ons - is the most secure that Cisco's collective network security expertise could muster, employing every defensive weapon in the Cisco arsenal. And it worked. After three days, the attack team could not find a perceptible disruption to phone communications.
Cracking the wireless security code
To get a good handle on whether or not you can deploy a secure WLAN, we assembled 23 products from 17 vendors and ran them through a battery of standards-based and brute force tests .
When the dust settled in the labs of testing partners Snyder and Thayer, we made some picks based solely on the products' security parameters. On the client side, they recommended wireless network interface cards from 3Com and Cisco because they offer a range of security options, don't have broken Wired Equivalent Privacy (WEP) implementations and offer a clear direction toward 802.11i, the most secure of the proposed wireless security mechanisms.
For access points, the decision is tougher. 3Com and SMC Networks passed all our tests, but we also feel that Cisco, HP and Proxim - which failed the WEP tests - should be on any short list because of the additional security features they offer. Even Compex, with its small office/home office access point, had the ability to switch users to different virtual LANs, which is a great security feature.
For wireless switches, we recommend the Aruba Wireless Networks, Airespace (recently bought by Cisco) and Trapeze Networks boxes, again, based on the variety of options offered. In corporations, these products will provide more security than any of the static access points tested.
Over the top anti-spam testing
What do you get when you cross 36 anti-spam products with a 10,000-message live data stream? In an environment where spam comprises as much as 75% of all e-mail, a whole lot of data.
All 36 software, appliance and service-based products underwent the first round of tests where we measured for spam catch rate (including false-positive and false-negative rates), and performance and throughput conducted by Snyder. From that field, we felt any product with a greater-than-90% spam catch rate and lower-than-1% false-positive rate warranted a closer examination .
Our short list included: services from Postini, Advascan and Mycom; appliances from BorderWare, CipherTrust, Barracuda and Messaging Architects; software packages tested on Unix from Sophos, Proofpoint and Cloudmark; and software bundles tested on Windows from Symantec and MailFrontier.
As Snyder noted, "It's not a question of better or worse. It's more a question of what solves your problem best."