Equipped to play

Vendors submit varying levels of hardware, software to meet test objectives.

We laid out our network requirements for our 60-day test of network intrusion-detection system products and let the vendors submit all the appropriate piece parts to fit the bill.

NFR Security provided two of its preconfigured appliance sensors, one for each site of our remote sites and a Central Management Server (CMS) for our network operations center. Each sensor fed alerts to the CMS system, which we examined and managed with the Windows-based Administrative Interface client.

NFR boots its sensors off of a CD-ROM, which doesn’t guarantee that they can’t be broken into, but certainly makes the job a lot harder. NFR recently released a new version of this product but it was too late in our test cycle to include in this review.

Intrusion provided two sensors and a management system, its SecureNet Provider. While still a product that needs some work, its slimmed-down management is an improvement over last year’s submission. On the client side, there are three pieces needed to manage sensors and rules, and conduct analysis. But at least they all run on the same system. To manage SecureNet Provider, we used tools that Intrusion pre-loaded on a management client. This installation was important, because Intrusion’s client caches event information in a local database to increase performance, and using the client isn’t as simple as just dropping it onto a Windows box.

Although the new architecture was welcome, it also was clearly hot off the presses. We found careless bugs, such as IP addresses sorting in the wrong order and events being mismatched to their labels during our testing. We even managed to crash the SecureNet Provider client when we used it for forensics research.Intrusion recently upgraded it’s software, but it was released too late for our testing.

Internet Security Systems (ISS) also supplied a three-tier architecture (sensor, management server and management client). ISS sent two Proventia A201 systems, its new appliance-style sensor. On the sensor side, ISS had more than its fair share of bugs that resulted in the appliances shutting down several times during the test period. Complementing the sensors were three other rock-solid ISS products: SiteProtector, Security Fusion and Internet Scanner. ISS’ architecture is centered on SiteProtector, its tool for managing and analyzing information from an entire suite of security tools.

Internet Scanner is ISS’ vulnerability analysis tool. Fusion helps to correlate IDS alerts with vulnerabilities and operating system detection information, upgrading or downgrading alerts as they flow in.

With Barbedwire Technologies, we received two appliance-style sensors and nominated one as the central management system. Barbedwire doesn’t provide a client; driving its GUI around requires only a Web browser. Two things quickly became apparent: first, Barbedwire spent a lot of time building an elegant interface on top of Linux, and second, the systems provided were underpowered even for our small network. Once the system ran for a few weeks, it came to a near-halt because it had collected too much data. Configuration pages would take more than a minute to display, reports tens of minutes to run, and on occasion even simple things (such as “15 most recent alerts”) would just timeout, returning only error codes.

Barbedwire’s failings were especially disappointing because the company’s offering contrasted with some nice thinking on the IDS front. One complaint we had about all the other products is that getting raw data out of them was impossible: Packets go in, but they don’t come out. With Barbedwire, which is built on the open source Snort detection engine, we could see the guilty packets nicely decoded.

The anti-minimalist award went to Cisco, whose enthusiastic security group gave us three different sensors and three management systems, and offered us firewalls and VPN security gateways to further complicate the picture. With Cisco, we saw an architecture in transition. Its core IDS configuration and analysis tool kit, integrated with the popular CiscoWorks management platform, uses a Web-based client interface not dissimilar from what we looked at a year ago. The difference is in Cisco Threat Response (CTR), a product that came to Cisco through its acquisition of Psionics earlier this year. The CTR concept is Cisco’s version of ISS’s Fusion: event correlation across sensors and vulnerability analysis scanners. Cisco showed this Web-based product to us as a stand-alone analysis tool kit, but promised that the technology would be integrated into the rest of the IDS product line. Because CiscoWorks’ forensics tools are much better than CTRs, the promised melding should improve both products.