How we did it

How we tested the various intrusion detection systems.

We tested gigabit IDS systems by generating background traffic and running easily obtained attacks. Our test-bed network consisted of an Extreme Network’s Summit 5i, an Extreme Summit 7i and two Extreme 48 switches. We used Net Optic’s Gigabit Fiber and Copper Taps and port mirroring on the switches to connect the IDSes into the network. We verified proper working of the network taps or port-mirroring using the Agilent Network Analyzer J6800, using software Version 1.1.15.

Our background traffic consisted of a combination of Perl scripts and Ixia’s 1600T chassis running IxWeb Version 1.25. The Ixia chassis contained six cards (each card had eight ports) capable of generating real Web traffic. The Perl scripts were developed in-house and were run on Debian Linux-based machines. We used Compaq ProLiant ML350 servers for resource-intensive scripts. Background traffic consisted of HTTP, FTP, POP3, SMTP, NNTP, DNS, Telnet and SSH.

The attacks were executed from Windows 2000 and Debian Linux machines. The victim machines were un-patched Windows 2000 and Redhat 6.0 computers. The attacks we ran consisted of surveillance, remote buffer overflows, distributed and regular denial of services attacks, and trojans. All of the attacks executed are for publicly disclosed vulnerabilities and have a known method of exploitation.

Gigabit IDS performance was gauged by the percentage of attacks it detected, the accuracy of the reporting and the response time. Tests were first run at baseline, without any background traffic, then again at 970M bit/sec.

We ran all tests three times and an attack was deemed detected if the IDS caught it in two out of three instances within three minutes. We also noted whether the attack was correctly identified and the time it took to make the detection. A quick response was approximately 15 seconds or less. If the product failed to detect an attack after three minutes, we noted that as a failure. We also recorded when an IDS made a partial detection. This consisted of catching an attack, but labeling it incorrectly. (For example, we ran Exploit A, and the system reported that it detected Exploit B.) We did not count these attacks as fully detected, however.

Back to the main review: “Gibabit intrusion-detection systems”