CAMBRIDGESHIRE, ENGLAND - After pounding on six intrusion-detection systems, a network equipment test lab in England has concluded that while IDS products spot their fair share of hacker or denial-of-service attacks, there is still room for improvement.
The NSS Group methodically tested three network-based IDS products and three host-based offerings by subjecting each group to dozens of attacks and evasion techniques. The test results and the description of the lab methodology used are available this week at www.nss.co.uk.
The results are similar to those Network World found by in its recent real-world test of eight IDS offerings.
"Some products have improved significantly since the last time we tested them, [Internet Security Systems'] RealSecure being the most striking example, while some have not improved in any real way," says Bob Walder, director of the tests at NSS Group, which last December tested 16 IDS products through.
Fewer companies participated this time around, citing reasons such as not wanting to pay $7,500 to have NSS Group test their products and because of where they are in their product development cycles.
NSS Group this time sought to more closely emulate real-world conditions by generating HTTP sessions with the Caw Networks' WebAvalanche/WebReflector test gear. NSS Group initiated attacks using hacker tools and commercial test suites.
IDS sensors and management consoles were placed on an Ethernet LAN with 3Com SuperStack and Foundry Networks' FastIron switches along with Intel autosensing 10/100M bit/sec network interface cards installed in each target host for Solaris, Linux or Windows.
The network-based IDS products tested were Cisco's Secure IDS 4230, ISS's RealSecure 7.0 and the open source IDS code Snort 1.8.6, which is fast becoming part of many commercial products.
RealSecure 7.0, which shipped just a few weeks ago, held up best, but was hardly perfect, detecting 94 out of 109 attacks, missing a few Trojan and DoS attacks and some HTTP and Internet Control Messaging Protocol attacks. RealSecure 7.0 was the only IDS to detect all 40 of the evasion techniques.
Host-based IDS products tested were Entercept Security Technologies' Entercept 2.5, NFR Security's HID 2.0 and Okena's StormWatch 2.1, which all faced a different battery of tests than the network-based offerings. That was because host-based IDS products are expected to notice attempts to tamper with files, operating systems and user accounts on a server or desktop.
These products performed well, although shortcomings surfaced, such as an inability to monitor logons or inadequate alerting on changes of user rights and audit policy.
The Okena and Entercept offerings are called "intrusion-prevention" or "behavior-blocking" products because they block activity and detect it.
While managing "behavior-blocking" products can be costly and difficult, corporate customers using them say they have blocked attacks by worms such as Nimda and prevented data tampering.
RELATED LINKS
Contact Senior Editor Ellen Messmer
Other recent articles by Messmer
Technology Insider: Network-based intrusion-detection systems
Our tests show serious problems. Network World, 06/24/02.
