Skip Links

Clear Choice Test: Intrusion-prevention systems
Inside this test package
View from the lab: Go inside the IPS test bed at Newman's lab
Full report of usability testing
Why no product stopped Cisco exploit

Downsides of IPS coverage

How we tested IPS systems
Archive of Network World tests | Subscribe to the Network Product Test Results newsletter

IPS performance tests show products must slow down for safety

Results indicate high performance doesn't always mean high security.

By and Network World Lab Alliance, Network World
September 11, 2006 12:05 AM ET

Network World - High-end intrusion-prevention systems (IPS) move traffic at multigigabit rates and keep exploits out of the enterprise. The problem is they might not do both at the same time.

In lab tests of top-of-the-line IPS systems from six vendors - Ambiron TrustWave (formerly Lucid Security), Demarc Threat Protection Solutions, Fortinet, NFR Security; TippingPoint, a 3Com company; and Top Layer Networks - we encountered numerous trade-offs between performance and security.


View from the lab: Go inside the IPS test bed at Newman's lab
Full report of usability testing
Why no product stopped Cisco exploit
Downsides of IPS coverage
Reviews of 6 products:AmbironDemarcFortinetNFRTippingPointTop Layer
How we tested IPS systems
Archive of Network World tests
Subscribe to the Network Product Test Results newsletter

Several devices we tested offered line-rate throughput and impressively low latency, but also leaked exploit traffic at these high rates. With other devices, we saw rates drop to zero as IPS systems struggled to fend off attacks.

In our initial round of testing, all IPS systems missed at least one variant of an exploit we expected they'd easily catch - one that causes vulnerable Cisco routers and switches to reboot. While most vendors plugged the hole by our second or third rounds of testing (and 3Com's TippingPoint 5000E spotted all but the most obscure version the first time out), we were surprised that so many vendors missed this simple, well-publicized and potentially devastating attack (see Can anyone stop this exploit?).

These issues make it difficult to pick a winner this time around (see link to NetResults graphic, below). If high performance is the most important criterion in choosing an IPS, the TippingPoint 5000E and Top Layer Networks' IPS 5500 are the clear leaders. They were the fastest boxes on the test bed, posting throughput and latency results more commonly seen in Ethernet switches than in IPS systems.

Product ipAngel-2500 Sentarus Network Security Sensor FortiGate-3600 Sentivist Smart Sensor ES1000 TippingPoint 5000E IPS 5500-1000
Vendor Ambiron TrustWave Demarc Threat Protection Solutions Fortinet NFR Security TippingPoint Top Layer Networks
Price $100,000 Sensor $37,000; Sentarus Threat Protection System management application starts at $25 per node. $30,000 Sentivist Smart Sensor ES1000, $75,000; Sentivist Management Platform, $10,000. TippingPoint 5000E, $170,000; Security Management System, $10,000. $80,000.
Pros Blocked all exploits in final tests; innovative, vulnerability-based configuration system. Blocked all exploits in final tests; vendor contributes signatures to open source Snort community; fastest to develop missing Cisco SNMP signature; well-designed dashboard gives instant status. Blocked all exploits in final tests. Blocked all exploits in final tests; very fine-grained control over traffic detection and response. Fastest performer for good (non-exploit) traffic; choice of fail-open and fail-closed modes; outstanding management interface overall. Strong performer with one or two port-pairs; good anti-denial-of-service protection features; rate-based management tools are top of the pack.
Cons Modest performance from beta hardware and drivers; initially missed Cisco SNMP exploit; weak forensics and alerting capabilites.
Relatively modest performer; searching for signatures is difficult; comprehensive forensics and analysis tools; weak IPS configuration, forensics and reporting. Lower port density than other products in this test; some software versions flooded exploit traffic (fixed in final version supplied by vendor); initially missed Cisco SNMP exploit; integration of IPS into UTM Firewall lacks features and manageability. Relatively modest performance; initially missed Cisco SNMP exploit; complexity of interface not for the casual user. Forwarded exploit traffic under heavy load; disables logging when overloaded. Forwarded some exploit traffic (possibly because of vendor misconfigura-tion); initially missed Cisco SNMP exploit; weak forensics capabilities.
One port-pair configurations
The breakdown
Top Layer Ambiron TrustWave TippingPoint Fortinet Demarc NFR
Baseline forwarding rate 10% 5 1.25 5 2.5 5 3.75
Forwarding rate under attack 15% 5 5 4.25 4 3.25 1
Baseline latency 15% 3.25 3.75 3.5 4 3.5 5
Latency under attack 15% 5 5 3.25 3.5 1.5 1
Protection from attack 25% 3 4 3 4 4 4
Usability 20% 3.5 2.8 4.1 2 2.7 3.9
TOTAL SCORE 3.94 3.75 3.72 3.38 3.28 3.21
Two port-pair configurations
The breakdown Top Layer TippingPoint Ambiron TrustWave NFR Demarc
Baseline forwarding rate 10% 5 5 1 1 2
Forwarding rate under attack 15% 4 3.75 1 1 1
Baseline latency 15% 2.75 5 2.75 4.25 3.5
Latency under attack 15% 5 2 5 1 1.5
Protection from attack 25% 3 3 4 4 4
Usability 20% 3.5 4.1 2.8 3.9 2.7
TOTAL SCORE 3.71 3.68 2.97 2.82 2.64
Four port-pair configurations
The breakdown TippingPoint Ambiron TrustWave NFR Demarc
Baseline forwarding rate 10% 2.5 1 1 1
Forwarding rate under attack 15% 2.75 1.5 1 1
Baseline latency 15% 5 4.5 4.75 2.5
Latency under attack 15% 3.25 4 1 2.5
Protection from attack 25% 3 4 4 4
Usability 20% 4.1 2.8 3.9 2.7
TOTAL SCORE 3.47 3.16 2.89 2.54
Scoring Key: 5: Exceptional; 4: Very good; 3: Average; 2: Below average; 1: Subpar or not available
Click to see: NetResults

Our Commenting Policies
Latest News
rssRss Feed
View more Latest News