IPS usability is a mixed bag

How usable are network intrusion-prevention systems?

The most important feature of an IPS is whether or not it does the job you bought it for. That said, it also needs to be usable in the sense that it can support the network manager in the day-to-day tasks that go hand in hand with using an IPS in an enterprise setting. After shaking out the IPS products for performance, we took them back into the test lab to look at them from another angle entirely: usability.

Downsides of IPS coverage

Reviews of 6 products: Ambiron | Demarc | Fortinet | NFR | TippingPoint | Top Layer

How we tested IPS systems

Archive of Network World tests

Subscribe to the Network Product Test Results newsletter


On the dark side of our scoring, though, were the management wares provided with Demarc’s Sentarus, Fortinet’s FortiGate 3600 and Ambiron TrustWave’s (formerly Lucid Security) ipAngel. While each of these three has its administrative bright spots, all three need substantial work before they can handle the tasks we think concern an IPS manager.

However, it’s important to keep in mind that both Demarc and Fortinet offer multifunction products (the Fortinet box is a UTM device, while the Demarc product is a combination of host and network-based IPS), where network-based IPS is only a piece of a bigger offering. Network managers may be willing to trade off IPS usability and features in exchange for the other security functions shipped with these products.

We set up a VPN between our test labs in California and Arizona to see how these products would work in an enterprise WAN environment. Because all of the performance testing was done at Network Test’s lab in California, we did all our usability testing from Opus One’s lab in Arizona. Where vendors provided stand-alone management tools for their products, we used those tools. Otherwise, we used whatever native tool was built into the IPS itself.

To evaluate products for usability, we set out five major task areas all geared toward operating an IPS in an enterprise-class network. We started looking at configuration and alerting capabilities, because these are the first tasks any security manager will do and subsequently be revisited repeatedly as networks, systems and security policies change.

Next, we looked at the dashboard features for each product, to see how easy it is to get an update on the security status of your network. We looked carefully at forensics features. Although an IPS is not a replacement for an IDS, most IPS products have a fairly strong set of IDS forensics and analysis features in them, and we wanted to see how well this was put together. Finally, we looked at the reporting features of each product.

Task 1: configuration

IPS configuration can be an ugly and complex job. To minimize false positives and maximize coverage, the network manager must push a considerable amount of network topology and configuration information into the IPS. For example, the IPS configuration may need to include signatures specific to the ports being used (such as 80, or 8080), Web server (such as IIS or Apache), application languages (such as PHP) and third-party applications (such as calendaring or discussion forum software). Without this level of specificity, the IPS manager risks either missing attacks (for example, by not looking at traffic on all the right ports or not having the right signatures turned on) or having false positives (for example, by having signatures on that don’t apply).

Most IPS products take an IDS-like view of things and encourage the network manager to turn on all signatures on the common ports, such as Web traffic on port 80. That makes configuration easy, but it guarantees that the IPS is going to perform in a suboptimal way. System throughput and latency will be affected by having extra signatures looking at traffic, and any unusual network configuration will open up a hole for attackers that the IPS may not be looking for.

Configuration also goes beyond turning signatures on and off. Many of the IPS products we looked at, especially those from TippingPoint and Top Layer, have extensive rate-base IPS features that need to be tuned to be effective. In addition, an IPS could have different actions (other than “drop the packet”) when a signature is triggered. For example, some network managers want to drop all future traffic from an IP address once the IPS has alerted on any attack.

Our testing showed that TippingPoint configuration interface offers an excellent model for defining configurations, based on rule sets. TippingPoint’s Security Management System's ability to give per-rule granularity, to version-control sets of rules and to apply globally defined sets of actions and alerts to rules and groups of rules really makes it easy to define and control your configuration. We found some things to pick at, such as the lack of rule-grouping tools, but this product demonstrates that TippingPoint really knows what network managers need in an IPS configuration tool.

If rate-based IPS is a priority, then the Top Layer IPS 5500’s configuration hits the top spot. While the TippingPoint 5000E does have some rate-based IPS features, no one has Top Layer’s precision and control when it comes to configuring rate-based rules. Top Layer’s signature configuration is not as strong, reflecting the roots of this product in the rate-based IPS world.

This brings us to NFR’s Sentivist combination, a product set that has to be faulted by a most network managers for being too granular and having too many options. Fundamentally, Sentivist is really an IDS, not an IPS, reflecting its roots as a capture-and-analysis tool for network traffic, and this comes through in its configuration options and management style. It’s easy to describe who would prefer the NFR approach: someone who really wants to control exactly what signatures will trigger an event on their network, to know how the signatures work (since NFR includes source code for most parts of the signatures), and to have the ability to tweak and adjust every aspect of the signature.

NFR’s Sentivist Management Platform is a tool for someone with a doctorate in network security who has the time and interest to spend hours a day getting everything perfect. For example, while most of the IPS systems we tested may have a dozen or so settings that are used to tune signatures, NFR has hundreds of variables to tune and tweak its system. The network manager who wants a dual-purpose IDS and IPS, and who intends to really use the tool, should put the Sentivist at the top of their short list.

We had mixed experiences with Ambiron TrustWave’s ipAngel product in the area of configuration. The ipAngel management code we received was a late beta version, and we ran into a number of bugs that hampered our usability testing of its interface. What ipAngel does that is different from other IPS products is closely tie vulnerability information gathered by the ipAngel device to the actual signatures in use on the IPS. Thus, ipAngel runs a scanner against your network and decides what systems are there, what ports they are using and what applications are running on those ports. Then, it can automatically turn on the rules to tightly tune the IPS just for those systems.

All of that sounds great, except that the code we got didn’t work that way. The scanner wouldn’t launch; the rules couldn’t be edited; the resulting configuration was locked in and couldn’t be changed. Network managers who want signatures on even if the host isn’t there (for example, to handle lab environments or systems undergoing rapid reconfiguration) had no way to do what they wanted — in the version that we looked at. Conceptually, ipAngel looks like it has some great ideas in it, but didn’t quite get everything finished in time for us to test. We were also hampered in our testing, because the ipAngel had no documentation or online help.

Network managers who are primarily interested in a network-based IPS will be disappointed in the configurations tools in supplied with Demarc's Sentarus and Fortinet’s FortiGate 3600. Both have only the most rudimentary IPS configurations. For example, in the Sentarus product, changing an IDS rule to an IPS rule requires that you bring up each rule, one at a time, and edit it by changing the word “alert” to “block.”

FortiGate’s configuration has its own restrictions. For example, rate-based IPS settings are defined globally, which means that you can’t have different thresholds for the Web server handling 5,000 pages per second from the one handling five transactions per minute. We also found the documentation on signatures to be so incomplete and misleading that we were unable to understand what many of the signatures in their database actually do, or what vulnerability or exploit they are testing for.

Task 2: alerting and actions

An IPS has to be able to drop suspect traffic, but the difference between a basic and an enterprise-class IPS lies in the capability to do more than drop packets. This is where alerting capabilities and notification controls come in. While these abilities are always present in an IDS, we believe that high-end IPS systems should also have the ability to deliver alerts and take actions during ongoing attacks.

As an aside, it is important to note that in the world of IPS and IDS, the term “alert” has several meanings and is sometimes used interchangeably, albeit confusingly, with the term “event.” An “event” occurs when a signature is tripped, which might cause an action such as dropping a packet or sending a Syslog message. In this article, we’re using the term “alert” to mean a higher-level message, such as an e-mail or page, coming out of the IPS-management system in response to one or more events. Typically, every event will be logged unless you specifically suppress logs for uninteresting events, but higher-level alerts would be generated only for more critical events.

The job of alerting in a large IPS deployment may not fall upon the IPS, though. Network managers who have Security Information Manager or Security Event Manager (SIM/SEM) tools may want the IPS to pass event information onto their SIM/SEM and let it make alerting decisions at a more global level, perhaps based on correlations across devices or even different types of security devices.

Network managers who care about alerting in the IPS itself will want to start with TippingPoint’s 5000E, although Demarc’s Sentarus and Top Layer’s IPS 5500 came in right behind TippingPoint in our assessment. In the TippingPoint 5000E, the network manager defines “action sets” that can be associated with signatures. An action set can be extensively customized. Actions can include dropping traffic, blacklisting attackers, rate limiting, connection resets, and a variety of alerting and logging actions, including capturing packet traces.

What impressed us most about TippingPoint’s alerting capabilities is the obvious ease of applying these actions to different signatures and signature groups. Rather than digging around through different parts of the GUI to figure out where to apply these actions or having no granularity to give different actions to different signatures, TippingPoint really hit the nail on the head with this design, combining ease of use with power in a way that makes it easy to get highly granular alerting when you want it.

Demarc’s Sentarus has another view of alerting that proves some great thinking. Rather than apply alerts to signatures, you define filters that watch the stream of attack events. The powerful feature of Sentarus that makes it act more like a SIM/SEM is the ability to filter any field in the attack. For example, you could place a alert on any attack event that is aimed to or from a particular IP address.

Unfortunately, the Sentarus developers left out any easy way to do the simpler task of applying and managing alerts attached to a signature or signature group. Demarc told us that the company plans to add more signature grouping facilities in a future release of the product, which would help with this.

With Top Layer’s IPS 5500 alerting capabilities, again the rate-based IPS features trumped signature-oriented IPS features. In the rate-based portion of the IPS 5500 5500, the alerting is excellent and provides a variety of thresholds and options to let you know when something is wrong. On the signature-based part of the IPS, the capabilities are extremely limited.

You could argue that alerting is much more important when it comes to rate-based IPS than on the signature-based side of the product. After all, rate-based IPS attacks are long-term events and may require some manual intervention or mitigation, so immediate alerts are pretty important. For signatures, most alerts are really confirming that the IPS has done its job, and that might be a job better suited to reporting.

NFR has built some SIM/SEM features into Sentivist that the company refers to as “correlators.” A correlator is a filter that watches attacks and generates events only when some condition occurs, such as a cluster of five attacks from the same IP address. The most interesting correlator we found was one that takes the results from Tenable Network’s Nessus vulnerability scanner and uses this information to generate an alert only when an attack matches a known vulnerability — behavior similar to Tenable’s Security Console (see test of Tenable product suite).

1 2 Page 1
Page 1 of 2
IT Salary Survey: The results are in