ArcSight’s flexibility and interface helps it lead the pack of security data organizers

We test security event management tools from ArcSight, e-Security, Network Intelligence, Tenable Network Security and netForensics.

Firewalls, intrusion detection, vulnerability assessment tools – oh my! These are just a few of the devices that generate megabytes (and sometimes gigabytes) of daily logs of interest to security professionals. And that’s before you count the piles of log data generated by anti-virus applications, operating systems, Web servers, file integrity programs and routers/switches .

The data is overwhelming at best, and analyzing it accurately without assistance is impossible.

NetResults

Archive of Network World reviews

Subscribe to the Product Review newsletter

Enter security information management (SIM), security event management (SEM) and/or enterprise security management . Whatever your naming preference, the goal is the same: to make sense of the data your security infrastructure provides.

The term SEM seems to best describe the task these products perform. Devices generate alerts or logs on security events, such as blocked packets, failed logons or attempted exploits. Managing these events is the next step in the evolution of the corporate security infrastructure.

ArcSight, e-Security, netForensics, Network Intelligence and Tenable Network Security agreed to participate in this review, while Consul, GuardedNet, Intellitactics, NetIQ, Open Service and Tivoli declined.

ArcSight 2.5 wins our Blue Ribbon Award based on its ease of use, flexibility and administration interface. E-Security v4 was not far behind. Its extensibility makes it stand out, but the product is not very easy to use.

Network Intelligence’s HA Series comes in a close third. It is the only product sold as an appliance, and it is easy to set up and use. NetForensics 3.1 has a lot of potential, but the user interface, SIM Desktop, could be improved.

Tenable’s Lightning 2.0 only focuses on vulnerability assessment and intrusion-detection system (IDS ) logs. This product is an excellent investment for small organizations getting started in SEM. It is less expensive than the other, more complex products and much easier to set up.

Implementing SEM

SEM implementations require careful planning and analysis, even before you decide which product to purchase. You need to fully understand what systems you want logged, how you want those logs gathered and how many logs each system generates on average and during peak times, such as worm outbreaks.

A further consideration new to most corporate security departments is data management. Enterprise SEM products use beefy database backends – usually Oracle or Sybase. Most corporate security teams do not have a database administrator on staff, so they try to work with the corporate database team or look at hiring some help. Hand in hand with database management and maintenance is data retention policy. Data retention policies can have a large effect on your SEM implementation because they mandate some of your hardware requirements.

The products we tested all handle SEM differently. One major difference is how they are sold. Network Intelligence is the only product sold as an appliance with hardware and software included. All other products are software only, so factor in the cost of hardware purchases in your budget. If you need to purchase anywhere near the same systems provided for our testing (see How we did it ), your hardware budget will be significant.

It all depends on the number of systems you plan to monitor, the number of daily events you expect to process and how long you need to retain the data on your system for analysis. Systems that vendors provided for this review typically included a multi-CPU system with 2G to 4G bytes of RAM.

In terms of licensing, Tenable Lightning 2.0 is licensed by the number of IP addresses active on your network. Network Intelligence is licensed by events per second (EPS). NetForensics 3.1 is licensed by devices being monitored. With e-Security v.4, you purchase the console plus pay an additional fee for each device being monitored. ArcSight 2.5 is priced on a combination of consoles, monitored devices and CPU in the Manager server.

For installation, we used each company’s professional services team when available, which is highly recommended. SEM products are complex, and although you could set up the software yourself, implementation will be much easier if you use the expertise available through professional services. Tenable was the only product that we completely installed ourselves. Most products come with at least one day of professional services included in the purchase price.

Each company, with the exception of Tenable, sent us pre-configured hardware. The installation team came in to configure the device for our lab environment and set everything up so alerts and events were being sent to their system from three initial devices in our test bed – a NetScreen Technologies firewall, a Cisco VPN Concentrator and a Cisco Catalyst switch – which all logged directly to syslog. The netForensics and Network Intelligence installs were the quickest, lasting just two hours for initial setup, device configuration and a quick tutorial. ArcSight and e-Security took four and eight hours, respectively.

The installation times directly correlate with the complexity of the product. Network Intelligence is the cleanest product when it comes to setup and adding new devices to monitor, but it is also the least flexible. We included a NetScreen firewall running an older version of its operating system in our test bed. Network Intelligence and NetForensics products could not evaluate events from this NetScreen firewall because they only supported newer versions of the operating system. ArcSight and e-Security handled the older operating systems just fine because they can create custom agents and support just about any product that generates a log.

With SEM products, there is considerable discussion about agent and agentless products. The word agent conjures up thoughts of a piece of software running on monitored devices. These products blur that line a bit. E-Security uses agents, but they run on a separate agent server. ArcSight uses agents, but it also can run agentless. But if you go this route with ArcSight, you lose some of the features the agents provide, such as agent-level filters for events you don’t want logged to the central server.

A major trial of the products was adding new devices to monitor. We gathered a test bed of various firewall, IDSs, Web servers, operating systems, network infrastructure devices and security integrity products, and attempted to monitor them. Tenable did not fully participate in this test because it only supported the Nessus and Snort systems.

Each product gathers data differently, and we were constantly reconfiguring our test bed just to log to a specific product. For example, most products supported the general syslog format of the VPN Concentrator, but Network Intelligence only supported the Cisco IOS logging format.

Snort logging was also an interesting setup. We ran a basic Snort installation logging only to syslog. NetForensics had set up a syslog agent on its server listening on Port 888, so we had to reconfigure our syslog on our Snort system to communicate over that port. Network Intelligence provided the quickest setup, as it required just a regular syslog configuration. E-Security provided a Snort agent on its agent server during installation, but when we went to set it up, we couldn’t get it to work, and documentation was not available. A quick message to support provided a new agent and detailed documentation that got us up and running. ArcSight provided a Snort agent, but it only worked if you used database logging. For syslog, you needed to install the syslog pipe, but this configuration was not noted in its Snort documentation. A question to the support team quickly resolved this issue.

Overall, Network Intelligence provides the best setup for new devices, but you are limited to the products they support. ArcSight has the best agent installation process. Their agent install program looks the same across platforms, provides a full list of devices to select and includes detailed installation instructions.

ArcSight also provides the best means of supporting proprietary or unsupported logs. Its Flexagent lets you quickly parse a log file to use in filters and correlation rules. E-Security also provides this ability, but the setup is more complex and time-consuming.

We spent a good deal of time setting up devices under each SEM product framework. Systems logging to syslog was usually the easiest, but we even hit a few snags with those – what port to use, what facility to use. Windows event logging was also tricky, usually being the one device that definitely required an agent on the actual Windows server. All products supported Check Point firewall logs, but this was not easy to set up for any product. Check Point has always made its product more complicated than it needs to be, and logging setup continues this tradition.

Because security analysts will spend many hours a day looking at the SEM interface, the GUI should be intuitive, easy to use and helpful. Again, Network Intelligence provides the most intuitive, easy-to-use interface, but it is not as flexible as some of the other products. ArcSight provides the most flexible interface and is still easy to use in spite of everything you can do with it. You can configure your workspace with any number of graphs and views, all completely customizable. You can drill down to more detailed information at just about every point, and you can turn anything into a graph.

NetForensics uses a desktop GUI that looks like an X-Windows desktop. This interface was a bit clunky and resource-intensive. It also was not very intuitive, taking quite a while and a lot of documentation searching to figure out how to view events in real time. Additionally, the desktop was easy to overload and clutter with windows. The e-Security interface is complex and comprehensive, but not very intuitive or easy to use.

Several of the products also include case management functionality to track and record incidents as they are investigated. Events can be tagged and added to incidents just about anywhere in the GUI. ArcSight and e-Security stand out in this area for ease of use. NetForensics includes a collaboration area – a screen where users can type messages and have them visible to all other users – and the ability to attach any file to a case.

After getting our devices set up, we launched Nessus and Internet Security Systems’ Internet Scanner scans to trigger firewall, Snort and system events. We created various filters, correlations and alerts on each product. ArcSight provided the best method of creating filters and correlation rules, and you are only limited by your imagination. In this category, e-Security is powerful, but just not very intuitive.

At the most basic level, SEM products aggregate security logs from various devices. Taking SEM to the next level, these products add correlation, which lets you create alerts for any combination of log entries. For example, you can create an alert if you see a port scan and an attempted attack (seen through IDS logs) for your Web server if the source IP address is the same. The next step, which some products (Tenable) support and others (e-Security) are beginning to support, provide correlation between vulnerability assessment and IDS. You do not get an alert on an IDS log unless the targeted system is vulnerable to the attempted attack. This feature is beneficial because it can help reduce IDS false positives.

SEM products include a number of canned reports and the ability to create customized reports. These reports also can be scheduled to run daily, weekly, monthly, and e-mailed to you. Reports vary from high-level executive summary to detailed packet analysis. We liked ArcSight’s overall reporting system the best for flexibility and ease of use. E-Security set up a separate system to serve as its Crystal Reports server, while all other products kept reporting functionality on the manager/console server for our review.

Each vendor discussed EPS ratings with us. Because of time constraints, we only ran one test with a sustained 300 EPS level in the lab, and none of the products had any trouble. Any SEM implementation should be able to handle incident spikes and not be maxed out at normal operating levels. Blaster, Welchia, MS-SQL Slammer and whatever the next worm might be generate significantly more events than your normal operations, and you need to make sure you can handle this.

SEM requires that you clearly define your requirements before starting the evaluation process. If your environment is very straight-forward and not running any proprietary applications, Network Intelligence provides the fastest setup time and easiest implementation. If your environment is complex and you plan to use SEM to collect security data from every production device, e-Security provides the most flexibility and extensibility, but it is not easy to use and includes a steep learning curve.