Network vulnerability assessment management

Eight network scanning tools offer beefed-up management and remediation.

A vulnerability rated as a low risk this morning could turn into your worst nightmare tonight. To meet the ever-increasing speed with which exploits are written and propagated, traditional network-based vulnerability scanners have morphed into more full-scale vulnerability management products.

In our latest Clear Choice Test of eight products - assessing their accuracy in pinpointing holes in the network and their usefulness in addressing those vulnerabilities - we found vulnerability identification success rates are still low across the board and the scans can wreak havoc on wireless access points. They also can do damage to some printers, and can suck up network bandwidth and CPU utilization on target machines (see How we did it).

Citadel focuses on vulnerability remediation

How we did it

Archive of Network World reviews

Subscribe to the Product Review newsletter

Vulnerability remediation and tracking are the major management features added to these products since our last test, providing mechanisms to assign and alert administrators to new vulnerabilities. These additions range from providing vulnerability remediation information to offering full-blown ticketing systems that automatically verify if an issue has been fixed.

Business analysis features have been included in many products. With this functionality, assets can be given values - in terms of cash or business-critical value. How vulnerabilities potentially could affect business and give management a more accurate picture of the company's overall security posture can be correlated. A critical vulnerability on the core, Internet-facing system that generates revenue should be treated differently than a critical vulnerability on a system inside a test network that's isolated from the rest of the company, for example.

The companies that provided products and/or services for this test are Lockdown Networks, nCircle Network Security, PredatorWatch, Qualys, StillSecure, Tenable Network Security, TraceSecurity and Visionael. EEye Digital Security, Internet Security Systems, Foundstone, NetIQ, Bindview and Harris declined. We also tested Citadel's Hercules (see story) and Sunbelt Software (see story), but because they offer no scanning module or management features, respectively, we could not directly compare them.

Qualys' QualysGuard is our Clear Choice winner based on its accuracy and strong management capabilities. NCircle's IP360 comes in second, only slightly trailing Qualys in vulnerability identification and general ease of use. Visionael Enterprise Security Protector and Lockdown's Auditor also rose to the top based on their developing management capabilities.

QualysGuard 3.3

QualysGuard - one of the two vulnerability assessment services we tested - has a 1U appliance that sits on your network and lets Qualys scan your internal subnets. Setup is easy, and the quick start guide will have you scanning in no time. Because it is provided as a service, the Qualys team seamlessly adds the vulnerability checks.

Our discovery assessment focuses on how well the products find and identify systems, system software and services running on the network. Our accuracy measurement takes into account how well the product identified vulnerabilities that existed on a sample of lab systems (see "How we did it").

Qualys scored highest in our operating system identification checks and was the only product to correctly identify the wireless access point. It performed as well as any of the other products in the vulnerability accuracy tests, but still reported some false positives and false negatives. It did perform strongest among the products in identifying Windows system vulnerabilities, though.

Scan impact was low from a network perspective, but we did need to restart a Red Hat Enterprise system that became completely unresponsive after the scan.

Overall, QualysGuard is very flexible and easy to use. IT staff and/or corporate executives can be given varying levels of access to system groups and reports. Scan and report templates provide flexibility in the types of checks that are performed and how the results are viewed.

Remediation policies can be configured to automatically assign tickets in the Qualys ticketing system to defined individuals based on scan results. Qualys could improve on remediation if it added some preemptive notification mechanism to tell IT folks they have been assigned a remediation task.

In terms of providing some business analysis capabilities, Qualys lets you rank assets in terms of how critical they are to your business. A score is then provided in the summary based on your overall exposure level that can be weighted based on how critical the vulnerable asset might be.

One of the best features of QualysGuard is its mapping functionality, which provides a graphical representation of all the devices it discovers on your network. You can drill down on the map to identify the operating systems and services running on these devices, but can't see information on identified vulnerabilities from this vantage point. In addition to the mapping, we'd also like to see some sort of overview console that provides high-level information on the state of vulnerabilities on the network.

NCircle IP360 6.2

NCircle provided a central reporting server, VnE Manager, and scanning point, Device Profiler. With this tiered approach, nCircle runs in a more distributed model than some of the other products tested.

The IP360 provides the best business impact and risk-rating features, offering unparalleled levels of detail. Users can provide asset values for each host and calculate risk scores for each system based on the asset value. This value is a quantitative number, generally dollars, of the value of the asset to the company. As a consequence of this increased functionality, it is not as easy to use as some of the other products tested.

For system discovery, nCircle uses dynamic host discovery, its technique for continuously evaluating environments for new systems on the network. After running on the network for a few minutes, the system had found all the devices in the lab.

For operating system identification, nCircle joined Qualys as the only products to correctly identify the Cisco VPN Concentrator. But it missed a few key systems that most other products identified, including the FreeBSD 5.2 server and the Quantum Snap Server.

For vulnerability identification, nCircle consistently reported the smallest number of vulnerabilities, minimizing false positives, but potentially introducing some false negatives as well.

While nCircle's scan results might appear to include false negatives, following the remediation guidelines for identified vulnerabilities will address the known vulnerabilities in the system.

NCircle accrued the lowest network and system impact, with no identified issues or spikes in network traffic or CPU utilization.

One unique feature of the IP360 is its continuous scanning mode, which provides non-intrusive, back-to-back scans of the whole network or of only select segments.

This is ideal for critical systems or networks that need to be monitored at all times. NCircle provides a classic scanning model of scheduling scans, grouping systems and providing detailed user access.

NCircle takes a different approach in providing vulnerability remediation information. For the sample of vulnerabilities we reviewed, nCircle provided links to patched versions or specific patches for a variety of operating systems. In a few instances, the vulnerability remediation information did not match the specific vulnerability identified, although following the recommended course of action would in most cases have fixed the vulnerability because one patch would fix several issues.

Visionael Enterprise Security Protector

Visionael uses Nessus as its underlying scanning engine and focuses on providing some of the best vulnerability management functionality, such as a customizable portal for viewing security trending information.

Installing Visionael on Red Hat Enterprise worked well, although we'd like to see Visionael better secure the assessment server by default rather than leaving that up to the systems administrator.

Upon initial logon, Visionael provides the best portal functionality, allowing customization for each user and quick views of identified vulnerabilities, current risk level, trending and trouble ticket status.

There were a few issues in terms of system identification for the hosts on the lab network, namely system identification was not happening as we configured it. Working with support, we enabled the detailed operating system checks and reduced the concurrent threads from 200 to 20. With these changes in place, we got operating systems identification results, but they were not as detailed as we would like to see. For example, all Windows systems, regardless of version, reported back as "Windows."

For network and system impact, Visionael is quite loud. The scan locked up the wireless access point, bluescreened a Windows XP system and consumed 30% of the CPU on the monitored target system.

Viewing individual scan results provides an overview of identified vulnerabilities, with a breakout summary of the SANS Top 20, which is unique to this product. We would like to be able to drill down into the report directly from the vulnerability numbers reported in this overview screen.

The reporting module provides a wizard to create custom reports. But the customization options are so abundant that they are almost overwhelming.

The ticketing system is very strong, although tickets only can be auto-assigned for SANS20 or high-level vulnerabilities, which is fine if you prefer to do more detailed analysis on the other levels of vulnerabilities before tasking them out.

Visionael can auto-remediate identified vulnerabilities, but this functionality was not enabled in the license we received for testing.

For business analysis, Visionael provides strong trending information, executive reports and business rank, based on assigning systems one of four levels depending on how critical it is to your business.

Lockdown Auditor 3.0

Auditor provides the most intuitive management features, but lags a bit with its scanning engine.

Lockdown's 1U scanning appliance is the most intrusive, utilizing 40M bytes of network bandwidth and on average 40% CPU on the target system. Administrators do not have any options to change scan configuration settings.

It performed fairly well in terms of operating system identification, missing only two devices and not clearly distinguishing a few Windows versions. In terms of accuracy, Lockdown Auditor hit and/or missed to the same degree as most competitors.

Scan reporting is strong, providing a summary of identified vulnerabilities. We liked the job queue functionality, which shows the percent completion of each system being analyzed in the current scan.

In terms of management, Lockdown's user interface is the best of the products tested, combining graphics and a workflow very effectively. While it does not contain a specific portal, the initial logon window defaults to the report section. This provides an online version of the Executive Summary, which contains an overview of scan results and trending information.

For business impact analysis, the system provides a rating number based on scan results. You can assign critical values to specific systems, which then will be weighted more heavily when calculating the overall rating.

Lockdown's vulnerability notification capability was excellent and lets administrators define policies that trigger alerts. You can configure a policy that sends a page or SNMP trap and opens a ticket if a specific port was opened on a system.

Other unique lockdown features are the ability to encrypt e-mail that has account information using gpg and the ability to authenticate users against a corporate Lightweight Directory Access Protocol directory.

For remediation, it provides an excellent breakdown of problem, solution and resolution for identified vulnerabilities, including Common Vulnerabilities and Exposures numbers and links to related security advisories or remediation steps.

StillSecure VAM 4.0

StillSecure's VAM was a solid performer in both scanning and management. Setting up VAM on the vendor-supplied server was simple. The software automatically installed on the system from a CD when it booted up.

StillSecure's user interface is not intuitive and is difficult to navigate. The screen is often cluttered, making it difficult to identify specific information or tasks.

However, StillSecure performed fairly well in scan tests. In terms of operating system identification, it only missed a few of the network devices, such as the NetScreen-100 and the Cisco VPN Concentrator. It incorrectly identified the wireless access point as an ATM switch.

It performed well on scan impact analysis, providing no noticeable issues.

The scan report provides a generic list of vulnerability titles that you can drill down into for more details, although report navigation is a bit cumbersome.

VAM includes a robust ticketing system for tracking vulnerabilities, but it doesn't provide any business impact analysis functionality.

Reporting functionality in StillSecure is functional and offers some trending and executive report facilities. But it doesn't provide all the flexibility in other products.