Eight network scanning tools offer beefed-up management and remediation
Eight network scanning tools offer beefed-up management and remediation.
A vulnerability rated as a low risk this morning could turn into your worst nightmare tonight. To meet the ever-increasing speed with which exploits are written and propagated, traditional network-based vulnerability scanners have morphed into more full-scale vulnerability management products.
In our latest Clear Choice Test of eight products - assessing their accuracy in pinpointing holes in the network and their usefulness in addressing those vulnerabilities - we found vulnerability identification success rates are still low across the board and the scans can wreak havoc on wireless access points. They also can do damage to some printers, and can suck up network bandwidth and CPU utilization on target machines (see How we did it).
Getting a second scanning opinion
Vulnerability remediation and tracking are the major management features added to these products since our last test, providing mechanisms to assign and alert administrators to new vulnerabilities. These additions range from providing vulnerability remediation information to offering full-blown ticketing systems that automatically verify if an issue has been fixed.
Business analysis features have been included in many products. With this functionality, assets can be given values - in terms of cash or business-critical value. How vulnerabilities potentially could affect business and give management a more accurate picture of the company's overall security posture can be correlated. A critical vulnerability on the core, Internet-facing system that generates revenue should be treated differently than a critical vulnerability on a system inside a test network that's isolated from the rest of the company, for example.
The companies that provided products and/or services for this test are Lockdown Networks, nCircle Network Security, PredatorWatch, Qualys, StillSecure, Tenable Network Security, TraceSecurity and Visionael. EEye Digital Security, Internet Security Systems, Foundstone, NetIQ, Bindview and Harris declined. We also tested Citadel's Hercules (see story) and Sunbelt Software (see story), but because they offer no scanning module or management features, respectively, we could not directly compare them.
Qualys' QualysGuard is our Clear Choice winner based on its accuracy and strong management capabilities. NCircle's IP360 comes in second, only slightly trailing Qualys in vulnerability identification and general ease of use. Visionael Enterprise Security Protector and Lockdown's Auditor also rose to the top based on their developing management capabilities.
QualysGuard - one of the two vulnerability assessment services we tested - has a 1U appliance that sits on your network and lets Qualys scan your internal subnets. Setup is easy, and the quick start guide will have you scanning in no time. Because it is provided as a service, the Qualys team seamlessly adds the vulnerability checks.
Our discovery assessment focuses on how well the products find and identify systems, system software and services running on the network. Our accuracy measurement takes into account how well the product identified vulnerabilities that existed on a sample of lab systems (see "How we did it").
Qualys scored highest in our operating system identification checks and was the only product to correctly identify the wireless access point. It performed as well as any of the other products in the vulnerability accuracy tests, but still reported some false positives and false negatives. It did perform strongest among the products in identifying Windows system vulnerabilities, though.
Scan impact was low from a network perspective, but we did need to restart a Red Hat Enterprise system that became completely unresponsive after the scan.
Overall, QualysGuard is very flexible and easy to use. IT staff and/or corporate executives can be given varying levels of access to system groups and reports. Scan and report templates provide flexibility in the types of checks that are performed and how the results are viewed.
Remediation policies can be configured to automatically assign tickets in the Qualys ticketing system to defined individuals based on scan results. Qualys could improve on remediation if it added some preemptive notification mechanism to tell IT folks they have been assigned a remediation task.
In terms of providing some business analysis capabilities, Qualys lets you rank assets in terms of how critical they are to your business. A score is then provided in the summary based on your overall exposure level that can be weighted based on how critical the vulnerable asset might be.
One of the best features of QualysGuard is its mapping functionality, which provides a graphical representation of all the devices it discovers on your network. You can drill down on the map to identify the operating systems and services running on these devices, but can't see information on identified vulnerabilities from this vantage point. In addition to the mapping, we'd also like to see some sort of overview console that provides high-level information on the state of vulnerabilities on the network.
NCircle IP360 6.2
NCircle provided a central reporting server, VnE Manager, and scanning point, Device Profiler. With this tiered approach, nCircle runs in a more distributed model than some of the other products tested.
The IP360 provides the best business impact and risk-rating features, offering unparalleled levels of detail. Users can provide asset values for each host and calculate risk scores for each system based on the asset value. This value is a quantitative number, generally dollars, of the value of the asset to the company. As a consequence of this increased functionality, it is not as easy to use as some of the other products tested.
For system discovery, nCircle uses dynamic host discovery, its technique for continuously evaluating environments for new systems on the network. After running on the network for a few minutes, the system had found all the devices in the lab.
For operating system identification, nCircle joined Qualys as the only products to correctly identify the Cisco VPN Concentrator. But it missed a few key systems that most other products identified, including the FreeBSD 5.2 server and the Quantum Snap Server.
For vulnerability identification, nCircle consistently reported the smallest number of vulnerabilities, minimizing false positives, but potentially introducing some false negatives as well.
While nCircle's scan results might appear to include false negatives, following the remediation guidelines for identified vulnerabilities will address the known vulnerabilities in the system.
NCircle accrued the lowest network and system impact, with no identified issues or spikes in network traffic or CPU utilization.
One unique feature of the IP360 is its continuous scanning mode, which provides non-intrusive, back-to-back scans of the whole network or of only select segments.
This is ideal for critical systems or networks that need to be monitored at all times. NCircle provides a classic scanning model of scheduling scans, grouping systems and providing detailed user access.
NCircle takes a different approach in providing vulnerability remediation information. For the sample of vulnerabilities we reviewed, nCircle provided links to patched versions or specific patches for a variety of operating systems. In a few instances, the vulnerability remediation information did not match the specific vulnerability identified, although following the recommended course of action would in most cases have fixed the vulnerability because one patch would fix several issues.
Visionael Enterprise Security Protector
Visionael uses Nessus as its underlying scanning engine and focuses on providing some of the best vulnerability management functionality, such as a customizable portal for viewing security trending information.
Installing Visionael on Red Hat Enterprise worked well, although we'd like to see Visionael better secure the assessment server by default rather than leaving that up to the systems administrator.
Upon initial logon, Visionael provides the best portal functionality, allowing customization for each user and quick views of identified vulnerabilities, current risk level, trending and trouble ticket status.
There were a few issues in terms of system identification for the hosts on the lab network, namely system identification was not happening as we configured it. Working with support, we enabled the detailed operating system checks and reduced the concurrent threads from 200 to 20. With these changes in place, we got operating systems identification results, but they were not as detailed as we would like to see. For example, all Windows systems, regardless of version, reported back as "Windows."
For network and system impact, Visionael is quite loud. The scan locked up the wireless access point, bluescreened a Windows XP system and consumed 30% of the CPU on the monitored target system.
Viewing individual scan results provides an overview of identified vulnerabilities, with a breakout summary of the SANS Top 20, which is unique to this product. We would like to be able to drill down into the report directly from the vulnerability numbers reported in this overview screen.
The reporting module provides a wizard to create custom reports. But the customization options are so abundant that they are almost overwhelming.
The ticketing system is very strong, although tickets only can be auto-assigned for SANS20 or high-level vulnerabilities, which is fine if you prefer to do more detailed analysis on the other levels of vulnerabilities before tasking them out.
Visionael can auto-remediate identified vulnerabilities, but this functionality was not enabled in the license we received for testing.
For business analysis, Visionael provides strong trending information, executive reports and business rank, based on assigning systems one of four levels depending on how critical it is to your business.
Lockdown Auditor 3.0
Auditor provides the most intuitive management features, but lags a bit with its scanning engine.
Lockdown's 1U scanning appliance is the most intrusive, utilizing 40M bytes of network bandwidth and on average 40% CPU on the target system. Administrators do not have any options to change scan configuration settings.
It performed fairly well in terms of operating system identification, missing only two devices and not clearly distinguishing a few Windows versions. In terms of accuracy, Lockdown Auditor hit and/or missed to the same degree as most competitors.
Scan reporting is strong, providing a summary of identified vulnerabilities. We liked the job queue functionality, which shows the percent completion of each system being analyzed in the current scan.
In terms of management, Lockdown's user interface is the best of the products tested, combining graphics and a workflow very effectively. While it does not contain a specific portal, the initial logon window defaults to the report section. This provides an online version of the Executive Summary, which contains an overview of scan results and trending information.
For business impact analysis, the system provides a rating number based on scan results. You can assign critical values to specific systems, which then will be weighted more heavily when calculating the overall rating.
Lockdown's vulnerability notification capability was excellent and lets administrators define policies that trigger alerts. You can configure a policy that sends a page or SNMP trap and opens a ticket if a specific port was opened on a system.
Other unique lockdown features are the ability to encrypt e-mail that has account information using gpg and the ability to authenticate users against a corporate Lightweight Directory Access Protocol directory.
For remediation, it provides an excellent breakdown of problem, solution and resolution for identified vulnerabilities, including Common Vulnerabilities and Exposures numbers and links to related security advisories or remediation steps.
StillSecure VAM 4.0
StillSecure's VAM was a solid performer in both scanning and management. Setting up VAM on the vendor-supplied server was simple. The software automatically installed on the system from a CD when it booted up.
StillSecure's user interface is not intuitive and is difficult to navigate. The screen is often cluttered, making it difficult to identify specific information or tasks.
However, StillSecure performed fairly well in scan tests. In terms of operating system identification, it only missed a few of the network devices, such as the NetScreen-100 and the Cisco VPN Concentrator. It incorrectly identified the wireless access point as an ATM switch.
It performed well on scan impact analysis, providing no noticeable issues.
The scan report provides a generic list of vulnerability titles that you can drill down into for more details, although report navigation is a bit cumbersome.
VAM includes a robust ticketing system for tracking vulnerabilities, but it doesn't provide any business impact analysis functionality.
Reporting functionality in StillSecure is functional and offers some trending and executive report facilities. But it doesn't provide all the flexibility in other products.
Tenable Lightning 2.5, NeVo 2.0 and Nessus 2.0
The primary author of Nessus founded Tenable, so it's no surprise that Tenable's suite of products taps deeply into the Nessus base code to yield some unique features, such as Unix authentication for local vulnerability checks.
In addition to the Nessus active scanning engine, Tenable's Lightning product is the management console, and its NeVo product is the passive vulnerability scanner.
The Lightning/Nessus combination provides a very robust vulnerability search mechanism with the ability to search databases of identified vulnerabilities on almost any criteria. However, it doesn't include any mechanisms to control trouble ticketing or remediation functions or offer any business impact analysis.
In scanning tests, Tenable performed fairly well. In terms of operating system identification, it missed some of the network devices but performed the best of the Nessus-based products on scan accuracy. The overall scan impact was minimal.
The reporting module inside the console automatically generates a few reports, such as 30-, 60- and 90-day vulnerability details. You also can create custom reports from a selection of filter criteria.
One area that could be improved is in its vulnerability plug-in descriptions, which is what Nessus uses for its vulnerability checks. When trying to view the checks Nessus scans run, the drop-down box of signatures lists them by a nondescript ID, such as "CSCdp58462." These aren't very useful when trying to figure out what check is being performed.
Overall, Tenable has a very strong foundation, and we would like its vulnerability management functionality improved.
TraceAudit is delivered as a service but also includes an ISO - a file that contains a complete image of a disk - and a hardened version of Red Hat that installs on a network system and provides TraceSecurity access to internal systems for scanning.
The results of your scans are sent over an encrypted channel to TraceSecurity's servers, with the results available from the Web-based management interface. The scan results are not encrypted when stored on TraceSecurity's servers, though.
Reports or general scan results do not include operating system information. Accuracy of scan results was in line with the rest of the products.
Scan impact was fairly low on the network, but it did cause a core dump on the HP print server.
The user interface is not intuitive from the start, but it becomes more usable once you understand the workflow. One oddity after the initial scan was that you couldn't view the results until you associate the systems to a group. If you launch a scan, the results should be automatically viewable through a default group or similar architecture.
TraceAudit doesn't have a formal ticketing system, but the company recommends customers use the system grouping functionality to manage vulnerabilities. While this might work for some organizations, it doesn't provide the full accountability and tracking we prefer in an enterprise vulnerability management product. It also doesn't include an overview portal/summary for reporting or the ability to filter out SANS 20 results. This product also doesn't offer any business impact analysis.
PredatorWatch Auditor 128 2.2
Predator Watch was a bit above average in its scanning features but really needs to bulk up its management capabilities to compete with this lot.
PredatorWatch runs on a small, square appliance that easily could fit on a desk. The software is available on a 1U appliance.
In our scanning tests, PredatorWatch was on par with several other products in that it missed some network devices and didn't differentiate between some versions of Windows. Scans locked up the wireless access point.
One function we couldn't get to work was launching an immediate scan. We would select a scan to start, but it never began. Scans would start fine when scheduled.
The management GUI is slow and unresponsive. We had requests processing for 30 seconds or more before the page hit the screen. The GUI is also difficult to navigate and not intuitive.
PredatorWatch doesn't provide a ticketing system, business impact analysis and user management functionality. It also only provides three reports - executive, management and administration - for each scan results set. Limited trending information is included from the previous scan, but we would like to see more custom report options and trending information.
PredatorWatch offers a unique feature with its compliance reports. Based on identified vulnerabilities, administrators can run reports to help identify weaknesses in Sarbanes-Oxley, Health Insurance Portability and Accountability Act and ISO 17799 compliance.
It's good news that vulnerability assessment tools are embracing vulnerability management functionality. Ticketing systems, business impact analysis, console dashboards and custom reporting options quickly are becoming standard features.
However, the number of false positive and false negative scan results still points out that vendors need to continue to refine their scanning engines. Users will benefit from strong management tools only if vendors make sure the vulnerabilities bubbling up to the management tools are complete, accurate and do not affect target system functionality.
Learn more about this topic
Andress is president of ArcSec Technologies, a security company focusing on product reviews and analysis. She can be reached at firstname.lastname@example.org.
Andress is also a member of the Network World Lab Alliance, a cooperative of the premier reviewers in the network industry, each bringing to bear years of practical experience on every review. For more Lab Alliance information, including what it takes to become a member, go to www.nwfusion.com/alliance.
I used Google's Nexus 9 tablet as my primary device and found that it was one of the best in its size...
The Raspberry Pi 2 Model B was recently released and it’s quite a step up from its predecessors. Here's...
You can use the CuBox-i4Pro as an Android machine, a general purpose Linux host with or without...
Sponsored by AT&T
Sponsored by Brocade
Google executive explains how the company attempts to avoid downtime using an innovative method.
The new rules end a 20-year bipartisan agreement on light regulation of the Internet, Republicans say ...
Tim Cook will address GW graduates on May 17.