How we did it

We set up the test network as a flat network, with all devices working off one Cisco Catalyst 3500 switch. This network had 31 devices running on it, varying from a wireless access point, NetScreen firewall, Cisco VPN Concentrator, HP Print Server, Snap Server, FreeBSD 5.2 server, and various flavors of Windows and Red Hat Linux on various-sized servers and workstations.

We installed all products under test according to vendor specifications and on vendor-supplied hardware where applicable.

To begin testing, we configured a "Lab" group in each product. We then performed device discovery, including operating system discovery options, where appropriate, to make sure the product could find everything on the network.

To continue reading, register here and become an Insider. You'll get free access to premium content from CIO, Computerworld, CSO, InfoWorld, and Network World. See more Insider content or sign in.

We set up the test network as a flat network, with all devices working off one Cisco Catalyst 3500 switch. This network had 31 devices running on it, varying from a wireless access point, NetScreen firewall, Cisco VPN Concentrator, HP Print Server, Snap Server, FreeBSD 5.2 server, and various flavors of Windows and Red Hat Linux on various-sized servers and workstations.

We installed all products under test according to vendor specifications and on vendor-supplied hardware where applicable.

To begin testing, we configured a "Lab" group in each product. We then performed device discovery, including operating system discovery options, where appropriate, to make sure the product could find everything on the network.

After discovery, we ran a full "safe" scan from each product, enabling local authentication checks when offered. We watched any status information the product provided during the scan and reviewed the results as soon as the scan completed. We also checked all servers being scanned for functionality to see if the scan had affected performance. We monitored a Red Hat Enterprise server with SNMP and Multi Router Traffic Grapher during each scan to graph CPU utilization, network bandwidth and TCP connection loads.

We then added a new server to the network, created a new user in the product, defined a SANS 20 scan template, defined asset classification and set up automatic ticket assignment with notification. Once these tasked were completed, we launched a second scan based on the new settings and policies.

To analyze the scan results, we compiled a chart of identified operating systems for 20 devices on the lab network, including NetScreen-100 firewall; Quantum Snap Server; HP Print Server; Belkin's KVM over IP appliance; Hawking Print Server; Avaya/Orinoco Wireless Access Point; FreeBSD 5.2; Solaris 8; Cisco Catalyst 3500 switch; Cisco VPN Concentrator; and several systems running various versions of Windows and Red Hat at different patch levels.

For vulnerability identification testing, we compiled the vulnerability results across all products for four systems running default installation of the following operating systems - Windows XP Professional, Windows 2003 Enterprise Server, Red Hat Fedora Core 2 and Red Hat 6.2.

To test reporting, we attempted to generate four reports - a SANS 20 report, a critical vulnerability report, a new systems report and an executive summary. We also attempted to export a report into the available format, such as PDF, and review any summary information provided by a dashboard or console in the product.

Read more about security in Network World's Security section.