How we did it

How we tested the various endpoint security products.

Because the products that compete in end point security focusing on policy enforcement varied wildly in implementation, we had to be creative in testing them in a way that would yield meaningful results.

To that end, we first defined our problem as the need to enforce end point security policy on computers accessing the network. We then defined the goal as not allowing systems on the network that do not adhere to a defined policy and be able to take action, either automatically or manually, to put a system in compliance before access to corporate resources is allowed.

We created a list of minimum requirements for each product. At the most basic level, each product had to be able to identify systems out of policy compliance and take action to remediate that condition. We also required centralized management and reporting capabilities.

To create the more specific requirements, we spoke to security managers and used our own experiences as security professionals to create a wish list of policy enforcement checks and product functionality we would like to see solve the problem. We understood from the beginning of this process that we would be hard-pressed to find a product that met all of our requirements, but we were open to companies submitting products together that interoperate to meet our needs.

We broke our requirements into five main categories: policy management, setup/deployment, remediation, resiliency, and reporting/alerting.

Our policy management assessment focused on the product's ability to implement compliance checks for our defined policy. We felt that setup should be fairly simple, with strong documentation. And that agent deployment should be a simple process and overall, the product should be intuitive and easy-to-use.

Our remediation tests focuses on how an out-of-compliant system is handled. Is it quarantined and all network access blocked? Can we redirect users to links to install missing software or patches? Can the remediation action occur automatically?

Our resiliency tests take a look at how the system responds to attack (see story, page XX). Is it easy to bypass the policy and gain full access to the network? Is the server vulnerable to attack? Can you simply uninstall the client to get around the policy?

Finally, our reporting and alerting assessment looked at how administrators will be notified of out-of-compliant systems and how to track activity over time for compliancy status and remediation history. Can you generate reports for management to understand our current security posture?

We installed all server components on Windows 2003, fully patched, if supported by the product under test. If 2003 Server was not supported (in the case of the Cisco CSA management server and the TrendMicro OfficeScan management server), we used a fully patched Windows 2000 Server. These servers were installed in VMware images, configured with 1G byte RAM, and running on a Pentium III 3GHz Shuttle server.

Clients were installed on both Windows 2000 SP4 Professional and Windows XP SP2 Professional machines. These clients did not contain any additional patches beyond the service pack unless required by the client software being installed. These clients were also installed in VMware images, configured with 500M byte RAM, and running on a Pentium III, 3GHz Shuttle server.

All clients were configured with Sophos AntiVirus and eEye's Blink personal firewall for compliance tests.

The core of the network was running on a Cisco 3500 switch, configured with two virtual LANs (VLAN). The first VLAN was the primary production network. The second VLAN served as the untrusted network for appliance products that functioned inline on the network. The second VLAN also served as the remediation VLAN when required.

During setup and client deployment, we noted any difficulties we had in the process, the quality of the documentation as we learned the product, and the methods supported for distributing client software. We also took notes on the general ease-of-use of the product through all phases of testing.

Testing policy management, we setup policies in each product to identify if the Blink firewall was running and if the Sophos anti-virus software was running and up-to-date on signatures. We installed the Dloader (as identified by Sophos) Trojan/spyware program to see if it was identified and stopped/quarantined. We configured a policy to identify if MS05-014 (security patch for Internet Explorer) was installed on the system and if not, remediate as far as the product supports, such as sending a link, uploading the patch file for manual install or automatically installing the patch.

For application control, we disallowed access to bittorrent, either through application access or network ports, depending on what the product supported. We attempted to block access to USB thumb drives. This could be disabling the port or preventing file writes, depending on the product. We then looked for default support of operating system security checks, focusing on password policy and registry key controlling null sessions if available in the product.

We tested the ability to enforce policy over a remote VPN connection, using a Cisco VPN Concentrator as our gateway, and the ability to enforce policy when not connected to the corporate network.

Finally, we looked for a mechanism to create custom policy checks and attempted to create any checks that were not provided by default.

In the remediation area, we tested the ability to fully block or control access policy for out-of-compliant systems, limiting access to an internal server. We also tested browser redirection, if applicable, and any remediation deployment actions available in the product.

For reporting and alerting, we first tested the ability to generate alerts, such as e-mail or SNMP, if an out-of-compliant system came online. We then looked at reporting, starting with the ability to look at basic system logs. We then looked at the ability to generate and export reports, ending with the ability to create reports for two specific areas: remediation history (system or group) and a detailed report showing systems out-of-compliance.

To test resiliency, we used switch span ports, hubs and ethereal on Linux (Centos 4) to examine and monitor network traffic. We used OpenSSL to manually exercise any SSL connections we found. We used NMAP and NESSUS to scan the servers to identify potential areas of vulnerability. We used basic Windows file manipulation tools to perform the client removal tests, to simulate what could happen within the constraints of code executing in an exploit payload.

Back to Clear Choice Test: Endpoint security products


Copyright © 2005 IDG Communications, Inc.

The 10 most powerful companies in enterprise networking 2022