How we did it

Because the products that compete in end point security focusing on policy enforcement varied wildly in implementation, we had to be creative in testing them in a way that would yield meaningful results.

To that end, we first defined our problem as the need to enforce end point security policy on computers accessing the network. We then defined the goal as not allowing systems on the network that do not adhere to a defined policy and be able to take action, either automatically or manually, to put a system in compliance before access to corporate resources is allowed.

We created a list of minimum requirements for each product. At the most basic level, each product had to be able to identify systems out of policy compliance and take action to remediate that condition. We also required centralized management and reporting capabilities.

To continue reading, register here and become an Insider. You'll get free access to premium content from CIO, Computerworld, CSO, InfoWorld, and Network World. See more Insider content or sign in.

Because the products that compete in end point security focusing on policy enforcement varied wildly in implementation, we had to be creative in testing them in a way that would yield meaningful results.

To that end, we first defined our problem as the need to enforce end point security policy on computers accessing the network. We then defined the goal as not allowing systems on the network that do not adhere to a defined policy and be able to take action, either automatically or manually, to put a system in compliance before access to corporate resources is allowed.

We created a list of minimum requirements for each product. At the most basic level, each product had to be able to identify systems out of policy compliance and take action to remediate that condition. We also required centralized management and reporting capabilities.

To create the more specific requirements, we spoke to security managers and used our own experiences as security professionals to create a wish list of policy enforcement checks and product functionality we would like to see solve the problem. We understood from the beginning of this process that we would be hard-pressed to find a product that met all of our requirements, but we were open to companies submitting products together that interoperate to meet our needs.

We broke our requirements into five main categories: policy management, setup/deployment, remediation, resiliency, and reporting/alerting.

Our policy management assessment focused on the product's ability to implement compliance checks for our defined policy. We felt that setup should be fairly simple, with strong documentation. And that agent deployment should be a simple process and overall, the product should be intuitive and easy-to-use.

Our remediation tests focuses on how an out-of-compliant system is handled. Is it quarantined and all network access blocked? Can we redirect users to links to install missing software or patches? Can the remediation action occur automatically?

Our resiliency tests take a look at how the system responds to attack (see story, page XX). Is it easy to bypass the policy and gain full access to the network? Is the server vulnerable to attack? Can you simply uninstall the client to get around the policy?

Finally, our reporting and alerting assessment looked at how administrators will be notified of out-of-compliant systems and how to track activity over time for compliancy status and remediation history. Can you generate reports for management to understand our current security posture?

We installed all server components on Windows 2003, fully patched, if supported by the product under test. If 2003 Server was not supported (in the case of the Cisco CSA management server and the TrendMicro OfficeScan management server), we used a fully patched Windows 2000 Server. These servers were installed in VMware images, configured with 1G byte RAM, and running on a Pentium III 3GHz Shuttle server.

Clients were installed on both Windows 2000 SP4 Professional and Windows XP SP2 Professional machines. These clients did not contain any additional patches beyond the service pack unless required by the client software being installed. These clients were also installed in VMware images, configured with 500M byte RAM, and running on a Pentium III, 3GHz Shuttle server.