Throughout this test process, we noted several features missing that we feel should be included in any enterprise audit and compliance product.

First, version control for policy development must be improved across the board. For most of the products tested, the best practice is to make a copy of the policy to be edited and work from the copy. Policy development should contain a version feature, so users know what was changed and by whom, and can revert to a previous policy or at least view what the policy was three months ago, if necessary.

Another issue is the accessibility of audit logs for the compliance system. Most of the products tested appear to log the information according to company contacts, but few vendors make the full details accessible through their administrative consoles. This functionality is needed frequently by administrators who implement controls on their compliance system, because auditors need to be able to trust the reports it generates.

We were also surprised by the lack of delta and remediation reports generated by these products. If a system was out of compliance and then went into compliance, the products should tell us that. Most couldn't tell us what was changed on the system to make it compliant. This information is very useful; especially if the issue is an improperly configured build script used by the operations team.

Andress is president of ArcSec Technologies, a firm focusing on security assessments, product reviews and analysis. She can be reached at mandy@arcsec.com.

Andress is also a member of the Network World Lab Alliance, a cooperative of the premier reviewers in the network industry, each bringing to bear years of practical experience on every review. For more Lab Alliance information, including what it takes to become a member, go to www.networkworld.com/alliance.

< Previous: How we tested 6 products | Return to main: Introduction >