I just finished a project for which I had to use many security products, and I was struck by how much of a difference there is between the good and the merely average. In particular, three characteristics separate great products from the rest of the pack.

Great products are self-documenting. It's rare to find good documentation nowadays, so security products must be designed to operate in a documentation-poor environment. Context-sensitive help, while a pain in the butt to build, is now a basic requirement for almost any product and is always present in great products.

However, self-documenting doesn't just mean context-sensitive documentation. It also means that the product's user interface design makes it clear what the user is doing and why, and what the effect will be, without using product-specific jargon. A single extra sentence of explanation in the management interface can save an hour of searching through documentation to figure out what is really going on.

Great products have good logging and debugging output. I've never seen a product that didn't require some troubleshooting, and the key to good troubleshooting is getting the information you need out of the debugging logs. When the logs are in a single place, and the controls for searching and managing them are well put together, it's easy to uncover what's going on. When the logs are in 10 different places, in different formats, or can't even be viewed and have to be sent off to some external syslog server, debugging can be a nightmare.

Recently, I spent over an hour reading thousands of lines of logs that a not-very-great VPN product generated in a transaction that took less than a half-second to complete. On the other end of the VPN, though, was a great product that put out just eight lines of logging, including a very clear pointer to where the problem was. Writing logs can be as much of an art as reading them.

Great products are instrumented so that you can find out what you need to know. I'm continually astonished at how hard it is to find out simple things about a product, such as what's the CPU load, how much bandwidth is being consumed and what are the outstanding critical alerts.

Security products especially have gone for the dashboard idea, trying to cram all kinds of status information into attractive pie charts that look good in a demo but don't really help the administrator determine what is happening now. This is one of the more difficult attributes to see in casual use, because often you're not sure what you care about until you've used a product for a while. But one thing is for sure: If you don't see any status information, there's something wrong. I'd rather see products err on the side of too much information, as long as it's the right information.