When a new security vulnerability turns up, you need to know when it was released, which products it affects, what it possibly could do to your network and how you can preemptively address it.

Now multiply that by 889 - the number of vulnerability alerts issues in 2004 - and you've got a serious pile of data that needs to be sorted and analyzed by knowledgeable security staff that you may, or may not, have at your fingertips.

In our test of vulnerability alerting services from Co-Logic Security, Cybertrust, PivX Solutions, SecurityMob, SecurityGlobal.net, Secunia and Symantec, we assessed how well each can provide timely, relevant, accurate and useful information directly to your in-box.

Overall we found these services to be very useful because they help filter the myriad alerts. With these services in place, IT security personnel can then focus on remediation plans rather than combing through mailing lists and vendor sites culling for newly issued alerts.

When the data analysis dust settled in our own tests, Symantec's DeepSight Alert Service - which the company picked up in its acquisition of SecurityFocus last year - came out as our Clear Choice winner. It provides the most delivery options, very detailed reports, detailed alert category configuration and quick response times.

Cybertrust's Alert Manager - a product coming from the merger of TruSecure and Betrusted, completed in November - was a close second, falling behind on alert delivery options, but standing very strong on alert information and format.

We'd also put the services from Secunia, PivX and SecurityGlobal on a short list. Secunia's Security Manager is one of the least expensive services in this test and is very consistent in its alert delivery. The interface driving PivX's ThreatFocus Diligence is intuitive and therefore makes the service very easy to use. And SecurityGlobal's SecurityTracker is very effectively focused in its one mission: getting alerts pushed out to its customers.

The services generally differentiate themselves in terms of how each collects and interprets vulnerability information. Some services, such as Co-Logic Security's E-Secure-IT and SecurityMob, act solely as data aggregators. They compile information gathered from hundreds of sources - mailing lists, vendor announcements, Web site scouring and direct messages from vulnerability researchers - and then send pertinent alerts your way. E-Secure-IT also provides a collection of security-focused articles from hundreds of different sources. As a security professional, it is a big bonus to have all this information collated in one place.

The other services tested focus on providing information and analysis for vulnerabilities to help users understand how quickly they need to react to a new announcement. Symantec and Cybertrust focus on broad coverage and perform their own analysis and write-ups. And both perform their own testing on reported vulnerabilities, provide detailed threat analysis and remediation options, and provide links to associated vendor patches. Security Manager, SecurityTracker and ThreatFocus Diligence offer their own descriptions but generally are not as in-depth as Symantec and Cybertrust.

These alerting services also vary in how high they raise the red flag on any particular alert. There seems to be a general tendency to set the risk and severity ratings to accommodate worst-case scenarios, which sometimes can be misleading. We advise, based on that observation, that users be wary about basing patch deployment decisions solely on these services. There is always some internal analysis any organization must perform to see how fast it needs to react to a given alert. For example, an alert announcing an Apache server vulnerability might not be critical to an organization that only runs Apache on its intranet Web server. But a company running Apache on all its Internet-facing Web servers might need to react immediately.

SecurityTracker doesn't provide any risk information, leaving the rating decisions solely in the hands of the organization. But the company says it is adding functionality to the service in the near future that will help organizations set their own internal risk ratings. DeepSight Alert Service stands at the opposite end of this spectrum, providing analysis and a 1-to-10 rating in the areas of severity, impact, ease of exploit and credibility. The remaining products fall somewhere in the middle of these two extremes.

For this Clear Choice Test, we focused on four main performance areas: alert information, or the details provided within an alert distributed by the service; alert delivery, which includes timing, methods and format; alert management, focusing on the GUI used to access and configure the service, general ease of use, and documentation; and alert coverage, which looks at how the service identifies new security issues and how many products it covers in its research.