Skip Links

Analyzing the spam test results

Digging deeper into the accuracy and performance data

By , Network World
December 20, 2004 12:18 AM ET

Network World - Back to index: Spam in the Wild, The Sequel

Accuracy
Vendor False positives Vendor Spam caught
BorderWare (MS=S) 0.04% 0Spam.Net 99%
Sophos 0.04% Netriplex 99%
BorderWare 0.04% Vircom 98%
Postini 0.08% Process Software 98%
CipherTrust 0.12% Postini 97%
Symantec (MS=S) 0.16% MailFrontier (MS=S) 97%
Symantec 0.16% Messaging Architects 97%
Advascan 0.19% NoSpamToday! 97%
Proofpoint 0.20% SpamStopsHere 97%
CipherTrust (MS=S) 0.23% BlueCat 97%
MailFrontier 0.25% Intellireach (MS=S) 97%
Proofpoint (MS=S) 0.29% Advascan 96%
Barracuda 0.30% Process Software (RR) 96%
Spamfighter 0.34% Roaring Penguin 95%
Cloudmark 0.35% MailWise 95%
NetCleanse 0.46% Solid Oak 95%
NetIQ 0.55% CipherTrust (MS=S) 94%
MailFrontier (MS=S) 0.54% Proofpoint (MS=S) 94%
Process Software (RR) 0.75% Barracuda 94%
Mycom 0.89% Clearswift (MS=S) 94%
Aladdin 0.92% Symantec (MS=S) 93%
Messaging Architects 0.94% Cloudmark 93%
Sybari 1.25% Mycom 93%
Vircom 1.63% Mail by Design 93%
NoSpamToday! 2.00% Symantec 92%
Mail by Design 2.13% Proofpoint 92%
Policy Patrol 2.16% BorderWare (MS=S) 90%
SpamStopsHere 2.34% Sophos 90%
Process Software 3.15% ZixCorp 89%
Intellireach 3.32% BorderWare 88%
Roaring Penguin 3.37% CipherTrust 88%
Sublimemail 4.00% Aladdin 87%
BlueCat 4.08% Policy Patrol 87%
Clearswift 4.13% Tethernet 85%
MailWise 4.32% Spamfighter 83%
eSoft 4.91% MailFrontier 81%
Intellireach (MS=S) 5.48% NetCleanse 81%
0Spam.Net 5.52% NetIQ 79%
ZixCorp 5.90% Clearswift 78%
Clearswift (MS=S) 8.46% Intellireach 62%
Netriplex 9.12% eSoft 58%
Tethernet 14.34% Sybari 54%
Solid Oak 19.86% Sublimemail 47%
Key: Services, Appliances, Software
Click to see:



Performance
Vendor Accept rate (msgs/sec) Delivery rate (msgs/sec) Platform
Aladdin 21.0 21.0 VMware
Vircom 25.7 12.2 VMware
Spamfighter 11.2 11.2 VMware
MailFrontier 15.2 10.9 VMware
NetIQ 10.1 9.7 VMware
Clearswift 17.6 8.3 VMware
CipherTrust 50.7 7.8 VMware
Cloudmark 7.7 7.7 VMware
Solid Oak 50.7 6.9 VMware
Symantec 15.7 6.4 VMware
Roaring Penguin 6.4 6.2 VMware
Sophos 22.3 4.1 VMware
Sybari 26.7 3.4 VMware
Proofpoint 2.8 2.8 VMware
NoSpamToday! 0.5 0.5 VMware
Policy Patrol 7.0 0.5 VMware

Messaging Architects 39.9 16.3 Appliance
BorderWare 10.7 10.6 Appliance
ZixCorp 10.5 10.5 Appliance
Barracuda 90 6.7 Appliance
Tethernet 5.1 4.5 Appliance
Intellireach 4.8 4.0 Appliance
BlueCat 3.8 1.6 Appliance
Click to see:

False positives: The best scores in our test all reflect products that have gotten the science of not tagging legitimate mail as spam down to the noise level. Any false positive is a problem, and non-delivery receipts (NDR) and mailing lists caused the most problems for the anti-spam products. Many mailing lists might be unimportant, but some are critical. The same is true for NDRs. If you send a mail and it doesn’t go through, your only clue is the NDR coming back from your own mail system or, sometimes, the other end. Anti-spam packages that filter these out a little too zealously (because they assume that most NDR messages are the result of a mass-mailing worm), which we found in many of the products we tested, break that feedback loop and make mail less reliable.

In last year’s test, false-positive rates were much higher, and we said a quarantine was a critical requirement. This year, while the false-positive rate has dropped overall, we still think that most businesses using e-mail as a critical communications tool need some way to deal with false positives.

Our Commenting Policies
Latest News
rssRss Feed
View more Latest News