Why our numbers work - Network World

Skip Links

DNSstuff.com
Get information about your IP
IP Information
50+ On-demand DNS and network tools

Security

Videos

rssRss Feed
Get instant email notification when white papers, webcasts, executive guides are added to our library.  Stay informed and up-to-date with the latest on IT Technologies with Network World's Resource Alerts.

Additional Resources

RSS

FEATURED WHITEPAPERS

Edison Group TCO White Paper HP

Edison analysts put the management software of an HP EVA system through a series of typical day-to-day storage management tasks. The same tasks were also evaluated on similar systems from NetApp and EMC. This study demonstrates how the superior user interface and virtualization offered by the HP EVA storage system can provide organizations with the benefits of higher administrative efficiency combined with the potential ability to utilize less expensive human resources.

RSS

FEATURED REPORTS

Executive Guide: Storage Heats Up HP

Get the latest on storage technologies that allow IT professionals to better cope with new IT demands. Learn how storage technologies can help you successfully tackle e-Discover, regulatory compliance, green data center initiatives and the data explosion. Get all the details now.

Why our numbers work

By Joel Snyder , Network World , 12/20/2004
  • Social Web 
  • Email 
  • Feedback 
  • Close

You may notice our numbers are not as optimistic as the marketing literature from vendors' products. There are four reasons for this:

1. Side effects from our test bed probably shaved a few points off of each product's ability to identify spam.


Main index: Spam in the Wild, The Sequel


2. We were very strict in our definition of false positives. Because many of the false positives are mailing-list traffic of marginal use, end users often don't count them when reporting errors. Missing a few messages a month from a list that generates 10 a day doesn't bother them. This contributes to optimistic numbers that vendors report based on user experiences.

3. Because we ran our tests on more than 10,000 messages from a real-time mail stream, our results are more representative of real product response than canned or contrived tests from vendors. Even a few hours' delay in processing mail causes significant deviations in performance of some products.

4. Most vendors choose to report false-positive rates by dividing false positives by the total messages processed. No statistician would do that. Some vendors don't explain what they mean by "false-positive rate." We used statistics rigorously defined and agreed on by researchers, and it makes a dramatic difference. In our tests, computing false-positive rates the vendor way would cut the numbers in half. For a detailed look at the statistics involved, see "What makes a false positive."

Comment
Login
Forgot your account info?
Add comment
Anonymous comments subject to moderator approval. Register here for member benefits.
Have a NetworkWorld account? Log in here. Register now for a free account.
First Name
Last Name
E-mail
Zip Code
IT Buyer's Guides

View All Buyer's Guides