After Microsoft questioned the results of our January test of Test of Microsoft Exchange 2007, Forefront add-on, we found significant issues, and we tested again.
After several discussions with the Exchange team, we did indeed discover a significant issue with the test as executed. We had installed Exchange 2007 and its ForeFront security software in 32-bit Windows test environment, a test configuration blessed by the Microsoft team who completed the onsite installation. What we, and they, didn’t realize is that 32-bit versions of Forefront don’t get antispam updates from the company. Based on this “no-fault” error, we agreed to retest Forefront and Exchange using the 64-bit versions of the applications.
Our results on the second test were certainly improved, but didn’t exonerate the Forefront antispam engine completely. With Exchange 2007 properly configured in a 64-bit environment, we achieved a spam-catch rate of 80% to 91% and saw a false-positive rate in the range of 0.42% to 2.21%. In the same retest, we saw Symantec and Ironport turn in scores of 92% to 98% in spam-catch rate, with false-positive rates between 0.11% and 0.33%.
Archive of Network World tests
Subscribe to the Network Product Test Results newsletter
During this subsequent round of testing, Microsoft pointed out an important feature of Exchange 2007 called “Safelist Aggregation” as a way to help cut the very high false-positive rate for Forefront Security. With Safelist Aggregation, users have the ability to bypass antispam processing for certain users and domains (as many as 1,024 entries per Exchange user). Entries on the safe list can get there any number of ways, including contacts from the user’s address book, explicitly added users and domains, and from outgoing messages.
|
Because of the short duration of our test, we couldn’t test this feature, so we analyzed the false positives for Exchange 2007 and attempted to ascertain whether each one would have been on the safe list. Based on this analysis, the false-positive rate would theoretically have been cut by about as much as 75%, possibly taking the peak false-positive rate down to 0.55%.
Microsoft also told us that they encourage the use of third-party reputation services. We ran two scenarios, one with the Spamhaus service and one with a one, two, three punch of the Spamhaus, Spamcop and NJABL services. Adding these services significantly increased the spam-catch rate for Forefront by as much as 10%, though they also added false positives.
What we discovered is that using Forefront without safe lists will smell as bad as an Èpoisses on a hot day. If you plan to roll out Forefront Security as part of your Exchange 2007 deployment, you must plan to include Safelist Aggregation. Another key strategy will be using a third-party reputation-based service to get the spam-catch rate up closer to other industry-leading products.
Snyder is a senior partner at Opus One, a consulting firm in Tucson, Ariz. He can be reached at Joel.Snyder@opus1.com.
Snyder is also a member of the Network World Lab Alliance, a cooperative of the premier reviewers in the network industry, each bringing to bear years of practical experience on every review. For more Lab Alliance information, including what it takes to become a member, go to www.networkworld.com/alliance.
