E-mail filtering options

Much like the telemarketer that calls your home, using equipment that is paid for by you to intrude in your life, spam is a big problem for users and companies alike. As the volume of spam increases, the time it takes to peruse your e-mail and delete the spam also increases, not to mention the drain on your bandwidth, e-mail servers and storage.

Federal and state governments are working on various pieces of legislation to help deal with the spam problem. However, until spam legislation is put in place and probably even after it’s put into effect, spam filtering seems to be the answer. There are a variety of filters out there to choose from.

Most e-mail client software, like Microsoft Outlook, offers keyword or sender filtering. You can define rules that use heuristics to filter the e-mail. While this is a relatively easy and inexpensive solution to the spam problem, the efficiency of spam filtering goes down as the number and granularity of rules increases. Performance can also be affected.

I used to use this kind of filtering, but found that there were a lot of false positives. False positives are legitimate e-mails that are identified and filtered as spam by mistake. For example, if you filter out all mail that contains the word “sex,” it will also filter out an e-mail that contains the word Essex because “sex” is in the e-mail, even if it is embedded in “Essex.”

Another approach of filtering software is having a blacklist and a whitelist. This method lists legitimate senders and unwanted senders. While this does filter out some unwanted mail, spammers have figured out a way around this; they frequently change the sending address of their spam. So if your spam filter hasn’t seen e-mail coming from an address before, it won’t filter it. Building and keeping the lists updated are a challenge.

However, a company called Cloudmark has developed a peer-to-peer approach, where a network of users automatically report their spam to a central site, so that the collective community of users benefits from the detection of “bad” spam. I’ve used the beta version of this software, and the number of false positives is very low. However, on the weekends, when the spammers seem to change their addresses, the number of spam that gets through seems to be higher.

There’s also a statistical approach, using Bayesian statistical techniques, that seems to be another effective means of filtering spam. The accuracy of this approach can be very high and it can have a low false-positive rate.

Other approaches for spam filtering are also available. When evaluating these products, you should understand what method of filtering they are using, as well as the effectiveness of their filtering, including false positive rates.

Some of the products out there are NetIQ’s MailMarshal, MessageLabs, Postini, FrontBridge, MIMESweeper, Tumbleweed, Trend Micro, McAfee, and Symantec. This is by no means a complete list, so be sure to check out all of your options.

In addition to spam filtering software that your company can purchase and deploy in-house, there are also outsourced services that deliver spam filtering as a service. Cloudmark and MX Logic offer these services.

The bad news is that spam is a big problem, but the good news is that there’s a variety of solutions out there.