Will new filters save us from spam?

BOSTON – The roughly 500 programmers, researchers, hackers and IT administrators gathered in a chilly classroom on the campus of the Massachusetts Institute of Technology (MIT) Friday aren’t just looking to slow the relentless onslaught of spam – they want to completely destroy its business model.

Their aim is to find a spam filter so effective, that spammers would receive few, if any, responses, making sending unsolicited bulk e-mail a financially prohibitive task.

“Spamming is a business, and the theft efficiency ratio is the same as stealing hubcaps,” said programmer William Yerazunis, speaking at what is thought to be the first Spam Conference ever focused on spam filters.

But the high payoff for sending spam could change if an e-mail filter like the one Yerazunis pioneered becomes widely adopted by large Internet service providers.

Yerazunis wrote a language for writing filters based on the Bayesian system which assigns statistical probabilities to whether or not an e-mail is spam. The language is called CRM114, and he wrote a filter program in CRM114 called MailFilter.

At the conference at least, MailFilter was being seen as the great white hope for battling the escalating spam problem.

In tests Yerazunis performed, MailFilter was 99.915% accurate in identifying spam.

“I’m only 99.84% accurate at identifying spam, so this is much more accurate than I am,” Yerazunis quipped.

MailFilter is still in alpha testing, however.

Still, Spam Conference organizer Paul Graham said he was extremely excited about Yerazunis’ solution.

“Bill’s filter looks like the most promising,” Graham said.

Graham himself is a big proponent of filters based on the Bayesian system and he has written his own research report on the subject called “A Plan for Spam.”

His paper, released last August and posted online, has generated a lot of discussion within the spam-fighting community.

And Graham has written his own filter based on the Bayes system as well.

“I believe in filters because I personally do not have a spam problem,” he said.

Graham added that the idea that filters alone could thwart spam did not get serious discussion until about a year ago. However, both Graham and Yerazunis believe that if there is widespread adoption of filters that are accurate enough to make spamming economically prohibitive, the problem will cease without the need for legislation or other measures.

According to Yerazunis, spam filters need to be at least 99.5% accurate to push the cost of sending bulk unsolicited e-mail to about the same as it is to send direct snail mail, making it a far less attractive method for sending solicitations.

The problem, of course, is getting large ISPs like Yahoo, America Online and Microsoft to adopt the filters. As it stands now, each ISP is taking its own approach.

Still, representatives from all three companies registered for the conference and showed interest in hearing what new ideas were being batted around.

When asked whether he was planning another Spam Conference, given the success of this one, Graham said, no.

“Hopefully we will solve this problem, and we won’t need another conference,” he said. “I don’t want to be working on the spam problem 10 years from now!”

The Spam Conference at MIT runs through 6 p.m. EST Friday. The event is being webcast and a link is provided at the event’s home page.