Spam-busters

MIT recently brought together the nation’s top spam fighters at its annual anti-spam conference. Network World caught up with some of the speakers and participants. Here are their stories.

Matthew Prince

Spam credentials: CEO, Unspam, a consulting company specializing in anti-spam laws; adjunct professor of law, John Marshall Law School, Chicago.

Most-hated spam: “That which contains inappropriate content and is targeted at children: solicitations for pornography, gambling, alcohol, tobacco. In most states it’s illegal to target these solicitations to children in the off-line world. It is disgusting to me that spammers can get away with doing it online.”

Favorite spam-fighting weapon: “I’m skeptical about filtering technology, on its own, as a solution to spam. Since there’s essentially no cost to sending e-mail, spammers’ response has been to increase the volume they send. The rise in spam almost exactly correlates with the deployment of filters.”

“I chose this work not only because unwanted messages constitute a modern plague I hoped I could help do something about, but also because spam presents challenging and cutting-edge legal issues,” Prince says.

As an example, he points out that while a number of states and countries have passed anti-spam laws “the problem is that an e-mail address alone doesn’t reveal its owner’s jurisdiction. So when you send to my address, there’s no way to tell what state or country’s laws you are subjecting yourself to,” he says. That causes trouble because “under just about every modern legal system, unless you have ‘purposefully availed’ yourself of a jurisdiction, you cannot be subject to its laws,” Prince says.

This loophole brings to mind the famous New Yorker cartoon by Peter Steiner with the caption, “On the Internet, nobody knows you’re a dog.” Such “semi-anonymity creates a problem for enforcement of anti-spam laws,” Prince says. “The caption could read, ‘On the Internet, no spammer can tell you’re a New Yorker.'”

This has been a problem on the state level and will continue to be a problem on the federal level. Prince says spam fighters tend to be passionate about their work. “Spam stirs people’s emotions because people view it, rightly, as an invasion,” he says. Unspam’s Web site gets reams of e-mail from users who are sick of spam, he says, and “the passion these people have for getting rid of spam spills over to those of us trying to come up with solutions.”

It’s not surprising that Prince says there is a place for law in the war on spam. “The law has one clear advantage over technology: It can impose costs,” he says. “While law will never be as efficient as technology, technology has no mechanism in an environment where the marginal costs are virtually zero to increase the cost.” As a result, he says, filters can stop a majority of e-mail from being delivered, but spammers just increase the number of messages they send. Therefore, virtually the same number of messages get through, and the spammers’ costs are unchanged. “But the overall costs to the network increase dramatically,” he says.

However, Prince says the first generation of anti-spam laws, including the new federal CAN-SPAM Act and European and Asian opt-in regulations, have been ineffective. “They did little to make prosecution cost-effective,” he says. “And an old legal adage says, ‘Without prosecution there is no law.'”

Thus, if new anti-spam laws are to be enforced, and therefore effective, Prince says he believes they must “decrease the cost of tracking down spammers, decrease the cost of bringing a trial, increase the likelihood of success at trial or increase the social benefit from winning a trial.” He applauds some state efforts that he says are headed in the right direction, such as child protection registries under consideration in Utah and Michigan.

“At its heart, spam is a problem of identity,” Prince says. “If you can tell who is sending the messages, then you can write laws to punish bad actors. Moreover, you can create filters that will actually be effective. As a result, the technologies that interest me the most are the ones that help establish and verify a sender’s identity. Until we can do that, I suspect spam will continue to be a serious problem, even as we develop better filters.”

Prince is optimistic about spam’s predicted demise. “If the spam economy behaves in a classical way, it should eventually defeat itself,” he says. “As the response rate for messages drops, eventually the costs should be high enough that being in the business of spam isn’t profitable. The advantage of a problem like spam over a problem like computer viruses is that the spammers aren’t just sending their messages for fun – they’re out to make money. If their costs get too high, they’ll move on to some other business.”

To hasten the death of spam, he says, “you need to impose a marginal cost on each message spammers send. This is why [Microsoft Chief Software Architect Bill] Gates’ proposed solution is to charge a small fee for every e-mail sent. That would work to impose significant costs on spammers and may stop a lot of spam, but I’m not sure the cure isn’t worse than the disease.”

Asked if he has conversed online or otherwise with spammers, Prince says, “The problem is very few people sending spam think of themselves as spammers. I’ve talked with a lot of people who think of themselves as ‘e-mail marketers’ but who engage in practices that are particularly troubling – trading lists, ‘losing’ opt-out requests, having an extremely loose definition of what it means to have opted in. Some of them genuinely don’t see a problem with what they’re doing. And if they were the only ones doing it, it wouldn’t be much of a problem.

“However, just as people who throw trash on the ground in a park justify it by thinking their trash alone won’t do much harm, some marketers I’ve talked to justify their behavior by arguing that their little indiscretions aren’t that bad. Unfortunately, just like trash in the park, a lot of small acts of bad behavior quickly multiply to a significant problem,” he says.

Terry Sullivan

Spam credentials: Software developer and Dallas-based researcher focused on statistical analysis and characterization of spam. Led the Internet Research Task Force workgroup on fighting spam.

Most-hated spam villain: “All of them! All spam is a form of attention theft, which is precisely what makes it bad. That said, I felt most heartsick when I ran across the foreign-currency-smuggling spam (commonly referred to as ‘Nigerian’ spam) that cloaked itself in religious language and Biblical quotations, trying to prey on the religious sentiments of unsuspecting innocents.”

Favorite spam-fighting weapons: “I favor a belt-and-suspenders approach. My in-box is protected with a heavily optimized rule-based classifier, a Bayesian filter, a small whitelist/blacklist and some minimal DNS-related stuff. Still, I do have a favorite genre of anti-spam solutions: the heuristic classifier (aka the rules-based classifier, sometimes called a feature detector).”

Sullivan joined the war on spam because of “a gradual, increasing frustration with the sheer amount of dreck that was accumulating in my in-box and a sense that if I wasn’t interested in ‘herbal Viagra’ yesterday, or the day before, why on Earth would I be interested in it today?”

When the aggravation hit a certain point, Sullivan says, “I finally said to myself, ‘I’m a bona fide expert in some very sophisticated, automatic, document-classification technologies. Why am I putting up with this?'” Within 24 hours, Sullivan says, two-thirds of all his incoming spam was being piped straight into an archive. “Two weeks later, it was around 98%. And I never looked back,” he says.

Sullivan is passionate about his work, “and that passion comes from having a sense of higher purpose, a sense of working on something really big,” he says. “People resent, viscerally and personally, any attempt to steal time from them. Spam fighters probably have a sense of ultimately allowing themselves and others to reclaim their time as their own.”

Asked about law and regulation, Sullivan says, “Legislation can only play an ancillary role in the fight against spam. Even with [the CAN-SPAM Act] in effect, the amount of spam sent continues to rise. And this should surprise no one. Spam betrays a fundamental dishonesty, as evidenced by widespread forgery of return addresses and subject lines. It’s reasonable to expect spammers to do everything they can to exploit the numerous loopholes in CAN-SPAM or simply to ignore it. There are also many spam-scammers who are known criminal frauds, and thus have always been illegal. Adding one more to the list of charges against them is not going to deter such folks.”

Sullivan says he believes most spam fighters are making a key mistake. At the MIT conference, he says, he found too much emphasis on “authentication technologies as the ‘solution.’ Every day users do not make their ham/spam judgment based on the source of the message. They make it based on the content of the message.” He adds, “Authentication doesn’t solve the problem of spam; it merely relocates it and in the process creates a thriving after-market in identity fraud/theft. I’m not sure why anyone thinks this is a good idea, unless it’s because authentication seems like an easier problem to solve than content analysis. But it’s still solving the wrong problem, and it raises disquieting possibilities for Big Brother-ism.”

Is spam on its way out in the relatively near future? “The turning point [in the battle] has already passed, but the denouement is still probably a long way off. The battle against spam will be much like the war in the Pacific Theater in World War II – slow, incremental progress, one island at a time. And remember, the battle of Midway – the turning point in that conflict – happened in June 1942, but the war didn’t end until August of ’45,” he says.

“The fact is, there are already plenty of brutally effective, though very heavy-handed, approaches that can make spam just about totally disappear,” Sullivan says. “The problem is that these approaches make e-mail much more inconvenient to use, essentially breaking it in the process. The larger challenge is to make spam go away without breaking e-mail.”

After reading interviews with spammers, Sullivan finds it hard to summon much sympathy for them. “The thing that surprises me the most is that they have the audacity to paint themselves as utterly innocent victims of some grand conspiracy,” he says. “It takes some pretty deep denial – or disingenuousness – for them not to recognize that the antipathy they provoke is a direct result of their own deceptive practices.”

Matt Knox

Spam credentials: Freelance anti-spam software developer in New York.

Most-hated spam villain: “Phishing,” in which a spammer sends out official-looking messages that purport to be from a legitimate company but are in fact attempts to steal personal information.

Favorite spam-fighting weapon: Naive Bayesian Filtering. Emerging technology: Naive Bayesian ffb, or “filter that fights back.”

For Matt Knox, the obsession with fighting spam all started with a pretty girl.

“One time [while traveling], I was talking to a very attractive young woman and asked her for her e-mail and phone number. She gave them to me. I lose things all the time, so I sent myself an e-mail containing her info from a friend’s account.”

A few days later, back at home, “I had lost the paper with her info, but it wasn’t a big deal because I had it in my e-mail account, right? Wrong. My in-box was jammed to the gills with spam, and the message had bounced,” he says.

It was then that Knox made a solemn vow: “That was the day spam died. It just hasn’t found out yet.” He says he believes that while spam is essentially a technical problem that demands a technical solution, some laws and regulations are eminently reasonable. “I love spam laws that require an ADV: tag or make illegal the forging of headers,” he says, referring to a proposed law that would mandate that the subject line in all spam e-mails begin with the four characters ADV:. “That is a superb instance of regulation – and will make filtering stone-easy,” he says.

“I would be less comfortable with [Digital Millennium Copyright Act] enforcement against spammers,” he says. Like many attendees at the MIT anti-spam conference he finds the DMCA grossly inadequate. Or, as he puts it, “brain-damaged.”

Knox says Bill Gates’ prediction of spam’s imminent demise is “probably close to true. There really is not a spam problem now, in the same way that, for most people, there is not a virus problem or a pop-up window problem. Anyone who doesn’t like pop-ups in [Microsoft Internet Explorer] can easily get another browser like Opera or Mozilla. And people who don’t like viruses can use Linux or OS X,” Knox says, conceding that “this may be more difficult.”

“Real spam filters are getting easy enough to be used by just about anyone who uses non-Web-based e-mail. People using Web-based e-mail are just stuck with whatever their service provider wants to give them. That’s usually a poorly done signature-based filter now – but those filters, too, will improve as time goes on,” he says.

So how deep does his resentment against spammers run? “I don’t despise spammers. I don’t like what they’re doing,” he says.

Shlomo Hershkop

Spam credential: Columbia University, New York, data mining lab, Ph.D. candidate.

Most-hated spam villain: “Virus spam – too many people are getting messed over and turned off from e-mail.”

Favorite spam-fighting weapon: Bayesian-based, such as SpamAssassin. “But I believe the next step is the user-based model.”

Hershkop entered the anti-spam arena because “we had a really cool algorithm in the anomaly detection field that needed to be applied outside of computer security,” he says.

Moreover, he likes that he’s doing work that could benefit end users and businesses: “Spam is moving away from being just an annoyance to being a security threat to both individual users and the e-mail system itself,” he says.

Asked if spam fighters are more passionate about their work than other technology professionals, Hershkop is skeptical. “I’m not sure,” he says. “Right now, spam is getting a lot of attention, the same way any other hot topic gets attention.”

Hershkop says he believes the spam plague will persist far longer than Gates’ prediction of mid-2005. “Too optimistic,” the grad student says. “Remember, the current e-mail system is in wide use not only between users, but also between systems. For example, I have a battery back-up system that sends out e-mails if there’s a problem. Upgrading users to new protocols will create huge headaches. Getting all the systems will take longer. You’d be surprised how much old – really old – stuff is still working on the Internet.”

In the end, Hershkop says, “It’s silly that we have to deal with these spam e-mails. Technology is already here that is smart enough to deal with it.”