Reputation systems are problematic

Opinion
Dec 14, 20043 mins

* Some thoughts on authentication and reputation systems

I recently received some interesting and insightful comments on reputation and authentication from two individuals at a Canadian anti-spam start-up.

Here are their paraphrased comments:

* * *

At a high level, a public domain reputation system backed by authentication seems to be a good idea. But it’s not a panacea, particularly from the vantage point of an ISP. Say an ISP sends out millions of low-quality e-mail messages each day as a result of zombies and poorly secured client computers, problems that are largely outside its control. As a result, one would assume the ISP’s domain would be poorly rated by reputation systems. The ISP’s legitimate customers’ e-mail will be more likely to be filtered out or throttled than e-mail from other domains.

While domain reputation works well for big domains like amazon.com or ibm.com, it doesn’t work well for big ISP domains. Unfortunately, domain reputation will be limited in effect to promoting the priority of traffic from domains that have a relatively homogenous and well-behaved sender population, such as large enterprises. Smaller organizations, in-home businesses and individuals won’t fare so well.

At its core, sender authentication is really domain authentication. As pointed out above, domain authentication is often not fine-grained enough to make accurate decisions about mail delivery. Where it does work very well is in preventing joe-jobbing (sending spam purporting to be from someone else’s domain). While sender authentication’s wide deployment will be useful even if that is the only benefit, it will never make a good general-purpose anti-spam system.

In addition to the problem of not being fine-grained enough, there is the problem of the reputation system itself. Hierarchical trust systems in which money is involved can become very corrupt very quickly. Anecdotally, one company was issuing OpenSRS SSL certificates with all contact information based in Sealand (a former U.K. gun platform turned principality six miles off the east coast of England). Each certificate took about 15 minutes to issue.

Individual organizations could implement their own reputation systems, deciding who they will trust on an ad hoc basis. This may be more workable, but leads to the problem of the sender never knowing if their mail will be deliverable. Sending mail then becomes unpredictable – what works on one domain doesn’t on another. It’s as if every domain becomes a tiny nation, each with its own strange customs laws.

For these reasons, we don’t see reputation as being particularly useful in fighting spam. Perhaps it will become useful if a system can be devised that can come closer to actually authenticating the sender, not just their domain or network.

* * *

Thank you to the two individuals who provided their comments (and who chose to do so anonymously). What are your thoughts on authentication and reputation? Please drop me a line at mailto:michael@ostermanresearch.com