Skip Links

Artificial intelligence scopes out spam

By Dave Strickler, special to Network World
April 14, 2003 12:10 AM ET
This vendor-written tech primer has been edited by Network World to eliminate product promotion, but readers should note it will likely favor the submitter's approach.

Network World - In the cat-and-mouse game of the antispam industry, staying one step ahead of spammers is difficult because they constantly exploit the weaknesses of e-mail keyword filtering. But the newest artificial-intelligence filtering technology may adapt faster than the spammers can alter their messages.

Artificial intelligence techniques closely resemble the way our brains learn. Once we learn a skill, we use it to reason with. Using artificial intelligence to detect spam is done in the same way.

Natural-language processors serve as powerful artificial intelligence tools in the fight against spam. These processors, which actually are an array of complex algorithms, scan e-mail messages to discover the content of the messages. The algorithms are packaged into mail-filtering software, which generally sits outside a firewall or at an application service provider's network.

Artificial intelligence mail-filtering software accepts all in-bound e-mail traffic, routing legitimate traffic to a corporate SMTP server and flagging other messages as spam. Suspect e-mail is sent to a quarantine area where an administrator can view the contents to determine whether to discard it or pass it along.

Humans can quickly skim a message to judge if it is spam. Referencing keywords by their location in a sentence lets us understand the difference between "chicken breasts" as food and "bare breasts" as pornography. Similarly, natural-language algorithms break down messages into sentences and analyze their meaning.

With considerable processing effort, natural-language processing technology pieces together the meaning of messages by analyzing the words, sentences and paragraphs in the reverse order from which the algorithms originally took them apart.

Consider this e-mail example: "These delicious chicken breasts look good enough to eat - let's cook out tonight. If you can bring Bill, call me at work, 800-262-2222 x231. Oh, and check out the pictures from our last cookout at>." A standard keyword analysis would flag the terms: breasts, look good, toll free number, Web site URL. But artificial intelligence analysis would determine the message was an invitation to dinner.

In the example, keyword-filtering technology picks out pieces of the sentence without really understanding its meaning. Its selective hearing incorrectly determines that the sentence is pornographic.

Another challenge is picking out legitimate business correspondence. For example, an e-mail from a mortgage broker to a client might say, "Sam, I did some digging, and I found some unbelievably low mortgage rates with no money down this morning. If you want to get one, you'll need to call me today so I can lock the rate in for you. One of the rates expires at midnight tonight." A standard keyword analysis would classify the message as spam based on the terms: Low mortgage rates, no money down, expires at midnight. However, artificial intelligence analysis would reveal that this was a correspondence regarding mortgages.

Our Commenting Policies
Latest News
rssRss Feed
View more Latest News