The traditional signature-based method to detect viruses and other malware is increasingly seen as an insufficient defense given the rapid pace at which attackers are churning out virus and spyware variants. All of which raises the question: What’s next?

The three security vendors that dominate the antivirus market today, McAfee, Symantec and Trend Micro, say they have no intention of abandoning signature-based defense, which calls for identifying a specific malware sample to create a matching signature in order to detect and eradicate it. However, the big three vendors acknowledge there’s a need to augment this decades-old methodology, and some of the new techniques they’re devising will be unveiled as products this year.

“Everyone agrees signature-based defense is not enough,” says Brian Foster, Symantec’s senior director for product management, who notes the security firm receives 200,000 submissions of potential malware each month. “The number of variants is increasing.”

To augment signature-based detection in its next enterprise antivirus release planned for this summer, Symantec will include whitelisting technology for policy-based control of applications down to a software-component level, says Foster. This future-looking malware protection from Symantec will also make use of behavior blocking that promises to be able to stop at least some malware from executing, holding it “in a frozen state on that machine,” says Foster. “The core of our strategy is, we will change the game.”

Security startups to watch

In the meantime, some brash start-ups say they realized years back the malware-defense game had changed -- and they’re now elbowing their way in by playing it differently.

One is SignaCert, launched earlier this year to market enterprise desktop and server software that can be used to create a white list that only allows specified applications and files to work.

“We’ve definitely reached a point of diminishing returns with traditional signatures,” says SignaCert chair and CEO Wyatt Starnes.

With its Enterprise Trust Server product, SignaCert has created encryption-based signatures of binary-software releases obtained directly from vendors, including Sun, Microsoft, IBM and Intel. “Because you know what the good code is, you don’t let the bad code run,” Starnes says.