WASHINGTON – Testing in-house and vendor-built software for security holes should be an enterprise priority, said a group of vulnerability research experts speaking on a panel at the Gartner IT Security Summit held here this week. But Rich Mogull, the Gartner analyst who hosted the panel, questioned how practical it would be for companies to dedicate the dollars and resources required for this testing.

Thomas Ptacek, founder of application security consulting firm Matasano Security, defined vulnerability research as analyzing software for holes that attackers could take advantage of before the product is deployed, using techniques such as reverse engineering and source-code auditing. Software vendors and many enterprises have teams of engineers in house to perform this testing, or rely on third parties such as the panelists’ companies that specialize in finding vulnerabilities.

The benefit of this testing is being able to avoid the damage an attacker could cause by fixing software problems before implementation.

“If you don’t find the problems, someone [else] will find the problems,” said Chris Wysopal, co-founder of Veracode. “If you leave crumbs on the floor the ants are going to show up. That’s a huge liability … for your company.”

For software built in-house, vulnerability testing should be part of the software development life cycle, not an afterthought, Wysopal said.

“Threat modeling to find out what are the weakest parts and easiest attack vectors [of an application] is what people should do when designing software; you find the weak points through threat modeling then start reverse engineering,” he said.

Simply using tools that scan for vulnerabilities is not enough, the panelists agreed.

“Scanning tools can reduce the amount of time you spend [analyzing the code] manually, but if you care about the security of the application … you need to go deep and augment the scanner,” says Ptacek. “The place of the scanner is to accelerate testing, but you can’t rely on them.”

Gartner’s Mogull did an electronic poll of the roughly 1,000 conference attendees, asking what level of vulnerability testing they performed at their organizations. The majority said their testing was limited to using commercial scanning tools.