The buy side

When Lab Alliance members go commercial, these are the tools they seek out.

Back to Open source, tried and true

While many Network World Lab Alliance members favor open source tools for testing, some say commercial tools are more appropriate at times. And at least one, Joel Snyder, senior partner at Opus One, will only use commercial tools.

"We don't actually use any open source tools anymore. We've had too many bad experiences . . . and they aren't oriented toward scientific testing," Snyder says.

He points to TCP and User Datagram Protocol (UDP) throughput testing as an example of in which commercial tools work better. Testers "like to use [Test TCP], which is a really simple program. But you can't get statistics as the flows go by - you only get a single set of numbers at the end. This means you can't see the behavior of the system as the load changes."

Plus, Snyder says, open source tools don't have a high degree of parallelism. "They sit on top of an operating system and are only good for the simplest of tests," he says.

Most testing of security hardware and software tools he performs at his lab can be done with commercial tools. Among his favorites: Spirent Communications' suite of network assurance and diagnostic tools.

"They're so much more reliable" than open source tools, he says. And if Spirent doesn't have what he needs, Snyder says he writes it himself.

Network Test President David Newman, who uses Spirent's tools for switch, router and Layer 4-7 testing, says open source tools oftentimes fail at scalability. "For the kind of box testing I do, there just aren't open source tools to generate tens or hundreds of ports at gigabits per second and then collate those results," he says.

Even open source advocate Thomas Henderson, principal researcher at ExtremeLabs, says sometimes the situation dictates the use of commercial tools. "I prefer tools that are open source or shared source so that I can guarantee they haven't been modified to suit the results. But no matter the source status, we have to use the best and most appropriate tools for the characteristics we're testing," he says.

Henderson counts among his trusty fallbacks Ixia's Chariot and Fluke's Layer 2 and Layer 3 tools such as OptiView .

Some Lab Alliance members have a set of rules for when commercial tools are a better bet. Rodney Thayer, a private network security consultant, offers this advice: "If you're in an environment that must demonstrate it uses equipment and tools with an active maintenance contract in place," then commercial testing tools are necessary.

Thayer, also a fan of Spirent's offerings and security scanners from Agilent, says he avoids using open source tools that require him to compromise confidential information in the products to satisfy the license.

Learn more about this topic

Be sure to read these other great Signature Series issues
Power Issue


Buzz Issue You Issue NW 200
Join the Network World communities on Facebook and LinkedIn to comment on topics that are top of mind.

Copyright © 2005 IDG Communications, Inc.