The beta benefit

Testing buggy products can give your company a competitive edge.

In 15 years at Case Western Reserve University in Cleveland, systems engineer Jim Nauer has been involved in six major beta tests. In each case, Nauer was motivated by the failure of existing technology to meet Case's technology or business needs. For example, in the mid-1990s, the university beta-tested a native version of Novell NetWare that ran on Apple's PowerPC platform. Nauer was excited about the product's potential.

"It meant guaranteed hardware compatibility and plug-and-play device drivers, and in those days, there was quite a bit of voodoo involved in getting Ethernet and SCSI cards configured correctly," he says. "You didn't just pull [the network operating system] out of the box and click through a wizard to install it."

To continue reading, register here and become an Insider. You'll get free access to premium content from CIO, Computerworld, CSO, InfoWorld, and Network World. See more Insider content or sign in.

In 15 years at Case Western Reserve University in Cleveland, systems engineer Jim Nauer has been involved in six major beta tests. In each case, Nauer was motivated by the failure of existing technology to meet Case's technology or business needs. For example, in the mid-1990s, the university beta-tested a native version of Novell NetWare that ran on Apple's PowerPC platform. Nauer was excited about the product's potential.

Click to see: Beta testers Jim Nauer and Barron Hulver

"It meant guaranteed hardware compatibility and plug-and-play device drivers, and in those days, there was quite a bit of voodoo involved in getting Ethernet and SCSI cards configured correctly," he says. "You didn't just pull [the network operating system] out of the box and click through a wizard to install it."

The beta product worked beautifully, he says. Unfortunately, Apple decided not to develop it commercially. "It would have been really neat if they'd finished debugging it and it solved some of our issues," he says. "But it didn't come to pass."

As Nauer has found, during the Apple beta and his other more successful tests, there are numerous benefits to being a beta tester. The most important is the ability to get your hands on cutting-edge technology to gain a competitive advantage.

However, be prepared to deal with the frustrations, the resource demands and the possible disappointments that accompany the experience. Not only does beta testing stretch the resources of IT staff members, but it also can try their patience, as they are, after all, working with buggy products. Vendor approaches to beta testing can range from very organized to haphazard, and your own testing and bug-reporting process has to be rigorous.

As Nauer found with the Apple beta, there's no guarantee the product will ever hit the market. Still, Nauer says Case Western came out a winner. "It enabled us to get our hands on NetWare 4.1 code before we had the PC native version," he says. "We got knowledge and experience we were able to apply elsewhere."

In fact, other beta testers report that even when things go wrong, they still feel the pros outweigh the cons. Reasons include early access to new technology that can solve long-standing problems, the opportunity to influence product development and a direct line to code engineers.

"You're doing it because you have a vision; you want to beat the guy down the street and improve your revenues quarter over quarter,'' says Louis Barton, executive vice president at Cullen/Frost Bankers in San Antonio.