Measuring satisfaction

Several weeks ago, Richard Nissel was almost done installing a Cisco Systems, Inc. PIX Firewall at the West Virginia state treasurer's office when the network went down. The outage brought business throughout the state government complex in Charleston to a screeching halt for more than three hours.

The problem was found after several frantic hours and numerous calls to Cisco's help desk. As it turned out, a help desk technician at Cisco misconfigured a router on the perimeter of the network, says Nissel, an automated information systems director for the state. Because of the incident, Nissel won't let workers access the routers for repair unless the technicians are on-site and tell him what they're going to do first.

Service and support is the invisible glue that binds networks. Under the best circumstances, it absorbs the stress of fractures, allowing business to continue with the fewest disruptions possible. In the worst situations, well-intentioned but faulty assistance exacerbates complex problems, fraying the nerves of overworked network managers and costing companies thousands of dollars in downtime.

Given its importance, service and support increasingly is becoming a key factor in purchase decisions. We conducted the inaugural 1998 Network World Service and Support Survey to find out which vendors do it best. Sponsored in part by Deloitte Consulting and conducted by STAT Resources, Inc., the survey shows that buyers of PC servers, groupware and internetworking hardware are mostly satisfied with their vendors' technical support. Still, with few exceptions, the results show there is plenty of room for improvement among vendors.

We mailed two individual surveys, one on servers and groupware and another on internetwork hardware, to 1,000 readers. Both surveys garnered responses from a total of 646 Network World readers employed in the IT field. Customers ranked their primary and secondary vendors on five principal service and support characteristics: cost, reliability, repair service, technical support and overall service and support. Not surprisingly, vendors usually scored higher among their primary customers rather than with users in general.

Respondents were then polled in more depth on their level of satisfaction with their primary vendors. Scores ranged from a possible low of 50 (not very satisfied) to a high of 100 (extremely satisfied).

We gave special Network World Honor Roll recognition to the few vendors that earned an overall satisfaction rating of 85 or more from their primary customers. Internetwork hardware vendors needed to score at least 85 in each of the switch, hub and router product categories.

The survey found the highest levels of satisfaction among customers of internetworking hardware vendors that sell switches, routers and hubs. This group notched an overall service and support score of 84, a figure nearing the upper edge of "somewhat satisfied." Server vendors were a smidgen behind, with users giving them an average rating of 83 for overall service and support.

Groupware buyers, however, were less satisfied. The average overall service and support rating of 80 hits the bottom end of the "somewhat satisfied" category, which hardly indicates customers are thrilled by service levels.

None of the vendors in any of the three categories — servers, groupware and internetwork hardware — ran away from the pack, though some managed to edge out the competition. Any difference greater than 2.5 between ratings is considered statistically significant.

David Lee, a senior manager with Deloitte Consulting in New York, is not surprised by the modest rankings. Among other things, the scores explain the growth in internal help desks and third-party knowledge bases, he says. "The whole niche industry around service and support has been created precisely as a result of the somewhat satisfied medium rating for these vendors," Lee says. While pinning the blame largely on vendors, Lee and others at his firm who reviewed the results say customers hold some of the responsibility for ensuring they receive adequate service and support.

"There are not a lot of companies that do a real good job of managing their vendors," says Rodney Runolfson, senior consultant at Deloitte Consulting.

"Often, they don't ask the right questions and don't have clearly defined service-level agreements."

Server buyers are generally satisfied
Click for complete server report card

On the server playing field, the scores of Compaq Computer Corp., Dell Computer Corp., Digital Equipment Corp., Hewlett-Packard Co. and IBM were too close to name a front-runner. As a whole, the group scored an average overall service and support rating of 83.

John Jaklitsch, a network specialist at the Pocono Medical Center in East Stroudsburg, Pa., has seen great improvements in the service provided by IBM in the past year. Jaklitsch had a problem getting software to work with an AS/400 server several weeks ago and got the problem solved almost immediately by calling IBM, even though the glitch appeared to be with the third-party software. "I waited about 10 minutes on the phone but then someone came on and said, 'Here's what you need to do.' [The fix] worked just like he said it would," Jaklitsch recalls. "In the past, they would have said, 'You need to call the third-party vendor."'

With the exception of Dell, server buyers as a whole were most dissatisfied with the cost of their vendors' service and support plans. Dell led the pack with a score of 85, while Compaq, HP and IBM turned in their lowest scores of the principal service and support characteristics.

The survey also queried each vendor's primary users about 13 more specific areas of the vendor's technical assistance, such as availability of required parts, Web-based support, escalation, and ease of submitting trouble tickets. Those questions elicited some differences.

Dell surpassed the other server vendors in the areas of timely problem resolution, availability of required parts and Web-based technical support. Dell received the high scores of 86, 86 and 86, across the board. IBM's score of 79 for availability of required parts indicates the vendor has the most improvement to make in that area. IBM also scored lower than its competitors for its support call escalation procedures. Compaq earned a marginal rating of 76 for following up with customers to see if they were satisfied with the way a problem was resolved.

Although Compaq didn't score nearly as high as Dell when it came to Web-based technical support, the vendor earned a respectable 81 and won raves from customer Todd Everett. As LAN administrator for Cutler-Hammer, Inc., a circuit breaker and electrical distribution equipment manufacturer in Pittsburgh, Everett regularly turns to Compaq's Web site to locate and download firmware upgrades and drivers for servers, video cards and network adapters.

"Compaq's Web-based support is magnificent," Everett says. "If you lose a driver, all you do is log on to their Web site, search for a driver, download it, turn on the executable, and then it creates two new diskettes exactly like the ones you lost. It's instantaneous, and I don't have to wait for anybody."

Novell grabs groupware lead
Click for complete groupware report card

None of the groupware vendors did well enough to make the Honor Roll. But among them, Novell, Inc. was the clear leader in overall satisfacting with a rating of 81 among primary users. Chief competitors Lotus Development Corp. and Microsoft Corp. tied with a rating of 78.

Novell topped the groupware rankings in virtually all individual categories, including the key areas of cost of service and support, bug fixes and patches, and telephone support. The vendor particularly excelled in the area of product reliability, with a score of 89, compared to 85 for Lotus and 84 for Microsoft. The results show that all three groupware vendors need to improve their on-site support. Microsoft earned a mere 70, which also was the lowest single rating recorded in the 1998 Network World Service and Support Survey. Lotus and Novell each scored a 76. Customers also would like vendors to solve their groupware problems more quickly. Novell's mediocre rating of 79 was the highest for time to resolve problems, while Lotus earned an unimpressive 74.

Cabelas, Inc., a Sidney, Neb., mail-order house for hunting and fishing gear, uses Novell's GroupWise 5.1 e-mail for its 2,000 employees. Lyle Borcher, the company's MIS director, has nothing but good things to say about Novell's support. Although he doesn't have a lot of problems with GroupWise, Borcher says Novell supports the product as well as it does IntranetWare.

Deloitte Consulting's Runolfson says Novell's higher ratings show that in spite of being smaller than Microsoft or the combined IBM/Lotus, the vendor is winning mind share among customers.

"Providing good customer service can be a cost-effective method for increasing or maintaining your customer base," Runolfson says. "When IT decision makers find out that Novell is not the biggest company but that it provides good quality service, maybe they are going to lean in its direction."

When it comes to groupware, or any area of service and support for that matter, some customers suggest paying to upgrade from "business class" to "first class." Michael Nelkens, vice president of market data and PC support for New York-based Global Financial Markets, says that's particularly true when dealing with Microsoft, which provides his company with Premier support for Microsoft Office and Exchange.

"With regular support it takes time to solve the problems. But with Premier you can get them to make it a higher priority, and you get a faster response," Nelkens says, adding, though, that he's unsure of the additional costs involved.

Network hardware ratings are closely huddled
Click for complete router report card
Click for complete hub report card

Internetwork hardware fared the best compared to all other products rated in the survey, though few results significantly differed among vendors. 3Com Corp., Bay Networks, Inc., Cabletron Systems, Inc. and Cisco all had closely matched high marks for overall service and support of internetwork hardware in general, while IBM had the lowest score, 83.

Primary customers were happy with their switches. Vendors' overall satisfaction ratings were too close to be significant, but the product category as a whole earned an average overall satisfaction rating of 85.

Switch buyers were particularly pleased with product reliability, giving all vendors impressive scores in the low 90s. Perhaps all four vendors tied with identical scores of 88 for repair service because it was so infrequently needed. Cabletron aced the competition when it came to technical support. Pulling in a 91, Cabletron scored five to six points ahead of its competitors. Cabletron was also tops for promptly responding to problems, with a score of 88.

Steven Boudreau, manager of network services for the Rhode Island School of Design in Providence, likes the customized maintenance program Cabletron designed for him. The program assigns different levels of technical support to devices, depending on their importance to the network.

His 35 100M bit/sec Ethernet backbone switches are covered under the Parts Express program, through which Cabletron provides an on- site technician and replacement parts within four hours, 24 hours a day. Meanwhile, items that aren't as critical, such as the dozen or so workgroup switches on the network, are covered under a separate portion of the program that promises replacement or repair by the next business day.

"I'm very pleased with their support," Boudreau says. "It makes sense to have Parts Express on mission-critical pieces, but why pay the premium on an old Ethernet repeater? For that, the program would have [cost] more than the item itself." Although Boudreau declined to discuss specific costs of the program, he says it's cost-effective for the school.

Boudreau first took advantage of Parts Express about a month ago, when a switch appeared to have failed. "Within three hours, I had a new 100M bit/sec switch," he says. "It turned out it wasn't a hardware problem but a software configuration issue. But still, there were no questions asked. It was terrific."

HP joined the big four internetwork hardware vendors at the hub party and came home with the best overall service and support score. HP also pulled off a close victory in three hub support categories: cost of service and support, repair service and product reliability. Indeed, HP scored a whopping 95 in the latter category, the highest score recorded in the entire survey.

At North American Products Corp. in Jackson, Ind., 10 HP 24TX AdvanceStack hubs are key components of the LAN that serves the 150 employees of the commercial cutting tool manufacturer and distributor. James Gatwood, North American's IS director, says the firm started two years ago with eight 10M bit/sec Ethernet hubs and recently added a pair of 100M bit/sec hubs to the network, which primarily handles administrative, bookkeeping and e-mail functions.

"It's definitely a case of 'put 'em in and they go,"' he says. "I haven't had a lick of trouble out of them."

However, Cisco beat its competitors in practically every support attribute rated by primary hub customers, earning an overall satisfaction rating of 88. Cisco earned high scores of 89 for support representatives' competency and availability of required parts, while Bay's scores were at the back of the pack for several hub support categories.

Meanwhile, Cabletron's hubs didn't do as well as the vendor's switches among the company's primary customers. The company's 81 for cost of service and support was the lowest among hub vendors, as was its 80 for Web-based technical support.

These weaknesses aren't apparent to Roman Hryniszak, a network operator for the Los Angeles Unified School District. The district uses about 30 Cabletron hubs to help tie together a network used by students, teachers and administrators in an 11-school segment of the district known as the Bell Cluster. "All the hubs do is work, and that's it," Hryniszak says.

While Cisco dominated the router category in terms of market share with 58%, it only narrowly edged its main competitors — 3Com and Bay — in the rating of overall service and support. The ratings for many support attributes were too similar to be considered significant.

In a sign that its competitors are using pricing as a way of trolling for market share, Cisco scored marginally lower than its chief router competition on cost of service and support. Bay earned the high score of 86, while Cisco scored an 83 for support pricing. However, primary Cisco customers gave the vendor an 88 for technical support in comparison to Bay's 84.

3Com won accolades for product reliability and repair service, scoring top marks of 93 and 89.

Nissel, of the West Virginia state treasurer's office, is generally happy with Cisco's customer support, but he's learned a few lessons from his experience with the router configuration mishap. Among other things, it highlighted the need to deal with technicians who have a good understanding of the whole network, not just the particular piece on which they might be working. In that particular case, the technician who usually works on the network had been called away to another job before the work was complete and didn't return, he says.

But Nissel also concedes he might have goofed by allowing someone who was unfamiliar with the network to fiddle with a router. He now makes sure his employees know when to tell vendors' technical support representatives not to do something, he says.

A spokesman for Cisco says the company did its best under the circumstances but acknowledges that in the world of service and support, sometimes things just go wrong.

"We pride ourselves on giving customers excellent service," says Robert Michelet, Cisco's director of corporate relations in San Jose, Calif. "Anytime somebody gets something other than that, we're obviously really disappointed."

Duffy is a freelance writer in Somerville, Mass. He can be reached at tduffy62@ compuserve.com.