Pager outage leaves managers in lurch

By Sandra Gittlen
Network World, 5/25/98

For network managers, a silent pager meant good news -- until last week.

That's when a satellite servicing more than 80% of the nation's 45 million pagers went on the blink, cutting off the primary means many net managers use to be notified about troubles on their nets.

"We didn't even know that the pagers were down so we couldn't go into emergency response mode," said David Brandon, director of new product services at SBC Communications, Inc. in Dallas. Brandon missed pages about an authentication problem in California that was blocking users from dialing in to the company's network. Had he known the pagers were down, Brandon said he would have called in to get his messages periodically.

"I live and breathe by the pager for everything from circuit reports to router reports to network events," he said. "I have monitoring systems configured to page me if a [network equipment room] starts to overheat. If the temperature in a room climbs to 90 degrees, I'll get paged."

And if that happens, timing is critical - somebody has to get to the remote site quickly because extreme heat shuts down the routers, causing severe traffic problems on the network.

From now on, Brandon said team members will have to keep their cell phones on at all times.

This is a policy Dwight Gibbs, chief technical fool at online financial advisor The Motley Fool, Inc., already has in place.

"My team has to keep their cell phones on at all times . . . except when they are at their desks charging them," he said. "Pagers are too limited. On a cell phone, at least you can contact someone and talk to them immediately."

He said the reliance on pagers for network managers started about five years ago when cell phones were cost prohibitive. "Cell phones are so cheap now, there's no reason not to use them."

For Brad Williams, senior telecommunications analyst at Pier 1 Imports, Inc. in Fort Worth, Texas, alerts via a cell phone would have at least allowed him to leave his desk. Instead, Williams said he was "chained" there last week, monitoring e-mail alerts.

The pager outage -- the first of its kind, according to wireless experts -- started last Tuesday when a PanAmSat, Inc. satellite, the Galaxy IV, lost its orientation to Earth. This problem resulted from a failure in the satellite's onboard processor and backup system. The processor keeps the satellite pointed toward Earth, according to PanAmSat.

The Galaxy IV will not be restored, PanAmSat said. Instead, the company is switching traffic to the firm's Galaxy III-R satellite and considering other backup plans. However, some pager companies have already redirected their traffic to other vendors' satellites.

"Almost all our pager customers have been offered an alternative [to the Galaxy IV]," said a PanAmSat spokeswoman. "Some customers had backup agreements that allowed them to immediately switch over and others are in the process now of reconfiguring their antennas to point to Galaxy III-R."

She said the likelihood of this type of accident happening again is slim. Only 1% of the satellites that have been sent to space in the past five years have been lost. She would not comment on payment penalties the company might be faced with.

"Almost every network in the world is making use of pager notifications," said Rodney Joffe, vice president of strategic technology at GTE Internetworking. "Net managers hadn't realized how vulnerable we are. We'll be spending some time thinking about the alternatives [to pagers]."

"This outage proves that people should invoke more than one method of alerts to more than one person," said T.M. Ravi, vice president of enterprise marketing at Computer Associates International, Inc. CA makes network monitoring software that can automatically alert network managers via pagers, phone, e-mail or fax.

Ravi said pagers are fine for less critical notifications, but as a problem escalates, other means of communication should be used to contact appropriate personnel, including cell phone calls and e-mail.

But Iain Gillott, vice president of worldwide consumer and small business telecom for International Data Corp., said cell phones aren't as reliable as pagers. "I look at this as a glitch," he said. "There really isn't much else that compares to the price and the geographic coverage of a pager."

Instead, Gillott said net managers should start pressing their paging services for service-level guarantees like those offered in the frame relay carrier market. He said some companies have SLAs, but "they don't offer them immediately. You have to ask."