Hungry for automation

 

Destination: Advanced IP paradise IPv6: The missing link? Trouble in paradise? Hungry for automation Thinking through virtualization Mainframe for rent How to get IT right

Restaurateur Red Robin is devouring new data center technology. Use of automated management, for example, has led to impressive performance and financial gains, such as a reduction of network downtime by 50% and a resulting $130,000 saved in 2004. Now the staff has discovered that security information management (SIM) can greatly cool the sting of Sarbanes-Oxley audits. Bill Randall, director of MIS infrastructure at the Denver company, shares his insights on the payback of proactive management, the realities of using today's automation tools and more in an interview with Signature Series Executive Editor Julie Bort.

Red Robin has been hopping with cool, new technology since building your new data center. How are you managing it all?

We started out with NetIQ AppManager Suite two years ago just to help us keep a grip on everything. We have over 40 servers sitting on a rack, and I don't want to pay an engineer for logging in and checking on them everyday. My engineers are paid to think and work on projects. So it made sense to have AppManager monitor our systems and check that the processors, temperature, memory and so on look fine. If it sees that, hey, memory utilization has been over 50% now for over eight hours, it notifies us so we can take the appropriate action.

Are you using advanced automation - where processes a human used to do by hand are now done automatically?

There are things that we let NetIQ do. For instance, if utilization gets to a certain point on our NAS box and stays there, we know that it just needs a restart. Somehow memory utilization creeps up on Windows boxes until you reboot. So it monitors for that and reboots the box at an off-hour and sends us a notification.

I don't know that we're ready to turn the keys entirely over to any automation program. Remember that Matthew Broderick movie - "WarGames" - where the computer was ready to declare World War III? I don't think that would happen in our network, but we're still careful. Automation technology lacks is the ability to test before it acts. We need to find out what the impact is going to be. If you change the way SNMP is done, for example, it may affect the backup server and several other environments. We look at automation from a standpoint of wanting [the monitoring system] to collect information, make recommendations, and let us use our human and engineering perspectives to determine whether that recommendation actually applies. Once we've tested it and know the right course to take, we can script and automate.

You attribute a 50% reduction in downtime, which saved some $130,000 last year, to automation. Can you give me an example?

We looked at what our downtime was a year prior vs. a year after implementing AppManager. Outages that happen with a network infrastructure are often due to silly things - no one notices the print server is turned on until it finally runs out of disk space and print services go down because it crashes the operating system. Then you've got to rebuild it from the [backup] image. We now take a proactive approach and we've alleviated most types of errors like memory or disk space [problems] that give you page fault errors. With a proactive approach, we'll replace that memory right away, before it can corrupt the application.