Script: Disaster recovery planning audio primer

The Sept. 11 terrorist attacks made many network professionals dust off their disaster-recovery plans. What would happen to corporate and customer information if an office or data center were destroyed or temporarily taken offline? Would business ground to a halt for weeks, or just a matter of hours or minutes?

In this primer, we'll take a look at how to start the disaster recovery planning process, what needs to be included in a plan and some of the options that are available.

According to the Federal Emergency Management Agency (FEMA), an emergency is any unplanned event that affects business operations, including such things as fires, earthquakes, snowstorms and civil disobedience. FEMA offers four steps in the disaster recovery planning process: Establish a planning team; analyze company capabilities and hazards; develop the plan; and implement the plan.

The core of the planning team should have a small number of active participants, but have advisory members representing every functional area of the company. Also, the team should have support and input from top company executives, as well as its own budget and schedule.

Once a planning team is established, the group should look at all existing resources for backup and recovery, as well as alternate telecommunications and power resources. FEMA also recommends that outside agencies be consulted about their plans for dealing with potential emergencies and disasters. Does the power company have plans for dealing with outages? Can the local phone company be counted on to get lines back up and running quickly?

Next, critical areas of business need to be identified. Which departments and resources are most critical for keeping the company running during and after an emergency? Production facilities, utilities, operations personnel and IT equipment should all be taken into consideration. The planning team should also look at the capabilities within its staff for support in an emergency - who can double up in which tasks?

With critical areas identified, it's time to check on and ensure back-up arrangements for data and networks. Among the questions to ask: Do you have offsite storage for tape backups or hotsite backups of data centers that can come online within seconds of your primary site going down? Can your suppliers deliver replacement equipment quickly? Do you have multiple telecommunications connections and fallback procedures in place in case your main connections go down?

Larger companies with multiple locations could have an easier time building hotsite backups ready to come online at moments notice. A hotsite backup is an exact replica of a primary data center. It can take the form of a dormant center that does not handle general traffic but does get a copy of the production data at regular intervals to make the transition from the primary to the backup invisible to end users. Another option is to build extra capacity into existing active datacenters to make them capable of handling their own traffic plus that of the downed facility. The latter could be a better option for companies with datacenters in multiple geographic locations.

Building and/or equipping alternate datacenters can be expensive, given the need for duplicate routers, servers and costly data connections. Also, all production data has to be copied at least daily, if not hourly, over to the back-up center, ensuring it can come online and fulfill customer and user requests.

Companies such as SunGard offer business continuity services that allow companies to outsource the need for back-up facilities. These companies have facilities set up around the country that clients can move to in the event of an emergency or disaster. The downside to some of these services is they offer their space on a first-come, first-serve basis, which could be a problem if multiple customers are affected simultaneously in the same area.

For smaller operations, off site tape backup may be the only affordable option. Data and applications should be backed up and sent off site in the case of an emergency. A rotating a set of daily back-up tapes over a week to two-week period should be suitable for most needs. In this case the first Monday tape would be overwritten two weeks later with a new daily backup. A monthly backup should also be made and sent off site for at least 6 months, maybe a year. The storage time depends on the needs of the business.

The offsite location could be a third-party storage service or another building your company occupies.

Also, tools for migrating a user's desktop from one machine to another can also be used to back up individual settings. These desktop images should be included on any tape back-up system. This could be helpful in getting each user up and running on new equipment.

One note of caution: in the event of equipment being physically lost, the new system that backups are being restored to may not be the same as the original equipment. This could cause some configuration difficulties.

Some natural disasters may not be long term problems. A snowstorm, more of an emergency than a disaster, could keep employees out of a datacenter for a day or two at a time. Remote monitoring and access could keep machines up and running even though a technician can not physically reach a building. United Parcel Service has two datacenters in different parts of the country. Staff at each center can run both sites in the case of a natural disaster at one of the locations. Plus, most functions are duplicated at each center, so if one goes down, the other can handle most of the load.

A few more things to consider when implementing a disaster recovery plan. More than one person must be aware of its existence and be able to implement it in case of emergency. Also, make sure the appropriate people have access to offsite storage to retrieve tapes. If only one person has access and that person is unavailable, restoration of services could be slowed.

In a worse case scenario, plans should be made in the event that the entire IS staff is lost in a disaster. Senior management has to be able to implement a plan to replace the lost staff and bring up core systems. Identify companies or services that can come in to help get your company's IS infrastructure on its feet. Consultants initially used to help build core systems may be called on to help restore damaged or lost equipment. Keeping track of former employees with knowledge of your system could also help.

Finally, testing the recovery plan periodically helps ensure its success in the event of a real emergency. The goal is to plan and prepare for the worst, but hope for the best.

Back to the main Disaster Recovery Planning primer page

Other disaster recovery resources

RELATED LINKS