Going the distance for business continuity

IP steps up as a transport option for out-of-region replication

Typical of any company in the financial industry, Euronext.liffe puts a premium on business continuity. After a number of acquisitions, the derivatives trading exchange spent the last year building a high-speed network and setting up dual data centers, one in London, the other in Paris. The company’s “dual-site live” strategy means neither data center is a backup to the other. Rather, the two constantly share the load, although the entire business can run from just one, if necessary.

To accomplish dual-site live, the company’s IT arm, Euronext.liffe Market Solutions, replicates data between storage-area networks (SAN) located at the two data centers. While such an application might connote a synchronous link using optical technology, Market Solutions is instead using Fibre Channel over IP (FCIP), implemented on its four Cisco MDS 9509 SAN switches. FCIP supports an asynchronous connection over an IP network between Fibre Channel-based storage equipment, which effectively eliminates the distance limitation that has relegated synchronous replication to metropolitan-area applications. That makes FCIP a boon for any company looking to replicate data out of region.

“Any natural disaster or terrorist action is unlikely to lead to losing two data centers in two different countries,” says Mark Hemsley, managing director of Euronext.liffe Market Solutions.

FCIP isn’t the only technology for asynchronous replication between Fibre Channel-based systems. The Internet Fibre Channel Protocol (IFCP), touted mainly by Nishan Systems (which was recently acquired by storage switch vendor McData), accomplishes the same thing, albeit in a somewhat different fashion.

FCIP and IFCP are implemented in SAN switches that take in a Fibre Channel feed and encapsulate the data in IP. The other part of the equation is software that handles the replication, ensuring data is the same on both sides. All the big storage system vendors have some sort of remote copy software that works at the block or disk level, such as Hitachi Data Systems’ TrueCopy and EMC’s Symmetrix Remote Data Facility (SRDF) and the recently announced asynchronous version, SRDF/A. Software-only replication from NSI Software, Topio, XOsoft and Veritas Software works at the file-system level, looking for changes as they are written. These offerings work with standard Windows, Unix or Linux servers and require no special switches or transport protocols – just a plain old IP connection.

“The thing that FCIP and IFCP are built for is to have identical storage at both ends, where you mirror data from one site to another,” says Marc Farley, president of Building Storage, a storage consulting firm. Products such as NSI Software’s Double-Take, on the other hand, replicate data from one server to another, but the two databases might not necessarily be mirror images of one another in terms of data structure, and the data might not be instantaneously available. Yet customers who are using the products say the process works in near real time, certainly fast enough to provide for business continuity, if not instant access to data after an event.

Proof of concept

The idea of replicating data to a far-away back-up site has taken on new urgency with the heightened awareness of business resiliency requirements since the Sept. 11 terrorist attacks. The federal government also has made noise about requiring companies in some industries to back up data to an out-of-region location, although it has begged off implementing exact distance requirements until at least next year.

Vendor efforts to demonstrate the viability of IP as a transport for storage data over long distances date back at least two years. In September 2001, eight computer, storage and network companies staged the “Promontory Project,” in which they achieved speeds of 2.5G bit/sec in shipping storage data coast to coast over an IP link. Nishan provided the IP storage switching gear for the demonstration, using IFCP to package Fibre Channel data for the long-haul link. IFCP encapsulates Fibre Channel data just as FCIP does, but opens each frame to look at the source and destination addresses and provides a substitute address, much like a network address translation function, says Tom Clark, director of technical marketing at Nishan. That makes IFCP a routable protocol, he says, whereas FCIP merely establishes a tunnel across the IP network between two Fibre Channel-based endpoints.

A couple of years ago, a number of start-ups were working on IFCP-based products , but ran out of money before the market developed, Farley says.

That left Nishan as the only company still actively pushing the technology, while vendors including Brocade Communications, Cisco and McData – at least before the Nishan deal – backed FCIP.

“FCIP is the de facto standard,” agrees Steve Duplessie, senior analyst with Enterprise Storage Group. “But Nishan is probably winning the war as far as product sales go, so it’s kind of a moot point.”

Cisco in July announced its own proof-of-concept distance demonstration, along with Sprint and Hitachi Data Systems. The vendors established an OC-3 loop between Sprint labs in Burlingame, Calif., and Overland Park, Kan., a distance of about 1,800 miles. They attached Hitachi storage systems to each end and used Cisco MDS 9000 IP Storage Services modules (an add-on to the MDS 9000 SAN switch) to establish an FCIP connection between the systems. The vendors achieved near line-speed performance at the 1,800-mile distance and better than 85% of line speed for a 3,600-mile loop, says Audrey Harman, manager, technical staff, for Sprint.

Euronext.liffe’s Hemsley is using much the same setup to replicate databases of some 7.5 terabytes each in London and Paris. The company uses about 655M of its 2.5G-bit/sec link between the two sites for the replication application. Hitachi’s TrueCopy handles the replication chores while the Cisco gear implements FCIP. The setup replicates data with only 10ms of latency between the SANs on each end, he says.

“It’s extremely effective,” Hemsley says. “We can start applications in minutes out of a second data center. The SAN isn’t the limiting factor – the SAN speed is far faster than the human reaction time.”

No switch required

Companies that have more limited bandwidth budgets, or less demanding replication requirements, might find the software-only replication a better option.

While the specifics vary depending on the product, in general these tools ship only the changes, or deltas, from one server to the backup, as opposed to sending entire disk blocks. How much WAN bandwidth needed is a function of how much of your data changes and how frequently. “Any vendor worth its salt will have tools to record your traffic and know exactly what you need,” Duplessie says. Most also have a throttling mechanism, through which they essentially buffer data until WAN bandwidth is available – at the cost of immediacy.

PowerDsine, an Israeli company that makes products for shipping electrical current over Ethernet networks, uses XOsoft’s WANSync to replicate data from four servers at its headquarters in central Israel to a backup in Farmingdale, N.Y., on Long Island. The company uses no more than half a T-1 Internet connection for continuous back up of its Microsoft Exchange server and product data management (PDM) system, which holds engineering data. At night, the company also backs up a file server and data from its ERP system, says Yaron Rachmany, IT manager for the firm.

“The hardware solutions we looked at were $80,000 to $100,000 and up,” says Asaf Silberstin ,vice president of operations for PowerDsine. “Software solutions were $20,000 to $30,000, max.” The company selected XOsoft largely on the strength of its compression feature, which saves bandwidth, and its rewind capability. In the event of data corruption, the rewind feature lets PowerDsine return to a last known good state, and then it replicates corrected files.

At its Israeli headquarters, PowerDsine replicates data from four servers to one server running WANSync. Data from that server is then compressed – 50% on average – and replicated to a similar WANSync server on Long Island. In tests the company ran before moving to production mode, it got Exchange and PDM backup servers up and running “in a few seconds, or a few minutes in the worst scenario,” Rachmany says.

An alternative to tape

Covanta Energy uses a WANSync competitor, NSI Software’s Double-Take, for a different sort of replication requirement. The waste-to-energy firm operates about 45 power plants around the globe, each with 25 to 50 computer users. In 2002, the company was upgrading servers in 27 U.S.-based plants and needed to implement a backup strategy. It looked at traditional tape backup, but the price came to more than $240,000 to outfit each server with a tape drive and back up software, says Tony Sica, IT director for Covanta, based in Fairfield, N.J.

Instead, the company decided to run Double-Take on each server it needs to back up and replicate the data in a many-to-one configuration to one of two target servers in the firm’s Fairfield data center. The company backs up Exchange servers at each site as well as data from its continuous emission monitoring system, to meet Environmental Protection Agency requirements. That system only generates about 3G bytes of data per year, but each Exchange server houses about 30G bytes of data, with daily changes running around 10%.

“[Double-Take] works when your systems change less than 20% daily,” Sica says. “If we needed to replicate the whole 30G each day, it’s not going to be a good fit.”

As is, the back-up application requires only about 64K bit/sec of bandwidth on the firm’s frame relay network, which has 512K bit/sec links to most locations. “Changes are instantaneously updated to New Jersey. We don’t have to go to last night’s backup if we lose a server in the middle of the afternoon,” he says. If a disaster destroys an entire server, Sica says he could configure a new one with data from the New Jersey site, put the server on a plane, “and they’ll have it four hours later.” The data center servers are backed up nightly to tape, which is stored off-site.

At a cost of about $2,000 per server, the Double-Take implementation saved Covanta more than $200,000 vs. the tape-backup route.

The upshot is providing real business continuity by replicating data out of region is no longer an exercise only for those with the largest IT budgets.

“We were very impressed with the technology,” PowerDsine’s Rachmany says. “When we started our project, we only dreamed of results this good.”

Desmond is a freelance writer in Framingham, Mass. He can be reached at paul@pdedit.com .