SalesForce outages show SaaS customers largely at "mercy" of providers

SalesForce's two recent outages point to difficulties in preparing for SaaS outages compared to IaaS, analysts say

Software as a service (SaaS) applications are generally more difficult to prepare disaster recovery plans for compared to infrastructure as a service (IaaS) offerings, analyst and consultants say. And given the recent example of two outages in as many weeks, it's a topic they say customers don't generally pay enough attention to.

SalesForce is the poster child of the SaaS cloud, with its CRM and platform as a service (PaaS) offering named Heroku and On the morning of June 28 a newly discovered software bug caused what the company called a "rare dual failure" in both its storage tier and secondary active standby storage tier, resulting in shared memory corruption and some customers not being able to access their SalesForce systems for as long as five hours. On July 10, manual upgrades at a West Coast data center the company rents space in resulted in a power failure, knocking out service to some customers for two days.

SaaS SLIDESHOW: 10 SaaS Delivery Companies To Watch 

IaaS SLIDESHOW: 10 Most Powerful IaaS Companies 

IMPRESSIVE: 15 amazing tech feats and world records 

Outages on the IaaS side, from providers such as Amazon Web Services, are occasional and there are a variety of tips for how customers can prepare for them.

But John Morency, a disaster recovery analyst at Gartner, says there are fundamental differences between SaaS and IaaS applications that makes apps like SalesForce much harder to create a disaster-recovery plan around compared to Amazon.

"In SaaS, the provider is the keeper of the entire application stack," he says. "Management, backup, recovery, you're really at their mercy."

In a SaaS environment, he says, the provider - in this case - manages almost the entire system for its customers, from the infrastructure all the way to the application. That means customers have less ability to tinker with the application and the underlying infrastructure it runs on to make it more highly available. In an IaaS environment, the provider is just supplying the underlying compute or storage hardware while the customer is responsible, in most cases, for the applications running on the infrastructure.

So what are SaaS customers to do?

A basic rule of thumb is for customers to do their homework, Morency says. Know what your SaaS provider's service-level agreement (SLA) ensures for uptime and make determinations of how tolerant your organization would be of downtime. "Before making contractual commitments to any SaaS provider, ensure that you have a complete understanding of the recovery management procedures and managed service levels," he says.

If downtime is not an option, there is a growing industry around recovery as a service (Raas), which are cloud-based systems that provide backups of virtual machines, data and applications. There are more than 50 providers in this space now, Morency says, ranging from big-name IaaS providers such as Amazon, Microsoft, Terremark and Sunguard, to other players such as Bluelock, and HP, along with others. These providers work with customers to create customized backup solutions, mostly on the IaaS side, but sometimes on the SaaS side as well, Morency says.

Robert Mahowlad, director of SaaS research at IDC, says it's difficult to insulate your SaaS environments from downtime compared to what IaaS users can do.

SalesForce, he says, is basically a large database of indices holding customer data. Using the CRM is basically the equivalent of making queries within the database and the application spitting the results back out. Unless a SalesForce customer backs up all of their data on their own premise, within SalesForce or with another provider, when SalesForce is down, access to that customer database can also be down. "Amazon has backup options within the same region, within various regions of the country, perhaps even with multiple cloud providers," he says. "You could conceivably have four or five redundancy levels on the IaaS side. On the application side, that's a little more difficult to do."

There are some steps that experienced SalesForce users can take, however, says Eric Hartye, an independent SalesForce consultant who helps businesses design applications running in SalesForce's PaaS offering. SalesForce offers developer sandboxes, which are meant to be a place for coders to test the application and for organizations to train employees on how to use it, without impacting the "production" data, which is the live version of the database. In a downtime event, Hartye says the sandbox could theoretically be used by a customer to access their SalesForce customer records and data, if the sandbox has been configured to be a full copy of the production workload.

The Enterprise Edition of SalesForce comes with one developer sandbox with some memory limitations, while the unlimited version of the CRM application comes with multiple sandboxes. Hartye says he hasn't seen the sandbox approach used, but in practice it could work. Overall, he says customers aren't typically thinking enough about disaster recovery. "DR doesn't make money," he says. Normally customers just want to get up and running, and they don't think as much about what happens when the system goes down.

A SalesForce spokesperson said in an e-mail it did not have any comments regarding the outage, other than pointing users to, where the company makes public statements related to the outages, and it explains in more detail the circumstances and next steps related to the two recent incidents.

Network World staff writer Brandon Butler covers cloud computing and social collaboration. He can be reached at and found on Twitter at @BButlerNWW.

Join the Network World communities on Facebook and LinkedIn to comment on topics that are top of mind.

Copyright © 2012 IDG Communications, Inc.