Report: Data warehouse failures commonplace

IT executives implementing data warehousing and business intelligence applications expect a failure in four of every 10 projects, a recently released study says.

A survey released by the Cutter Consortium, an IT analysis firm, says as many as 41% of data warehousing projects fail because they don’t meet the business objectives of the company or because they ignore what users really need out of a data warehousing application.

A data warehouse consolidates data and information from multiple corporatewide departments. The idea is to ease resource and analysis by data mining, business intelligence, CRM or decision-support systems such as those from SAS or Cognos.

“There is no one reason for data warehousing project failure,” says Curt Hall, an analyst for the Cutter Consortium. “You see a culmination of things – customers don’t meet end-user expectations, they don’t have a clear view of the business problem they want to use the data warehouse for, and there are issues where the quality of the data is debatable.”

Using the information in a data warehouse effectively is big business – IDC estimates that revenue from servicing CRM applications will increase at a compound annual growth rate of 25% from $61 billion in 2001 to $148 billion in 2005.

Hall says that companies are uneasy about implementing data warehousing technology – only 25% of the customers surveyed said they felt comfortable with data warehousing and 38% of the companies have brought in outside consultants to steer a stalled project back on track.

Philip Holland, a SAS technical consultant with Holland Numerics Limited, a consultancy in Royston, England, is one of those consultants. He says that the continual addition of features is the biggest cause of project failures.

“The scope of the project progressively increases over time,” he says. “All the data warehousing projects I have worked on have missed deadlines due to ‘scope creep.’ The projects generally appear to have clear goals, but secondary goals tend to be added by senior management as new ideas emerge.”

IT managers and consultants say companies can avoid problems in data warehousing projects by following several easy-to-implement rules.

“A common failure is that end users really don’t buy in and participate to the degree required, says Michael Davis of Basset Consulting Services. “IT folks know the technology but don’t always know the business processes. The problem can be minimized if senior management buys in and frees the key end users to work on the data warehousing projects.”