Why IT should start throwing data away

It can be a storage nightmare: Given expanding regulatory requirements and the key role that electronic records now play in lawsuits, some enterprises are saving every bit of data they have, just to be safe. As a gauge of storage demand, IDC says the total amount of disk storage shipped last year grew 40.5 percent from 2007.

Sure, storage media are getting less expensive. The cost of a gigabyte of disk storage fell more than 27 percent from 2007 to 2008, according to IDC. But with the storage requirements of average enterprises rapidly growing, keeping it all forever can create long-term management challenges and lead to headaches when something needs to be found. Analysts, attorneys, and vendors say enterprises are better off getting rid of some data -- but doing it judiciously. For IT departments, that means planning, carefully executing, and not going it alone.

"More companies are sensitive to the fact that we can't just keep throwing storage at the issue," says IDC analyst Rick Villars. Letting fast-growing data stores build up year after year isn't sustainable, he says. Companies that save everything are often those that aren't sure what needs to be saved or deleted, he adds -- a sign that they're setting themselves up for trouble.

On top of the price of disks, tapes, networks and management, piling up too much data can come back to haunt a company in case of a lawsuit. It may cost $1 million to find and compile the data requested in "e-discovery," the process of collecting electronic data as evidence, says Andrew Cohen, vice president of e-discovery and compliance at EMC.

Those who've studied this dilemma recommend a variety of steps to make sure that a company is neither saving too much nor endangering itself by improperly deleting information. There are technologies out there to help, but it also takes human input, they note.

The data-storage problem Every organization needs to save information for its own purposes, such as institutional memory, transaction lookup and analysis, and so on. Plus, regulations such as the Sarbanes-Oxley Act and Health Insurance Portability and Accountability Act (HIPAA) require enterprises to save certain kinds of content for a prescribed period. And more such regulations are in the works, notes Enterprise Strategy Group analyst Brian Babineau. "If you're in business, you're going to be regulated somehow," he says.

But storing information sets you up for a risk: E-discovery requests in lawsuits expose a company's data and data management to close scrutiny. The more you store, the more they can ask for, increasing the odds of damaging findings. But if your information storage and deletion policies aren't rigorous and consistent, you can lose the benefit of doubt in court as to why some information is not available.

In some cases, regulations tell you what you must do, but often they tell you just the minimum you must do or, in the case of the Federal Rules for Civil Procedure, provide guidelines how to make storage decisions but not prescribe them.

The IDG News Service is a Network World affiliate.