At almost every conference I go to, I get asked, “How long should I keep documents, e-mail and other records?”

Document retention is one of the leading drivers of the growth of storage. Most companies are facing growth that exceeds 20% a year. And although disk is getting cheaper, storage administrators are getting more expensive. So how do we balance the needs of regulatory compliance and litigation with the rising cost of retaining electronic records? You won’t like the answer, but it seems that the best approach is to try to retain documents forever.

When I asked participants at a recent security-research benchmark what their retention policies were, more than a quarter said they keep records forever. Why? These folks decided the risks of not having information that might someday be asked for in court outweighed the costs of retaining data permanently — a perspective that’s increasingly valid. Another quarter said, “it varies.” In this case, the time frame varied according to the kind of information being retained. Sometimes the time frame was based on legal requirements and sometimes it wasn’t; and in some cases time frames were reviewed regularly, but most weren’t. The remaining participants retained records for various fixed periods, typically seven to 10 years, or as long as the law required (and often a few years more).

Outside of such heavily regulated industries as financial services, the main driver for retention is litigation. Electronic-discovery rules, recently updated by the federal courts, require companies to take reasonable measures to produce electronic records deemed relevant to litigation. Many executives have decided that deleting records regularly might be a better approach: Less to find means less costly discovery and fewer surprises. I see two problems with that approach.

First, the other party in the litigation may end up with better evidence because you have destroyed all of yours. Imagine a lawsuit, for example, where one party has retained all the evidence that supports its position, while the other has destroyed all evidence — including that which could be used as a defense! Second, companies with short-term retention policies have to enforce them through deliberate and consistent record-destruction. If records linger past the official retention period, a company could find discovery even more costly. Judges could frown on a company that has claimed everything is destroyed, only to have partial evidence surface after it has searched more carefully.