- 15 Non-Certified IT Skills Growing in Demand
- How 19 Tech Titans Target Healthcare
- Twitter Suffering From Growing Pains (and Facebook Comparisons)
- Agile Comes to Data Integration
CSO - Big Data is viewed as a very good thing by most enterprises. With the right analytics, it can generate meaning and business value. But like with many things there can be too much of a good thing, say a number of Information Governance (IG) experts.
Their message is that enterprises need to do more than protect their data from theft or infection -- they need to get rid of some of it, for both economic and legal reasons.
Dumping data has a variety of names, so far, including defensible disposition, defensible deletion and active expiration. Barry Murphy, cofounder and principal analyst at eDJ Group, prefers defensible deletion (DD).
What is more important than the label, Murphy wrote in a post in eDiscovery Journal, however: "Companies can reduce costs and decrease risks by proactively getting rid of unnecessary information."
Murphy told CSO Online that it is true that the cost of storage, both on-premise and in the cloud, continues to decrease. "One could argue that the decreasing cost of storage combined with lower-cost information processing platforms like Hadoop makes keeping information in perpetuity economically viable," he said. "But the rate at which information grows is faster than the rate at which the cost of storage decreases. So much corporate information is either duplicate or unnecessary that the cost of retaining it is greater than that of getting rid of it."
Jim McGann, vice president of marketing for Index Engines, said in an interview with Government Technology last year that in the past five years he had seen organizations taking steps to "clean up the 'data lake' that has been generated."
[See also: The security risks and rewards of Big Data]
The motivation is legal as well as economic, he said. Until about 15 years ago, organizations could save anything and easily hide the content that could become a liability, bu he saidt that won't work these days. "Lawyers and judges are more tech savvy and they won't accept excuses about complexity and cost issues anymore," he said.
Barry Murphy agrees. "The cost and risk of eDiscovery can poke a giant hole in any economic assessment of information management costs," he said.
The rules governing electronic information are different than those for paper documents, since it usually includes metadata, which can be important as evidence. An example is the value of the date and time a document was written to a copyright case.
This doesn't mean a company can get rid of any electronic documents it fears might create a liability. But Murphy said federal Rules of Civil Procedure give companies a so-called "safe harbor" from liability for information deleted in accordance with standard operating procedures, "as long as a legal hold process is in place to stop deletion if information may be relevant to a litigation or regulatory matter."
Murphy said that in general, "any information assets that are duplicate or have no business value would fall into the pile of 'to be deleted.'" But he said too many organizations are not yet "mature" enough to put an accurate value on information. Instead, he said, they have "time-based retention policies."