CSI: Lost e-mails

Forensic experts sift through electronic data looking for key bits of evidence.

Your CIO calls. He just had a visit from the CEO, who just met with the company lawyer. Maybe an ex-employee is suspected of stealing trade secrets. Maybe a sexual-harassment suit has just been filed against the company. Maybe there’s a Securities and Exchange Commission or even an FBI investigation.

In any case, what the CIO wants is e-mail. Not just any old e-mail but e-mail going back five years from two-dozen end users, some of whom no longer work at the company.

Retention is key

And you just can’t dump terabytes of raw data on the CIO’s desk. You need to sort the e-mails by cross-referencing them against a list of 25 keywords. And you have to weed out all the cc’s and other forms of duplication.

Oh, and you need to produce hard copies of the e-mails in 14 days, no ifs, ands or buts.

If this scenario occurred at your company, would you be prepared to handle it with ease or would you be headed for a serious meltdown?

In today’s world of rampant litigation and regulation, even if you’re not an Enron or a WorldCom, even if your company is squeaky clean, it’s wise to assume that sooner or later, you will be compelled to produce e-mail records.

In fact, Renew Data, an Austin, Texas, computer forensics company, met with more than 100 of the Fortune 500 this year and found that each company is facing an average of 125 non-frivolous lawsuits at any given time. In any sizeable corporation, “A lawsuit is not an event, it’s a process,” says Renew Data CEO Bob Gomes.

In addition to lawsuits, regulations can result in requests for e-mails and other documents. Financial-services firms are watched closely by the SEC. Companies involved with the healthcare industry, even tangentially, must comply with the massive Health Insurance Portability and Accountability Act. Inquiries, requests for information and penalties can come from any number of state, industry and federal bodies.

The bottom line is if you haven’t recently overhauled your policies and procedures for saving e-mail, now’s the time.

Lawyering up

For the past decade, IT managers have worked hard to strengthen lines of communication with the business side of the company. Experts say it’s now time for IT to extend that effort to the company’s legal department.

“IT should not and cannot dictate the policy” on saving data, says Ray Paquet, an analyst at Gartner. “This is fundamentally a legal issue, especially in publicly traded or highly regulated companies.”

When meeting with your employer’s general counsel, he says, IT should “make it clear that you’ll do whatever he needs done – but that nothing in life is free. Increased data retention costs money.”

“Whether companies save too little information or too much, we find that they’re often surprised by their data retention,” says Simon Platt, leader of the computer forensics practice at Deloitte & Touche in New York. According to Platt, it is IT’s job to eliminate this surprise factor. “There’s got to be communication between the CIO and general counsel. IT can help [the legal department] understand how much e-mail they’re retaining, how much of it can be realistically accessed and how expensive that will be. General counsel can help decide how much [e-mail] the company should be retaining, then set and enforce policies.”

Your decision on how and how long to save e-mail should merely be part of an overall data-retention policy. IT, legal counsel and top business executives should formulate this policy, Paquet says, after considering industry regulations and legal precedent. For example, the SEC dictates that brokerages must be able to produce three years’ worth of records “immediately” upon request.

So how long should you hang on to old e-mail? There’s no concrete answer, but the ever-declining cost of storage is driving many companies to save e-mail for long time periods, even if regulators don’t force them to.

Solvay Pharmaceuticals, an Atlanta subsidiary of Solvay S.A. in Brussels, Belgium, recently needed to quickly find a year’s worth of e-mail messages from certain employees regarding certain topics (Solvay is reluctant to discuss the specifics of the case). The problem was complicated because Solvay uses a third party to manage its Microsoft Exchange servers.

After evaluating various retrieval options, the company turned to Renew Data. Bruce McMillan, Solvay’s manager of emerging technologies, says the vendor “was able to search through the e-mail month-by-month, capture it, remove duplications [such as cc’s] and put it in any format we needed” more quickly than Solvay’s internal IT staffers could have.

After that experience, Solvay changed its data-retention procedures, McMillan says. “We used to recycle tapes after a certain period of time. Not anymore – we’re saving everything.”

On the witness stand

Many businesses turn to e-mail-recovery services for their expertise. “Computer evidence is so fragile – it can be lost or corrupted so easily,” says W. Reid Wittliff, an attorney at Graves, Dougherty, Hearon & Moody in Austin, Texas. The firm used Renew Data in a trade-secret case. Although the case was settled before it went to court, “We needed someone who could find the evidence, capture and preserve it so it could be effectively used in court, and defend against attacks” by opposing attorneys, Wittliff says.

The latter point is important, and often overlooked. IT employees on the witness stand might be turned inside out by a savvy lawyer, who will make every effort to imply that they might have tampered with the evidence out of loyalty to their employer.

“You need someone who can testify and be neutral and professional about what they’ve done – and can explain it so a judge or jury can understand,” says Wittliff, who was a computer-crimes prosecutor before joining Graves, Dougherty. (Attorneys similarly can attack firms like Renew Data on the basis of who’s paying their bills, but their third-party status tends to make them solid witnesses, analysts say.)

While many consider e-mail retention as a defensive need, it is just as likely to help businesses prove wrongdoing by others, such as when former employees are shown to have stolen proprietary data. Michelle Lang, an attorney with forensics firm Kroll Ontrack Data Recovery, recalls a case in which a secretary claimed she’d been sexually harassed by her boss, and produced filthy e-mail messages that “proved” her case. Using forensic techniques, “We found she was using his account and sent all the e-mails herself,” Lang says.

Searching for byte marks

So how does a search for e-mail work? The process varies, but typically a company’s outside counsel receives a subpoena or discovery order from some source – a plaintiff’s attorney, for example, or a regulatory agency such as the SEC. Outside counsel meets with the company’s head of litigation. Eventually, general counsel meets with the CIO and explains what e-mail must be produced.

The parameters vary by case. A typical large-scale corporate civil suit might require all e-mail (and user files) from the past five years for 50 employees responsive to 30 keywords.

Then the CIO calls the IT department and relays the request – and asks how long it’ll take to pull together the information.

Which is when IT has a nervous breakdown. “It’s a dirty little secret that it’s hard to get information off back-up tapes,” Renew Data’s Gomes says. Even if a corporation never recycles tapes and labels them scrupulously, it might have 10,000 tapes to go through, with untold terabytes of data. To pull the needed information, IT must recreate the native environment as it was when the backup occurred – the proper version of the backup software, operating systems, applications, with appropriate patches, passwords, and so on. In a typical business, this can be nearly impossible.

Many businesses, facing time pressure and the difficulty of the task, call Renew Data or competitors such as Data Recovery Group and 911 Forensic Data. Such services use proprietary techniques to recover e-mail without restoring the native environment. Their trick is that they’ve backward-engineered popular back-up software such as from Legato Systems, and are familiar with all the widely used data structures.

Where the data extraction is performed is determined partly by the circumstances of the case. Renew Data recently worked on a case for “one of the largest banks in the U.S.,” Gomes says (declining to be more specific). The bank didn’t want its back-up tapes to leave its facilities. This is typical, and sensible as well – data extraction typically requires only a tape drive, so it’s easy to do on customer premises.

There are two critical factors in the extraction that might be unfamiliar to IT professionals. First, when working with hard drives as opposed to back-up tapes (a common occurrence because many users save some of their e-mail to their own PC), specialists don’t merely copy the drive – they create a byte-for-byte forensic image of it.

It’s also important to start a chain of custody so that any evidence gathered will stand up in court. “If we produce something, we need to be able to say exactly where it came from under very aggressive cross-examination,” says Kevin Bluml, a forensics expert at Kroll Ontrack. “Creating a chain of custody means tracking every single place a piece of media has been, who’s touched it, and so on.” This demand for precision is one reason experts discourage “fishing expeditions.” Generally, the actual data retrieval is performed at the vendor company’s site.

Protect yourself

Today, it’s critical that companies form, implement and enforce a data retention policy with e-mail as its centerpiece. “E-mail messages are critical business records,” Massachusetts Secretary of State William Galvin recently told The Boston Globe when his agency fined SG Cowen. “The fact that [e-mail messages] are a little more casual than traditional business records makes them no less significant.”