Digital evidence comes in all shapes and sizes: pallets full of computers, a hard drive with an AK-47 bullet hole in it, audio tapes fished out of the ocean, mangled floppies, garbled 911 calls.

Whenever U.S. government agencies investigating a crime or a cybercrime has digital evidence that's too difficult to analyze, they send it to the Department of Defense computer forensics lab.

The evidence can arrive in a military vehicle, via FedEx or through the U.S. Postal Service. However it gets there, it's accepted at the loading dock of an unmarked commercial building on the outskirts of Baltimore.

It's then logged and sent to an evidence custodian, who inventories, tags and stores it in a locked cage.

Network World was invited into the Defense Computer Forensics Lab (DCFL) for an inside look at how computer investigators at the cutting edge are using digital evidence to help solve crimes.

The purpose of the lab is to analyze evidence gathered at crime scenes involving the military. Whatever crimes occur in the civilian world, you also see in the military. It could be homicide, child pornography, identity theft, counterfeiting, misconduct, terrorism, espionage, contractor fraud or misuse of government property.

With these crimes, there's often digital evidence in cell phones, pagers, PDAs, geo-mapping systems, digital cameras, cockpit recording systems and anything else with flash memory or ROM.

"We estimate that 95% of criminals leave digital evidence at the scene," says Donald Flynn, attorney adviser for the Defense Department Cyber Crime Center, which houses the DCFL.

That evidence must be able to stand up in court, particularly now that judges and attorneys are becoming savvy enough to start asking questions about the integrity of digital evidence. The DCFL addresses this through rigorous training and advanced tools such as certified, high-capacity extraction and imaging processes and tools.

Inside the lab

My tour guide at the high-security lab pushed a button at the double-door entryway into the lab that triggered blue ceiling lights, which blinked incessantly to alert technicians that unclassified visitors were on the premises.

The lab includes your standard office cubicles, but every cube is outfitted with state-of-the-art processors, multi-system server stacks and 42-inch flat-screen monitors.

"Some of the evidence comes in on pallets - cases full of servers, CPUs, RAID disk arrays, floppy diskettes, Palm Pilots, digital cameras," says special agent Bob Renko, director of operations for the lab. "We've even gotten evidence in buckets of water - for example, video tapes recovered from jets crashing into the sea during training exercises."

The first stage in evidence extraction is digital imaging. This is trickier than it sounds because contents can be altered in the process - such as adding a date stamp when copying a hard drive, thus tainting the evidence and rendering it inadmissible.

Then there's the sheer volume of data. In 1999, analysts examined their first terabyte-sized case when they received a palette of computers belonging to a defense contractor accused of violating Environmental Protection Agency guidelines in its handling of toxic waste. If analysts had tried to use technology that copied and examined one drive at a time, they still would be investigating that case, says the lab's director, Lt. Col. Ken Zatyko, special agent with the Air Force Office of Special Investigation.

The busted boyfriend A suspect said the gun used in the murder of his girlfriend was stolen earlier that day from his car by someone who smashed his right passenger car window. A video technician at the Department of Defense lab painstakingly upgraded a grainy image of the suspect’s car taken from a military surveillance camera three hours after he claims the window was broken. Finally, she enhanced the image to provide the damning evidence against the suspect: light refracting off an intact, passenger-side window. The suspect got a 25-year sentence.

Click to see:

So analysts created their own script, which moves images of all the media into one place. In this location, searching and extraction is conducted across all the data simultaneously using the same search phrase.

Last month,the lab received several palettes, containing more than 3T bytes of data to image and extract. The evidence, which filled a 20-by-10-foot windowless room, required its own storage-area network .

The recovery process begins with entry-level technicians checking evidence out of lockup. Then they create bit-stream mirror images onto cleaned hard drives to prevent contamination.

They make the copies using a modified Linux  tool dubbed DCFL Data Dump. The tool is akin to private-sector imaging tools such as SafeBack, which takes a mathematical hash of the image and compares it to the original hash to prove the image is an exact replica.

Whitepapers

• The Trend from UNIX to Linux in SAP(r) Data Centers
• Getting in Compliance With Government Data Regulations By Leveraging Online Security Technology
• How to Offer the Strongest SSL Encryption
• Maximizing Site Visitor Trust Using Extended Validation SSL
• Seven Steps to Reducing your Security Risk
• The Latest Advancements in SSL Technology

View more SECURITY whitepapers

Webcasts

• The Secure Web Gateway. Mission Critical For Business - Webcast
• Mobile Data Security
• Best Practices for Deploying WAN Optimization for Disaster Recovery
• Key Considerations for a Successful 802.11n Deployment
• Solutions to the Toughest IT Challenges in Remote Offices
• Lowering the Cost of Email Archives

View more SECURITY webcasts

Special Reports

• The Evolution of Network Security
• Taking Virtualization Up a Notch
• The self-managed network

View more SECURITY special reports