Bulk data collection most effective for intelligence gathering

Federal study says more targeted approaches could save some privacy headaches though

rtr4ga63
Credit: Reuters

When it comes to grabbing-up as much information as possible, there’s no substitute for bulk collection techniques – but isolating collected data, restricting queries that can be made against those data, and auditing data usage could help to enforce privacy safeguards and calm some civil liberty concerns, a report issued last week by the National Research Council stated. The study was sponsored by the Office of the Director for National Intelligence.

The report stemmed from request last year from President Obama that the intelligence community look into ways they could gather information on terrorist or other nefarious activities without just gathering across-the-board data on everyone. The request came on the heels of the investigation around former intelligence contractor Edward Snowden and his revelations about the vast intelligence-gathering program of the National Security Agency.

+More on Network World: 26 of the craziest and scariest things the TSA has found on travelers+

“Improving the relevance of collected information to future investigations could also be achieved with new approaches to targeting, the report says.  Rapidly updating filtering criteria to include new targets as they are discovered will help collect data that would otherwise be lost, and if done quickly enough and well enough, bulk information about past events may not be needed.  However, targeted collection cannot substitute for bulk collection if past events were unique or if the delay in collecting the new information is too long.”

The report defines “collection” as the process of extracting data from a source, filtering it according to some criteria, and storing the results.  If a significant portion of the collected data is not associated with current targets or subjects of interest in an investigation, it is considered bulk; otherwise, it is targeted, the report stated.

As an alternative to controlling the collection of data, automated controls on the use of collected data can help to protect the privacy of people who are not subjects of investigation, the repot stated.

“Three key technical elements are required to control and automate usage: isolating bulk data so that it can be accessed only in specific ways; restricting the types of queries that can be made against stored data; and auditing the queries that have been done.  The way these controls work can be made public without revealing sensitive data, so that outside inspectors can verify that the intelligence community has and abides by adequate procedures to protect privacy.”

While some of the necessary technologies to enhance targeted collection or improve automated usage controls require further research and development, some of the techniques are already in use in the intelligence community or in private companies, some have been demonstrated in research laboratories, and many are feasible to deploy within the next five years, the report says, although it does not recommend adoption of any specific technology.

“From a technological standpoint, curtailing bulk data collection means analysts will be deprived of some information,” said committee chairman Robert F. Sproull, former director of Oracle’s Sun Labs in a statement. “It does not necessarily mean that current bulk collection must continue.  A reduction in bulk collection can be partially mitigated by improving targeted collection, and technologies can improve oversight and transparency and help reduce the conflict between collection and privacy.”

 Check out these other hot stories:

Virgin Galactic wants to launch 2,400 comm. satellites to offer broadband to everyone

What advanced tech will dominate your car by 2025? IBM knows

Quick look inside IBM’s snazzy new mainframe

NTSB: Distracted driving among Top 10 transportation safety challenges

3D-printed car-maker targets mass distribution

Spaceshot: 3,200-megapixel camera for powerful cosmos telescope moves forward

DARPA takes aim at energy conversion technology

IBM: Retail attacks down, but when they hit cyber-attackers get more

FBI wants you to become a cyber agent

Join the Network World communities on Facebook and LinkedIn to comment on topics that are top of mind.
Must read: Hidden Cause of Slow Internet and how to fix it
Notice to our Readers
We're now using social media to take your comments and feedback. Learn more about this here.