Publically available data that could be aggregated and used by intelligent systems to predict future events is out there, if you can harness the technology to utilize it.
That's one of the driving ideas behind a program that the Intelligence Advanced Research Projects Activity (IARPA) group will detail at a Proposer's Day conference in Washington, DC next month.
More interesting stories: The weirdest, wackiest and stupidest sci/tech stories of 2010
The program, known as the Open Source Indicators (OSI) will aim to "develop methods for continuous, automated analysis of publicly available data in order to anticipate and/or detect societal disruptions, such as political crises, disease outbreaks, economic instability, resource shortages, and natural disasters," IARPA stated.
According to the agency: "Many significant societal events are preceded and/or followed by population-level changes in communication, consumption, and movement. Some of these changes may be indirectly observable from publicly available data, such as web search trends, blogs, microblogs, internet traffic, webcams, financial markets, and many others. Published research has found that many of these data sources are individually useful in the early detection of events such as disease outbreaks and macroeconomic trends. However, little research has examined the value of combinations of data from diverse sources."
"The OSI Program will aim to develop methods that "beat the news" by fusing early indicators of events from multiple data sources and types. Anticipated innovations include: development of empirically-driven sociological models for population behavior change in anticipation of, and response to, events of interest; collection and processing of publicly available data that represent those population behavior changes; development of data extraction techniques that focus on volume, rather than depth, by identifying shallow features of data that correlate with events..." IARPA stated.
According to IARPA, OSI will not fund research on US events, the identification or movement of specific individuals, collection mechanisms that require directed participation by individuals, or advanced natural language processing. It is expected that performers will use existing, off-the-shelf technologies to extract features of interest in publicly available data, and that research will focus on methods for correlating combinations of data with events, the group stated. "Collaborative efforts and teaming among potential performers will be encouraged. It is anticipated that teams will be multidisciplinary, and might include social scientists, mathematicians, statisticians, computer scientists, content extraction experts, and information theorists. IARPA anticipates that academic institutions and companies from around the world will participate in this program. Researchers will be encouraged to publish their findings in academic journals."
IARPA has a number of interesting ongoing operations., You may recall that the agency in May said it wanted to build a repository of metaphors. Not just American/English metaphors mind you but those of Iranian Farsi, Mexican Spanish and Russian speakers.
In the end the program should produce a methodology, tools and techniques together with a prototype system that will identify metaphors that provide insight into cultural beliefs. It should also help build structured framework that organizes the metaphors associated with the various dimensions of an analytic problem and build a metaphor repository where all metaphors and related information are captured for future reference and access, IARPA stated.
IARPA also runs the Automated Low-Level Analysis and Description of Diverse Intelligence Video (ALADDIN) program which looks to build and analyze what it calls open source video clips.
Follow Michael Cooney on Twitter: nwwlayer8
Layer 8 Extra
Check out these other hot stories: