Government open data proves a treasure trove for savvy businesses

Hoping to capitalize on free government information, IT leaders are discovering the value -- and vexation -- of converting terabytes of data into new revenue streams.

1 2 Page 2
Page 2 of 2

Despite privacy concerns and questions about the security of open data, there's no shortage of organizations, from government entities to nonprofits, pushing for greater access to public information. Here are a few examples:

aC/ The home of the U.S. government's open data.

aC/ Open Institute: A Kenya-based think tank of domain experts that provides open data advisory services.

aC/ Open Knowledge Foundation: A Cambridge, England-based nonprofit organization dedicated to promoting open data.

aC/ Socrata: A provider of open data and government performance products that are designed to make data more accessible and easy to use.

aC/ The GovLab: A New York University research center that seeks new ways to solve public problems using advances in technology and science.

aC/ Sunlight Foundation: A nonprofit organization that provides tools, policies and recommendations to drive greater government openness and transparency.

- Cindy Waxer

A team of nearly 15 data analysts processes more than 60,000 retirement plans per year while integrating data captured from SEC filings, financial industry regulatory authorities, census data and public websites. Once collected, the data is then cleansed and linked together using proprietary mapping algorithms.

"There's a big difference between free open data and actionable intelligence," says Alfred. "Companies don't want to purchase data that requires data analysts and infrastructure and integration. So there's a ton of work on both the manual side and the engineering side to get the data into a format that can be reliably used by our clients." That's a complex process that, he says, could be simplified with a bit of help from the government.

The good news is that the government is taking steps to improve data quality. Calcbench has benefited from that effort. The New York-based startup has created a sophisticated engine that turns complex financial data such as earnings statements, cash flow statements and companies' balance sheets into a more readable format. To ensure accuracy, Calcbench uses proprietary artificial intelligence tools that sift through the data to detect errors such as misdated year-end reports. Finance professionals such as investors, auditors and industry researchers use the resulting repackaged and cleansed data to compare financial ratios, examine entire industries and review competitors' disclosure data.

"A lot of what we're charging for isn't the data," says Alex Rapp, Calcbench's CTO and co-founder. "It's how we structure it, how we store it for you and how we solve a tremendous number of business problems by making data comparable across different companies and industries. That's something the data doesn't do itself."

A lot of what we're charging for isn't the data. It's how we structure it, how we store it for you and how we solve a tremendous number of business problems by making data comparable across different companies and industries. That's something the data doesn't do itself. Alex Rapp, CTO and co-founder, Calcbench

Nor is it a service that would have been possible without the U.S. government's insistence, starting in 2009, that the financial information in corporate SEC filings must be in XBRL (Extensible Business Reporting Language) -- a freely available and global standard format for exchanging business information. Whether you're downloading quarterly filings directly into spreadsheets or analyzing year-end statements using off-the-shelf software, XBRL converts fragmented financial data into a machine-readable format. And that makes life easier for Calcbench's techies.

But as government agencies inch toward cleaner, more structured data sets, a handful of startups are offering powerful new tools for dealing with older data. "These intermediaries are going to make open data more accessible to other types of companies because the reality is, open data is still pretty messy at the local, state and federal level," says Gurin.

One of those startups is New York-based Enigma Technologies, which has created a platform that allows users to access open data via an API or a Web dashboard. The company uses Web crawlers that gather public data from sources such as dot-gov websites and FCC documents regardless of whether the information is stored in a Zip file, an Excel spreadsheet or a multigigabyte database. Next, the platform creates a relational infrastructure that centralizes all of the open data and creates links between matching data sets. A single interface offers users access to more than 100,000 sources of finely curated data. And because Enigma's solution is offered as an API, developers can borrow the technology to build dashboards that serve their industry-specific data retrieval needs.

"We're able to provide access to this universe of public data that is otherwise totally obscure and in the shadow," says Enigma co-founder Marc DaCosta.

In fact, there are plenty of ways for IT departments to tap the business value of open data. Some will opt to use tools from companies like Enigma. Others will staff their IT departments with Ph.D.s, develop sophisticated algorithms and build robust storage networks. And some will wait until government protocols catch up with corporate demands. Whatever the game plan, says Gurin: "Open data may not be the way for everybody, but it is the way forward-thinking CIOs are thinking about business."

Waxer is a Toronto-based freelance journalist. She has written articles for various publications and news sites, including The Economist, MIT Technology Review and

Read more about business intelligence/analytics in Computerworld's Business Intelligence/Analytics Topic Center.

This story, "Government open data proves a treasure trove for savvy businesses" was originally published by Computerworld.

Join the Network World communities on Facebook and LinkedIn to comment on topics that are top of mind.

Copyright © 2014 IDG Communications, Inc.

1 2 Page 2
Page 2 of 2