In 2008 the FBI's Criminal Justice Information Services division (CJIS) embarked on an ambitious effort to enable information sharing among every federal, state, tribal and local law enforcement agency in the United States. It launched the National Data Exchange (N-DEx), an $85 million data warehouse project, and waited for the data to roll in. Kevin Reid, the program manager at that time, expected the majority of agencies to be voluntarily participating by 2009 -- two years ahead of plan.
Today, five years later, around 4,200 of the nation's 18,000 law enforcement organizations -- around 23% of agencies -- are contributing data to the system. Money, politics and technology have all played a role in the delays.
CJIS has faced a difficult challenge for any IT project: How do you get a diverse array of independent organizations to engage with a new technology for the common good when each has its own priorities -- and when participation requires a substantial investment in both time and money to get connected?
Although CJIS does not charge agencies to use the service, software upgrades and integration of existing records cost money -- sometimes in the tens of thousands of dollars, experts say.
Launched in the depths of the recession, the program quickly fell behind schedule. There are signs, however, that N-DEx participation may finally be gaining some momentum.
The idea behind N-DEx was to establish a set of data sharing standards and a central hub, a giant data warehouse into which CJIS could pull together law enforcement incident reports from thousands of disparate, proprietary and often incompatible federal, state, local and regional databases and data sharing networks.
In this way, the theory went, investigators at every level could identify patterns of criminal activity that span multiple jurisdictions to help solve crimes. Authorized users could access N-DEx through a Web portal or by way of their own agency's records management system, once it was configured to do so.
Although it has yet to reach critical mass, N-DEx has already shown promise in allowing investigators to "connect the dots" across state borders when investigating crimes, says Maury Mitchell, director of the Alabama Criminal Justice Information Center, a major contributor to N-DEx.
N-DEx at a glance
Over 4,200 agencies sharing 214 million records, including those from:
Federal Bureau of Investigation
Bureau of Alcohol, Tobacco, Firearms and Explosives
Federal Bureau of Prisons
Drug Enforcement Administration
Department of Justice and Joint Automated Booking System
United States Air Force Office of Special Investigations
Department of Homeland Security. Access to 35 million records from Immigration and Customs Enforcement, and Customs and Border Protection, provided to N-DEx users.
D-DEx DoD Exchange. Includes Pentagon police, NCIS, Army and other DoD law enforcement.
Law enforcement incident reports and corrections data.
Arrest reports, missing persons reports, calls for service, booking reports, and other incident data.
Other information includes data related to pretrial investigations, warrants, supervised releases, citations, probation, corrections departments and more.
The federal database augments smaller networks and one-to-one sharing agreements and integrations between jurisdictions with a central point of exchange and a common memorandum of understanding, or legal agreement between parties.
Right now, for example, the state of Alabama shares information with Kansas, Wyoming and Nebraska, but has no such arrangement with Florida. "Significant crime crosses the panhandle into Alabama and vice-versa," Mitchell says. CJIS, he says, could make information sharing easier -- if Florida makes the commitment to participate.
N-DEx functions as a giant law enforcement search engine, allowing investigators to enter text strings and limit results by geography, date ranges, contributor and other criteria, and it already contains some 180 million records that track over 1 billion people, places and events.
Results so far
"N-DEx allows law enforcement in another state or city to help solve your crimes for you. It puts the solution to the crime where the criminal is, not where you are," says N-DEx program manager Michael Haas.
For example, he says, in 2011 when the suspects in a murder case in the Pacific Northwest suddenly turned up in the Southwest during an unrelated incident, investigators there had immediate access to the incident report containing the homicide details. Detectives were then able to elicit information that resulted in the suspects being returned to the Pacific Northwest, where they were subsequently charged and convicted for murder.
In another case, David Heim, a state trooper, now retired, with the Kansas Highway Patrol and an early user of N-DEx, accessed it from his laptop using the FBI's Law Enforcement Enterprise Portal. During one routine traffic stop in 2010, he says N-DEx revealed that one person in the vehicle was wanted for a drive-by shooting. That person was arrested.
But in his report he identified another occupant in the vehicle, who had no record. "The other guy, he's not associated with the gang, but the gang task force certainly wants to know that he's associating with a gang member." So they entered him into the system as a known associate.
"There's a feature where if I'm watching someone I can put him in the system, and if he gets a record I'll get an email," he says. "Little things like that can sometimes tie a case together later on."
During another stop in 2010, Heim says, the driver of a vehicle carrying 13 illegal aliens claimed he didn't have his ID and provided a false name that, when searched on N-DEx, turned out to be an alias known only to the Department of Justice's Joint Automated Booking System, another N-DEx contributor. The system returned the man's mug shot, along with information on his conviction for human trafficking. Correctly identified, he was then arrested and prosecuted for human smuggling.
Though not the system's core function, N-DEx also includes some analytics tools, such as geo-visualization, timelines and other linking resources. It can geographically map specific types of crimes and identify relationships between them.
"That is one of the strengths of the system," says Tim Bryan, program manager of information sharing initiatives at the International Association of Chiefs of Police (IACP), an organization that provided feedback throughout the development of N-DEx.
Such linking products are particularly of value to task forces, cold case investigators, fusion center analysts and the prosecution community, Bryan explains. For example, if serial arsons were occurring across a multi-jurisdictional region, investigators from the law enforcement community as well as fire marshals could create a chronological timeline with addresses and responding entities to share with the larger investigative group. The disparate data could be used to develop and eliminate certain suspects, he says.
Haas hopes that N-DEx will also be used as a gateway for access to other CJIS databases, including the division's Next Generation Identification system, an updated version of its Integrated Automated Fingerprint Identification System database that contains wanted lists and arrest warrants, criminal histories, fingerprints, palm prints, facial images and other biometric data used for identification purposes.
N-DEx has continued to evolve in phases. In 2012 CJIS began casting a wider net, encouraging states to submit records from prosecutions, court records, corrections, probation and other related areas. So in addition to the incident information already provided by law enforcement, CJIS is adding data about pretrial investigations, warrants, supervised release details, citations, field interviews and incarceration information.
So far, four state corrections agencies are contributing, and four more are in the data mapping phase. (See maps, below, which show all law-enforcement contributions.)
CJIS also established controls that participating agencies can use to determine how the records they contribute may be accessed. Owners can restrict access by geography, agency or individual, and can assign each submitted record an openness level ranging from green (full access) to yellow (returns a phone number to call for more information) or red (record does not display due to the requestor's location, job function or agency).
To date CJIS has 18 state agencies participating, along with thousands more local/tribal agencies, with even more committed. "Once we reach the tipping point, N-DEx will be a vital criminal justice tool for law enforcement," says Mike Wagers, director of provincial police at the IACP. But getting the other 14,000 law enforcement agencies onboard will be more difficult. The technical challenge lies in getting smaller agencies hooked in, he says, noting that 75% of all law enforcement agencies have 25 or fewer officers.
Most agencies want to participate at some level, state and local law enforcement officials say. To help this along, CJIS worked with representatives from law enforcement through the IACP to make sure the tool delivered what they required.
"States are working with CJIS to reduce the cost by standardizing and using the tools CJIS deploys so they don't have to pay for redundant systems that provide a similar function," says Haas. But agencies still must pay to upgrade database software and configure their local record management systems to exchange data with N-DEx.
The total cost varies with the size of the agency and the capabilities of the record management system technology it has in place, but total price tag, including integration, can be in the tens of thousands of dollars, says Steve Ambrosini, executive director of the Integrated Justice Information Systems (IJIS) Institute, an industry consortium of vendors of law enforcement software. Not every vendor supports standards-based N-DEx connectivity yet, he says, but those that do can reduce the cost of an N-DEx agency connection by "several multiples."
"Who assumes the costs of mapping the data to allow local data contributors to talk to the FBI architecture?" says Bryan. "No one can afford to spend another thin dime right now, and that has delayed the progression across the country."
"It all comes down to money," says Mitchell, "and most states are still technologically challenged." Of the 80% of law enforcement agencies that serve populations under 50,000, the majority are not well equipped technologically, says Ambrosini.
These smaller organizations have adopted a wide range of commercial record management system products, and they may run older versions -- if anything. In many cases even the most current versions don't fully comply with the more than eight-year-old CJIS National Information Exchange Model (NIEM) data exchange standards and the Global Reference Architectures services standard that enables automated data sharing with N-DEx. "Many agencies still have antiquated, even DOS-based records management systems, and a few don't have any records management system," Bryan says.
"Systems integrators are building in base capabilities, but we are still climbing the hill," Ambrosini says. "We need to get commercial providers in that market to adopt the standards."
Law enforcement organizations don't have to contribute to N-DEx to access the system. But they do need to sign a memorandum of understanding, comply with security requirements such as changing the password every 90 days and be subject to audit. "A lot of investigators found that cumbersome, [and] agency coordinators and state CSOs don't need one more thing on their plate," says Bryan.
A change in focus
So over the last few years, CJIS has gradually morphed its focus from an individual-agency access strategy to providing access through regional systems.
The road forward is being paved by regional law enforcement data sharing networks, which can act as data aggregators that feed into the N-DEx system. Some states have developed regional systems that can collect data from multiple jurisdictions and exchange data with N-DEx. For example, Commander Scott Edson at the Los Angeles County Sherriff's Department spearheaded an effort to pay IBM to modify its i2 Coplink database so that the department's regional node could share data with, and query, N-DEx.
Edson obtained a $900,000 federal grant and paid to have his version of Coplink modified to access N-DEx. Later on, IBM rolled that development work into the basic product and made it available to all customers.
"It became part of the basic Coplink product," which is one of the most widely used law enforcement databases in the U.S., he says. All 4,000 law enforcement agencies in the U.S. that use Coplink and have software maintenance contracts now can access N-DEx, including all 88 police departments in Los Angeles County alone, Edson says.
But each department still must find the time and resources to upgrade and configure its records management system to access N-DEx. "We still have people waiting to get this upgrade. It's a resource issue, and it takes time" he says.
Overall, Edson is happy with the results. "I encourage other states to regionalize with an inexpensive data warehouse," he says.
That's exactly what the State of Alabama did. It didn't have the budget to buy a commercial product like Coplink, and some of its 350 police departments and 67 sheriffs departments weren't using any computerized incident reporting systems at all.
So Mitchell's organization -- the Alabama Criminal Justice Information Center --decided to develop and roll out its own statewide data warehouse system that would act both as a reporting hub and a records management system for departments that didn't have one.
The Uniform Crime Report Local Template for Reporting and Analysis (ULTRA) data warehouse and the Mobile Officers Virtual Environment portal became the statewide standards in Alabama in 2010. On January 1, 2012 the state began requiring all 600,000 incident reports it receives each year to be submitted electronically, whether local agencies use the statewide system for their own records management or not.
"We do not require that anyone use the products we provide; however, we do require the submission of the data," Mitchell says. Many agencies use their own software, "which is fine with us as long as they get us the data."
Instead the state offers a carrot: The system is available free of charge for crime reporting and arrest reports. And there's a stick: Agencies that don't comply with reporting requirements aren't eligible for federal grant money. ULTRA now sends all incident reports to N-DEx. "We are one of the largest contributors," he says.
Still other geographic regions have connected into the Law Enforcement Information Exchange (LInX), a government-funded, data sharing initiative developed and operated by the U.S. Navy to share incident data with local law enforcement near its bases in Virginia and the Pacific Northwest. "It is a way to have situational awareness with local law enforcement," says Chris Cote, assistant director and CIO at the Naval Criminal Investigative Service.
That network, which has since expanded into 10 regions, now has more than 1,350 participating organizations, including law enforcement agencies within the Department of Defense. "We are a regional aggregator, a force multiplier for N-DEx," says Cote. About half of the regions are now accessing data from and sending data to N-DEx, and the regional governance boards in the other areas have all agreed to participate. "It's just a matter of local and state CSOs saying they're good with it and for N-DEx and the technical side on our end to make it happen," Cote says.
While the Navy pays for the service in areas where it has bases, other areas, such as South Carolina and Atlanta, have also joined and pay the contractor that manages LInX to participate in the service. "This could be considered the law enforcement data cloud," Cote says. But at some point, he adds, "N-DEx will become the center of gravity."
Limits of sharing
Despite the benefits, some agencies have been reluctant to participate in N-DEx. Besides the costs and technical issues, some states are less than excited at the prospect of sharing local law enforcement data with the Feds. "Egos and politics enter into this," says Bryan.
Also, "the states don't like the feds telling them what to do," says Mitchell. "There's a hesitancy. Are we really going to give everything we know about everybody to the FBI? It's a huge Big Brother." Today, Alabama shares only incident offense reports with N-DEx. The state is hesitant about broadening the scope of sharing to include other data types, such as corrections records, Mitchell says.
"I'm not being negative on N-DEx. There's a place for it," Mitchell says. It is possible that in the future, other types of data from state and local agencies might be made available to the FBI and other law enforcement agencies through a federated access model, rather than exclusively through N-DEx, he adds.
That's something Alabama already does with incident records with Kansas, Wyoming and Nebraska. "If there could be links back to the states' original data repository, there would be a broader desire to join N-DEx," he says, because the state would retain control over the data.
Despite the challenges, most agencies like what N-DEx offers, and the consensus is that its role as the dominant national information sharing hub for law enforcement is inevitable. "Technology, thank goodness, is finally overcoming bureaucracy," Mitchell says. "But we still have a ways to go."
This story, "It's criminal: Why data sharing lags among law enforcement agencies" was originally published by Computerworld.