Isn't one of the definitions of insanity to keep doing the same thing over and over again expecting different results? You might get that feeling after reading a Government Accountability Office report out today that says despite myriad problems with its handheld computer systems the US Census Bureau is pushing ahead with the program - albeit with changes.
Certainly using wireless handhelds makes tons of sense as Census-takers gear-up for the 2010 nationwide census. The idea is to save time and improve the accuracy of the Census and Harris Corp. was given a five-year, $600 million contract for 500,000 handhelds to make that happen.
But during a Census dress rehearsal last Fall, the GAO said the system in may cases fell on its face and issues continue to arise. For example, during the test, field staff (known as listers) experienced most frequently reported issues with transmission, the device freezing, mapspotting (collecting mapping coordinates), and problems working with large blocks of data. When problems were identified, the contractor downloaded corrected software to the handhelds. Nonetheless, help desk resources were inadequate, the GAO said.
The Census Bureau also reported that 5,429 records were lost and not recorded in the mapping and address database because multiple handhelds had the same identification number assigned to them. As a result, when a handheld transmitted information, it overwrote any data previously recorded for handhelds with the same identification number. According to the GAO, bureau officials this problem was identified and corrected during the address canvassing dress rehearsal.
Meanwhile, data show census staff productivity exceeded expectations in rural areas but did not meet Bureau expectations in urban/suburban areas, which represent a greater share of housing units across the nation, the GAO said. The GAO added that it previously testified that the Bureau had not sufficiently measured the performance of the handhelds during the dress rehearsal, nor fully specified how it will measure performance during the 2010 Census.
The Bureau received data from the contractor on average transmission times, but the bureau hasn't used the data to analyze the full range of transmission times, nor how transmissions may have changed throughout the entire operation. Without this information, the magnitude of the handheld computers' performance issues throughout the dress rehearsal was not clear, the GAO said.
In addition the Census Bureau has not developed an acceptable level of performance for measures on total number of failed transmissions or average connection speed. The contract supporting the Bureau's field data collection calls for the contractor to provide near real-time reporting and monitoring of performance metrics and a control panel/ dashboard application to visually report metrics from any Internet-attached personal computer. Such real-time reporting may be helpful to the contractor and the bureau to monitor ongoing address canvassing operations in 2009, but was not used during the dress rehearsal, the GAO said.
The bureau has developed a preliminary list of dashboard metrics, which include such daily measures as average transmission duration, and expects to use the dashboard for address canvassing in a full dress rehearsal expected to take place in late January 2009.
One factor that may have contributed to these performance problems was a compressed schedule that did not allow for thorough testing before the dress rehearsal. Given the tighter time frames going forward, testing and quickly remedying issues identified in these tests becomes even more important, the GAO said.
For its part the Census Bureau is trying to fix problems and acknowledged most of the GAO's findings. But it's not like the problems haven't been simmering. In April in fact, the bureau made significant changes to its plans in an effort to address system problems.
It seems likely we haven't heard the last of this one.
Layer 8 in a box
Check out these other hot stories: