Report details biggest IT failures of the past decade

Decades-long overcharging and billions of dollars in spending on code are among the staggering examples of 'IT systems gone wrong.'

oops key showing mistake error or failure 96395168

This month, radio electronics publication IEEE Spectrum is commemorating the 10th anniversary of its ground-breaking 2005 article, "Why software fails."

The now-archived article studied some troubled, large-scale IT projects. IEEE said they were preventable failures and explained why.

Along with the celebration, the publication has just brought out an updated database of IT debacles. This bunch covers the last 10 years. It makes for fascinating reading.

Financial waste, endless delays correcting things, and the vast numbers of people affected contribute to the horrific, gory screw-ups.

Rewind to 2005

The original 2005 article "analyzed a dismal list of IT projects gone awry," as IEEE explained it at the time, and included such scary projects as Kmart Corp's supply chain management system cancelation in 2001, after $130 million was spent. Another example was when the state of California squandered $44 million before canceling a project to upgrade its Department of Motor Vehicles computer system in 1994. The list goes on.

Today's version of the article, entitled "The Staggering Impact of IT Systems Gone Wrong," explores the "many ways in which IT failures have squandered money, wasted time, and generally disrupted people's lives."

Billions in monetary cost

Among the largest fiscal IT debacles that IEEE's Risk Factor blog has placed in its latest "rogue's gallery" is the $18 billion that the U.S. Army spent on a spiraling "Future Combat System" program that was eventually terminated.

Late in the FCS project, final cost estimations were coming in at between $200 billion and $300 billion when the job was cancelled. Problems that IEEE wrote about in 2009 included the sheer amount of code.

"When first proposed, FCS was supposed to contain 33.7 million lines of code; by the end of 2007, it had grown to 63.8 million. This year, the GAO said it had climbed again to 114 million," IEEE said at the time.

Decade-long durations

Two of the biggest failures lasted 10 to 12 years, according to IEEE.

One was the Veterans Affairs' failure to upgrade software that led to 12 years of shortcomings for families of deceased veterans. In that case, 400,000 individuals were affected.

"The VA never updated its computer systems," the Associated Press said at the time, quoted by IEEE in 2008 when the problem was discovered.

Overcharging customers for years

The second of the biggest screw-ups in terms of duration was the Indiana Bureau of Motor vehicles' overcharging of 180,000 customers over 10 years. The dollar amount of the incorrectly calculated vehicle tax was $29 million, IEEE estimated.

Poor programming was to blame.

"Someone apparently started asking how the excise tax was being calculated, and this led to the discovery of the tax overcharges starting in 2004, IEEE said in its article about the issue in September 2012.

Numbers of people affected

Some whoppers are listed in IEEE's database when it comes to the number of people affected by some of the failings, too.

Skype's 2007 two-day crash caused by a software update gone wrong affected 220 million people, IEEE estimates.

"The disruption was triggered by a massive restart of our users' computers across the globe," Skype said of the outage on its website then.

"A flood of log-in requests, which, combined with the lack of peer-to-peer network resources, prompted a chain reaction that had a critical impact," it added.


Rounding-out the biggies in terms of impact on people is Gmail's two-hour outage in 2009, which affected an estimated 150 million people, IEEE says.

The "problem was due to an 'upgrade,'" IEEE said in 2009, quoting the Register.

'Rogues' database

For me, two sayings come to mind after studying the debacles: "There but for the grace of God go I," is the first one.

"We're going to ensure to our customers that this kind of thing does not happen again," would be the second.

Join the Network World communities on Facebook and LinkedIn to comment on topics that are top of mind.
Must read: 10 new UI features coming to Windows 10