Backblaze report shows slight uptick in HDD failure rates

Cloud backup vendor Backblaze issued its latest quarterly findings for hard-disk drive (HDD) reliability, and it shows a slight uptick in failure rates — but hardly something to fret over.

All told, Backblaze has 106,238 hard drives spinning in three data center colocations, and every quarter it highlights the failure rate of each model drive it uses. The company first came to prominence several years ago when it highlighted an abnormally high failure rate of Seagate drives.

The problem arose about two years after massive floods in Thailand (around 2011) ruined the manufacturing facilities of several hard drive manufacturers, with Seagate taking it especially hard. I did some reporting back then for a now-defunct publication and found out that some corners were cut to get hard drive production going again and that those cuts resulted in a bunch of time bomb hard drives with higher than average failure rates.

Since then, things have settled down and the Backblaze reports have gotten rather boring. There haven’t been any great abnormalities or problems to be had in recent years, and the one thing Backblaze noted was that consumer hard drives were often more reliable than “enterprise” hard drives supposedly made for the data center.

One of the knocks on Backblaze is that its report covers its entire inventory, and comparing a brand-new 12TB drive to a 4TB drive that has been in deployment for a few years isn’t exactly fair. Backblaze currently uses 4TB to 14TB drives, but it is retiring the 4TB drives as fast as it can.

Annualized failure rates for hard-disk drives increase slightly

With Backblaze’s new report, things have changed. The company uses primarily HGST (owned by Western Digital) and Seagate drives with very little WD in its inventory. Over the last three years, the trend for annualized failure rates (AFR) has fallen, but with Q1 of 2019, things have gone back up.

Andy Klein, director for compliance at the company, noted in a blog post that “while Seagate has reduced their failure rate over 50 percent during that time [a three-year timeframe], the upward trend over the last three quarters requires some consideration.”

The highest AFR, at 2.6%, comes from a 12TB HGST brand, but context is everything here: They had one failed unit out of 520 deployed. That’s easily an aberration. The next highest failure rate is 2.22% for a 12TB Seagate helium drive, also a very new model. It had 180 fail out of 34,708 deployed.

After that it drops to 1%, and some are in the less than 1% range. Sure, there is no doubt Seagate drives had higher failure rates than HGST drives in general, but when the failure rate is 1% vs. almost 0% you can’t complain too much — unless you are buying hard drives in the tens of thousands like BackBlaze does.

“The Annualized Failure Rate (AFR) for Q1 is 1.56%. That’s as high as the quarterly rate has been since Q4 2017, and its part of an overall upward trend we’ve seen in the quarterly failure rates over the last few quarters,” wrote Klein. He said the company would monitor the trend going forward to see if anything new comes of it.

Klein noted that while Backblaze reduced the total number of hard drives in service in Q1 by 648 drives compared to the prior quarter, the company added nearly 60 petabytes of storage. That’s due to the retirement of more than 8,000 4TB-8TB drives and replacing them with 92,000 12TB drives.