The Zettabyte Era is Coming – Are you Ready?

To manage the enormous growth of data, your data infrastructure must scale efficiently and economically without sacrificing performance. Enter Zoned Storage.

istock 1094275754

It’s no secret that data growth is at an all-time high. IDC expects 103 zettabytes of data to be generated worldwide by 20231. Yes, zettabytes. With the proliferation of IoT devices, 5G-enabled technologies, and the massive growth of video, we’re just scratching the surface of how companies will store and extract value from data.

Machine data will play a leading role, as it is expected to outpace commercial/consumer data. By 2023, more than 90% of data will be generated by machines2 (i.e. smart cities, IoT, endpoints, autonomous cars, etc.). In addition, video, surveillance footage, and large AI/ML data sets – this “natural streaming” or sequential data – will be everywhere. 

Another challenge we face: as more data is generated, less of it is being stored. This could be because of storage economics or the perceived lack of value in the data. By 2023, only 12 ZB – around 12% of data created – will actually be stored3.  Yet our ability to store data is vital to realizing its potential – utilizing it for analysis, mining, intelligence, and value creation.

Data infrastructure must also be re-architected to address the growing scale and complexity of workloads, applications, and AI/IoT data sets.

With all this in mind, there’s one thing clear about the Zettabyte Era: it is not feasible to just keep adding storage. Large data sets fueling big data analytics and other write-once, read-many types of sequential data continue to grow. To manage the enormous growth of data, your data infrastructure must be able to efficiently and economically scale without sacrificing performance. These new constructs will involve multiple tiers of workload-optimized storage as well as new approaches to system software that can leverage higher-density storage with better economics at scale.

A Roadmap to Scaling in the Zettabyte Era

Zoned Storage is a new paradigm in storage motivated by the incredible explosion of data. It is an open source initiative that brings together new storage innovations and industry standards for cloud and hyperscale data center architects to design efficient storage tiers that help maintain competitive TCO and achieve greater economies of scale in the Zettabyte-scale Era. 

As more and more data will be sequential in nature – video, IoT/edge data, surveillance, or data that can be grouped or "zoned" into larger chunks such as large AI/ML datasets – there are opportunities to organize the way workloads are written to storage media for better performance and efficiencies.

SMR HDDs and ZNS NVMe SSDs are leading the way for a Zoned Storage Approach 

The Zoned Storage architecture enables host applications to orchestrate data placement and take full advantage of the highest available storage capacities. These “zoned” devices such as shingled magnetic recording (SMR) hard disk drives (HDDs) and the emerging zoned namespaces (ZNS) standard for NVMe solid-state drives (SSDs) are leading the way and provide optimizations for sequential workloads.

SMR enables today’s highest-capacity hard drives. SMR is patterned much like the shingles on a roof. By overlaying tracks on a disk, more data can be squeezed into the same space – increasing areal density by up to 20% compared to conventional recording. SMR drives are purpose-built for the data center and allow for optimized performance, economies of scale, and system-level intelligence of data placement.

ZNS divides flash media into zones. These zones can only be written sequentially, much like data written sequentially to tracks on SMR HDDs. ZNS divides the flash media into zones, where each zone is an isolated namespace. Cloud providers can, for example, separate workloads or data types to different zones so that usage patterns are predictable among multiple users. The end result will be SSDs that deliver better density and are far more cost efficient. Zoned storage devices will reduce the need for background garbage collection and require less over provision of the media. ZNS devices complement new storage technologies like QLC Flash by providing for more efficient use of the media.

ZNS complements SMR technology, enabling developers to take advantage of both SMR and ZNS under a single storage stack, regardless of media type. This delivers intelligence to application architectures and, because they are seen as one technology, makes SMR and ZNS instrumental building blocks of the new Zettabyte-scale Era.

Rearchitecting the data center with a Zoned Storage approach can require some effort initially, but the density and cost benefits are substantial and demonstrate all the advantages of purpose-built hardware and software-aware constructs. Many Western Digital customers are already deploying SMR technology today, and the company expects that 50% of the HDD exabytes it ships will be on SMR by 2023.

Open Source is Imperative

It’s important to note that Zoned Storage is an open-source, standards-based initiative that provides a unified framework to manage purpose-built ZNS SSDs and capacity-optimized SMR HDDs. Participation enables customers to leverage SMR technology with their HDDs and integrate them into their applications. The advantage is that in preparation for ZNS SSDs, customers can leverage their SMR investment and apply that to their ZNS deployments. In fact, if they have designed their application around the concept of zones, their applications will likely require minimal changes.

 [1] IDC Global DataSphere Forecast, 2019-2023: Consumer Dependence on the Enterprise Widening, January 2019, DOC #US44615319

[2] Applied Materials, SEMICON West, AI Design Forum, July 2019

[3] IDC, Worldwide Global StorageSphere Installed Base Forecast, 2019–2023: The Global StorageSphere Installed Base by Core, Edge, and Endpoint, April, 2019, DOC #US45009319

western digital post 3 image jpeg Western Digital

Forward-Looking Statements:

This article may contain forward-looking statements, including statements relating to expectations for Western Digital’s products, the market for these products, capabilities and applications of its products for data strategies. These forward-looking statements are subject to risks and uncertainties that could cause actual results to differ materially from those expressed in the forward-looking statements, including development challenges or delays, supply chain and logistics issues, changes in markets, demand, global economic conditions and other risks and uncertainties listed in Western Digital Corporation’s most recent quarterly and annual reports filed with the Securities and Exchange Commission, to which your attention is directed. Readers are cautioned not to place undue reliance on these forward-looking statements and we undertake no obligation to update these forward-looking statements to reflect subsequent events or circumstances.

Copyright © 2020 IDG Communications, Inc.