The technology industry is buzzing about intelligence, analytics and other ways to make insight-driven decisions based on real data. What is not talked about so much is where these applications gets all of this information.
Metadata is the data about data, and it is the key to significant insights for the enterprise. Knowing anything about the data used in an enterprise starts with metadata, which notes important details, such as when a file was last opened, how often it has been accessed, who accessed it, its size, its location, and so on. In fact, Stewart Baker, general counsel to the NSA has said, “Metadata absolutely tells you everything about somebody’s life. If you have enough metadata, you don’t really need content.”
+ Also on Network World: Managing data by business objectives +
While we might not all be keen on the NSA minding our metadata, if the insight metadata offers is rich enough for national security, imagine what it can do for enterprise IT. Big Data applications already make use of metadata, but now the intelligence made possible with metadata is crossing over into the storage side of IT. This is a great leap forward that enables IT to move past a storage-centric focus, which has historically been blind to the activity of the various files stored in their datacenters, to a much deeper focus on the data itself.
See what you’ve been missing
In traditional architectures that don't have metadata management tools, storage admins cannot see how active (or hot) or inactive (or cold) their data is. But with the visibility provided by metadata, IT can see how data is actually being used by applications. For example, when applied to data access, metadata reveals how frequently a file is used, which applications or users are opening it, if the file is changed or updated, and when it tends to be opened and closed.
Not only does metadata give unprecedented insight into data usage, but it also uncovers patterns that enable predictive intelligence to improve storage efficiency. When you finally know what data is cold, you know which datasets can move safely to an archival storage option like the cloud. This delivers substantial savings to the enterprise, and it preserves your high-performance storage capacity for data that needs speed to keep applications and business moving rapidly. Automating these capabilities and delivering them without application interruption makes this even more powerful.
Smart storage starts with metadata management
A metadata engine is software designed to separate and offload the architecturally rigid relationship between applications and where their data is stored. By collecting metadata of a client’s data access and how it experiences storage (IOPS, latency, bandwidth and availability), a metadata engine can apply intelligent analytics against business requirements to provide the level of performance, cost and reliability; make real-time automated decisions; move data without disruption to overcome or prevent outages; and maintain compliance to service-level agreements or objectives.
Not only does a metadata engine deliver insight, but it also improves storage performance. Offloading metadata access delivers predictable, low-latency metadata operations by guaranteeing that metadata operations do not get “stuck” in the queue behind other data requests. Rather than having to wait for sequential operations to complete, a metadata engine can leverage parallel access with the latest optimizations of the standard Networked File System (NFS) v4.2 protocol in use at many large enterprises. Leveraging NFS v4.2 significantly speeds up metadata and small file operations by requiring less than half of the protocol-specific network round trips compared to NFS v3.
Get more of what you paid for
Metadata requires some significant software innovation to enable different types of storage to be simultaneously available for applications, but deep expertise is not needed to deploy metadata management and gain the value it opens up for enterprises with petabytes of data. Most large companies have invested significantly in a wide variety of storage that was once cutting-edge technology. While those systems may not be the fastest on the market today, many older assets can still meet the needs of most moderately active or even cooler data.
Enterprise data cools quickly, and it is often estimated that 60 to 80 percent of an enterprise’s data is mostly inactive. Businesses operate in the present, and this week’s sales information or project development data is critical. But even once cool, that data also has future value for reporting or future revisions.
Rather than being blind to data and bound to one type of storage until it is time for a long-term migration, IT can use metadata management to maximize the value of their existing storage investments and get the insight needed to see what data really needs the fastest storage money can buy. Once data can move easily according to evolving business needs, IT becomes much more agile and able to respond in real time to support rapid growth.