Taming the data beast

When it comes to managing data, who’s minding the store? Is it the database team? The storage management team? The ERP team? The e-mail folks? The application management team? The e-business team? The users?

Although each of these groups has a vested interest in the data, they also have very different perspectives and different interactions with the data. The problem with the way many companies manage data is they approach data management from a technology perspective, managing data by the device where it resides rather than managing data by the value it has for the business. This is a silo approach to data management, but what’s needed is an overall, strategic data management plan.

Data can be dispersed across a company – on personal computers, departmental machines, storage-area networks, scattered servers, PDAs, RAID arrays, tape, laptops and more. Dispersed data present a huge management challenge, with many different pockets of constantly changing data to be managed.

These pockets of data aren’t always effectively managed if there’s no comprehensive plan to manage all company data. Examples of inadequately managed data are the company price list or address contacts on an employee’s PDA.

Adding to the problem is that the data increase daily. The sheer volume creates another issue of its own: Those responsible for managing the data don’t necessarily have a clear understanding of the specific characteristics and data management requirements of the different kinds of data they have to manage. Managers of the data need to understand the needs of the application that creates the data and the needs of the applications and users that use the data downstream.

There can also be regulatory requirements, business policies and special user needs to consider.

Plus, some pieces of information are more valuable to a company than others. When dealing with so much data, we end up “genericizing” the data. With “genericizing,” we tend to categorize data into broader categories and then handle the bulk of data in each category as an entity, or as a generic group. This is done for the purpose of organization and simplification. The problem with “genericizing” is that not all of the data in a broad category are the same; to properly handle data based on its importance to the company, the data should be given much more granularly.

For example, using a broad category of “e-mail,” and setting up one set of back-up and retention policies for the “e-mail” category may not be appropriate in all cases. The better approach is to develop different data management policies for executives, the legal department, human resources and low-level staff people. 

Management of data includes backup and restore, database management, data integration, data retention, hierarchical storage issues, and so forth. However, proper management goes beyond the operational, technical level – so a data management strategy must be choreographed at a higher level in the organization. Unfortunately, in some organizations, managing the data has been more operationally focused, tactically instead of strategically.

Strategy provides the context within which IT managers can appropriately apply the correct resources (both technology and cost) to address the company’s data management needs. The strategy also pulls together the management effort for your staff – from the database staff, storage staff, operational staff, and others. The advantage with this approach is that it takes into consideration the use of the data across technologies and company divisions.

For example, a strategic approach could help when trying to determine what storage hardware is needed. Let’s say an application creates data and needs sub-second access to at least two months of transactions, while transactions more than two months old are no longer accessed. In addition, the data must be retained for three years due to regulatory requirements. Also, the auditing department may need access to the past year’s transactions, but access needs are not immediate and can be scheduled. Having this kind of context about the requirements of the data through its entire use at the company is helpful in making the correct tactical decision of what hardware to buy.