Staff time and storage requirements eat away at IT budgets as companies grapple with logging and archiving compliance data
Surprisingly, despite the volume and intensity of griping it generates, SOX is not the most onerous regulation, according to survey participants. Instead, that honor goes to the various vertical-specific federal regulations, such as HIPAA, GLBA and the Communications Assistance for Law Enforcement Act.
These activity-specific regulations were called out as most expensive nearly 37% of the time, as opposed to just more than 26% of the time for SOX.
Just less than 16% of “most expensive” requirements were state laws, especially California SB1386, and about 13% were federal agency rules, such as SEC or Federal Financial Institutions Examination Council requirements. Rounding out the categories are commercial association rules, such as PCI standards or NASD rules, and other regulations generally.
No one expects compliance costs to go down. A full 60% of benchmark participants expect their compliance spending to increase, and the remaining 40% expect it to at least stay level.
This is because of ongoing changes in the regulatory landscape — including the creation of new requirements or extension of old ones. In short, companies are gearing up to invest in compliance for the long haul.
Capturing the costs of compliance
Even if they can’t be quantified precisely, enterprise compliance costs can be broken down into two broad areas: staff costs, and tools and infrastructure costs.
With regard to staffing, survey participants reported a median of 3.25 and a mean of 4.8 full-time employees devoted to compliance activities within their IT security organizations, ranging from 15 positions in a large transportation company to half a full-time person (based on a set number of hours per year) for a small educational institution.
Figuring $130,000 for a senior security executive (including salary and benefits), companies are spending between $422,500 and $624,000 on security salaries alone — not counting IT executives in areas other than security, or personnel outside the IT department.
To look at this another way, the typical enterprise is spending 2% to 3% of its entire IT budget on staffing for compliance. (The median IT budget for benchmark participants was $20 million.)
What are these people doing? Much of the compliance work in IT comes down to auditing/reporting, and managing the tasks of archiving and data recovery. Despite the “T” in information technology, IT departments still rely on human eyes to review, and human hands to generate compliance reports on security log data. These are not the best uses of time for security staff, as much of the work can be automated.
The auditor cometh
Nearly three quarters — 71% — of participants conduct internal audits, and most of those who don’t believe they should, but can’t because of resource constraints.
Of those who carry out regular internal audits, 54% tend to do them annually; 25% do them quarterly, and almost 17% do them at least monthly. The annual audits tend to be comprehensive, but more frequent ones cover only a subset of policies, procedures or systems.
Slightly fewer participants perform external audits: just shy of 66%. Among those who do, the breakdown of frequency looks very different than for internal audits: The vast majority, nearly 74%, are audited annually. However, of the remainder, more than a third (10.5%) have frequent, ad hoc external audits, often whether they want them or not, courtesy of regulating bodies or corporate headquarters.
Auditing significance is multifaceted. Audits are regulated by numerous regulatory bodies — such as PCI — and therefore represent “table stakes” in some lines of business. They also provide a backstop to IT when it is impossible or impractical to implement full separation of duties. Lastly, in an IT organization striving to continuously improve its security profile and execution — in security shops with mature metrics for success — audits provide the best feedback possible (except for compromise post-mortems) on how well IT is executing according to policy and plan.
It’s important to note that the amount of work the rest of IT does for compliance — tightening security controls on data or programs, for example — is significant, but harder for managers to measure and report, as it is often part and parcel of other security improvements.
But precisely what compliance staffers do is only part of the story. Another question is who is doing what.
Separation anxiety
Laws such as SOX or professional practices codes such as PCI also require separation of duties among the staff.
This is the practice of breaking up the parts of complex business processes and assigning permissions on each portion in such a way that no one person can perform successive steps.
The intent is to make it significantly more difficult for people in the process to perpetrate frauds or propagate errors. For example, consider the process of making cash deposits in a bank and later reconciling account statements: If the same person can do both, he can more easily steal money.
SOX leans heavily on the idea of separation of duties being applied to IT with respect to financial processes and data, yet companies are struggling to keep up with it. Just less than half of participants have a policy dictating separation of duties.
Where separation of duties is impossible because of costs, limitations in the systems or staff shortages, practices such as fuller logging, log auditing and frequent process reviews can serve as a backstop to make up for the lack of real separation.
Compliance tools and infrastructure
The second half of the compliance-cost equation is technology. Ironically, despite the high costs of having humans perform compliance duties, the overwhelming majority of IT executives in Nemertes’ benchmark weren’t buying tools specifically to assist them in compliance. They are instead relying on existing logging tools to develop audit trails, and existing security tools to maintain access controls.
They are, though, increasingly aware of a sometimes hidden cost in compliance efforts: storage. The steady increase in logging and in meta-data generation around production data, archival data and logs means that the amount of required storage is increasing.
Moreover, a little more than half of our participants kept compliance-related archival data on their storage-area networks for a while before dumping it off to tape, which means that in most places archival data retained only for compliance is occupying prime real estate in the high-speed RAID arrays and SANs of the enterprise.
Further increasing the storage costs of compliance are retention practices. More than a quarter of benchmark participants told us they retained records forever (see graphic, below).
These folks have decided that the risks of not having information that might someday be asked for in court outweigh the costs of retaining data permanently, a perspective that’s increasingly valid. Another quarter said that time frames vary according to the kind of information being retained. In some cases time frames are based on legal requirements and sometimes are reviewed regularly, but mostly aren’t. Finally, the remainder retain records for various fixed periods, typically seven to 10 years, or as long as the law requires, potentially plus a few years.
How much is all this records retention costing in terms of increased storage costs? It is hard to say. Reported storage growth rates range into the high triple digits (100+% year over year), with many enterprises attributing records-retention as a significant contributor to that growth.
Storage hardware costs are dropping nearly in half every 18 months (for disk space, at any rate). So despite consumption doubling, with costs halving the overall rate of infrastructure cost increase is probably in the single digits. However, the cost of powering up, cooling off and managing all that extra space is not dropping at a similar rate, so the operating costs of owning and running the storage are rising at a rate more similar to that of space consumed.
Information protection and identity management
Either as a part of or in conjunction with other compliance efforts, many participants in Nemertes’ benchmark are grappling with information protection. At its base, the problem information protection addresses is making sure that information can be seen only by the people who should see it. Clearly, it’s important, and information protection was identified 38.6% of the time as a top security-spending priority in 2007 and 2008.
Information-protection technologies include network and storage encryption, as well as enterprise rights management. Topmost on IT executives’ minds this year is protecting the data residing on enterprise laptops, and encryption is the tool of choice. Solutions range from hard drives with on-board encryption to freeware storage-encryption software to commercial products. About 10% of participants have something deployed or in deployment; more than twice that many are evaluating their options.
Information protection relies on identity management (the ability to keep track of who is who). Identity management was cited by 27.3% of participants as a spending priority in the coming year, and about 60% of those willing to speculate on spending in 2008 expect it to increase.
These projects are intrusive, difficult and expensive, yet also are increasingly seen as unavoidable, especially by large enterprises.
Auditing tools and technologies
In one important way, the key to compliance is logging. If you don’t have sufficient audit trails in the form of logs of network, system and application configurations and activities, it is difficult to demonstrate compliance. This, along with the increasing sophistication of log analysis for security, has driven the rapid adoption of log aggregation.
About 64% of participants collect logs from many sources and aggregate them for analysis and retention. Just less than half of that group aggregate all infrastructure-related logs — server, router, firewall — and some collect desktop logs as well.
About 90% of participants, whether they aggregate logs or not, do some kind of log monitoring, either manually or with a logging tool. At least a quarter of these do so only during business hours (9 to 5), or even less frequently. About half of participants, however, have some kind of real-time analysis and response to logged events running all the time.
This is mostly managed via intrusion-detection/prevention systems tools, or via alerts or alarms generated within the system logging the event. About a quarter of participants use dedicated log-management systems or a full-blown security-information and event-management (SIEM) tool, and another quarter are developing or evaluating them. Unfortunately, participants also are finding implementing a SIEM to be tricky.
Important for considerations of compliance and e-discovery, about 78% of participants archive some or all logs they collect. This can equate to a phenomenal amount of disk space, even if some sort of log-normalizing system is in place to reduce the amount of duplicative information to a bare minimum. However, this information is rarely used outside the log-management and security purposes for which it is initially collected. About two-thirds of participants do not do any sort of data mining against the logs.
All risk, no reward
Costs are just half of the case; determining the benefits of all these compliance efforts is more challenging. With compliance, there is no expected ROI, no reward: Any business case is built around risks, losses and costs avoided. Ultimately, the calculation each enterprise must seek to guarantee is this:
(risk of payouts/penalties/losses/damages) * (expected costs/losses) > costs of compliance.
Yet defining those costs is challenging. One approach is to assess the likelihood, and expected size, of a noncompliance fine. One benchmark participant — a large manufacturing company — found it could not afford to update certain back-end systems to put in proper access control or auditing quickly or easily. Because the system was pegged for an upgrade within the next few years, the company decided to pay fines for noncompliance in the meantime.
Conclusion and recommendations
From the emerging perspective of risk management, compliance is about managing risks to the enterprise, in the forms of fines, legal sanctions, penalties or expulsion from professional organizations and business lost or never won because of bad PR following disclosure of a major compromise.
Each enterprise should focus on process improvements, and specifically on follow-through in the form of internal compliance audits, to confirm that defined processes have been adhered to. Despite the prevalence of auditing, internal adherence to policy on such issues as system configuration continues to be a problem generally.
Each enterprise also should define, or re-evaluate, its data archiving and retention policies, taking into account compliance requirements and risk factors associated with wiping data after it is no longer required. If the legal staff feels lawsuits calling for such data are likely, IT should work up the costs of permanent retention so that an informed decision can be made concerning risks and costs of mitigation.
Burke is principal research analyst at Nemertes Research. He can be reached at john.burke@nemertes.com.
