Service management metrics significant for CSIRTs

I subscribe to the Network World e-mail newsletter service just as you do. I particularly appreciate the notifications about white papers in relevant areas that I work in for my consulting practice beyond security such as help desk management and data center operations. Today I want to discuss some recent research that bears on computer security incident response team (CSIRT) management.

Recently I was alerted to a valuable paper entitled “IT Service Management Metrics that Matter,” available free in return for a brief registration process. The paper was written by Gene Kim, co-founder and CTO of Tripwire and co-founder of the Information Technology Process Institute (ITPI).

Why do some organizations manage to run their IT services efficiently and effectively? According to the research published in the ITPI’s study, “Not All IT Controls Are Created Equal: Understanding the performance improvement potential of Foundational Controls,” (available free by registering with the ITPI), there were 21 controls in six categories out of a total of 65 controls studied in a survey of 98 North American companies that had "the greatest correlation with the operations, security and audit performance measures." The group’s research shows that the foundational controls were implemented significantly differently in top-, medium- and low-performing IT groups.

In the “resolution controls” category, the four key controls were:

* Track the percentage of incidents that are fixed on the first attempt (first fix rate).
* Use a knowledge database of known errors and problems to resolve incidents.
* Rebuild rather than repair to resolve and incident.
* Have a defined process for managing known errors.

In the Tripwire paper, Kim discusses the following key measures of IT team performance:

* Mean time to repair: the best-run organizations focus on analyzing what may have changed when problems arise; poorly run groups bumble about rebooting systems without reason.
* First fix rate: good groups fix the problem on their first try in a high percentage of cases.
* Change success rate: how many changes to production systems are implemented without causing disruptions?
* Server-to-system administration ratio: “…high performing IT organizations were not only the most effective, but they were also the most efficient - those with the best Mean Time to Repair, First Fix Rate, and Change Success Rate also had the highest Server to System Administration Ratio.”

Chief information security officers will do well to study these reports and think about how to apply the insights to security management. For example, a CSIRT can fruitfully measure how quickly a team analyzes a security incident to find an immediate correction - and also apply the analysis to finding the underlying causes of the vulnerability that has been exploited or the error(s) responsible for security violations.

Similarly, a security team will be concerned with patch implementations as well as planned changes to functionality of production systems, working closely with the programming group and the operations team.

M. E. Kabay, PhD, CISSP-ISSMP, specializes in security and operations management consulting services and teaching. He is Chief Technical Officer of Adaptive Cyber Security Instruments, Inc. and Associate Professor of Information Assurance in the School of Business and Management at Norwich University. Visit his Web site for white papers and course materials.