Skip Links

Security metrics research

Author reviews several references on security metrics.

Security Strategies Alert By M. E. Kabay, Network World
May 27, 2009 12:02 AM ET
Sign up for this newsletter now!

The long view of security strategies for your network.

One of the most difficult aspects of managing risk in information assurance (IA) is that our statistical information is so poor. We don't know about security breaches that we have not noticed; we don't report all the breaches that we do notice to any central collection point; and we use dreadful methodology for collecting information using poorly constructed surveys that have tiny percentages of respondents, no internal validation and no follow-up verification.

On a practical level, the question arises of just exactly what we should be measuring (such as how to define security metrics) as ways of understanding and managing security issues.

Dr. Gary Hinson, CISSP, CISA, CISM, MBA of Isect wrote an excellent paper entitled "Seven myths about information security metrics" that was originally published in the ISSA Journal in July 2006. Hinson thoughtfully and articulately challenges these seven common assertions (quoting the headings):

1. Metrics must be objective and tangible
2. Metrics must have discrete values
3. We need absolute measurements
4. Metrics are costly
5. You can't manage what you can't measure and you can't improve what you can't manage
6. It is essential to measure process outcomes
7. We need the numbers!

In his section on "Some pragmatic design considerations for information security measurement systems," Hinson discusses key issues (again, quoting his headings):

1. Which things are we going to measure
2. How will we measure things?
3. How will we report?
4. How should we implement our measurement and reporting systems?
5. Setting targets

One of Hinson's references is to the National Institute of Standards and Technology (NIST) Special Publication (SP) 800-55, "Security Metrics Guide for Information Technology Systems." Since his article was written, this publication has been revised.

NIST SP 800-55 Revision 1 is entitled, "Performance Measurement Guide for Information Security." Published in July 2008, this 80-page document was written by Elizabeth Chew, Marianne Swanson, Kevin Stine, Nadya Bartol, Anthony Brown and Will Robinson.

The authors summarize the benefits of using measures (§3.2) as follows (they explain each of these points at length):

• Increase accountability
• Improve information security effectiveness
• Demonstrate compliance
• Provide quantifiable inputs for resource allocation decisions.

The document provides thought-provoking analysis of the organizational implications of developing and using security metrics, including a perspective on U.S. federal government pressures such as the Federal Information Security Management Act (FISMA). Section 5 has a valuable schema for developing appropriate metrics and Section 6 makes practical suggestions for implementing the data collection for those metrics.

Another useful reference for everyone interested in security metrics is the recent (March 2009) draft publication entitled "Directions in Security Metrics Research" (NIST Interagency Report, NISTIR 7564) by Wayne Jensen. This short (26 page) paper has an intriguing abstract:

M. E. Kabay, PhD, CISSP-ISSMP, specializes in security and operations management consulting services and teaching. He is Chief Technical Officer of Adaptive Cyber Security Instruments, Inc. and Associate Professor of Information Assurance in the School of Business and Management at Norwich University. Visit his Web site for white papers and course materials.

Our Commenting Policies
Latest News
rssRss Feed
View more Latest News