A common comment from engineering and technical personnel is that if we can't measure something, we can't manage it effectively.
Security consultant Gordon Merrill continues his series on fundamental management tools for information assurance (IA) professionals in general and IA security consultants in particular. His insights and recommendations will also help clients choose consultants wisely and judge their performance appropriately.
* * *
As a security professional, you may be tasked with gathering data that will become part of a senior officer's presentation to upper management. If you are an external consultant, you will probably not have full access to the company information technology (IT) services as you find and analyze the data you need to create appropriate security metrics; you need to convince client personnel to find the data for you. Part of the task is knowing the full scope of the project before you begin planning what metrics to gather. It may be as easy as looking at data they are already gathering and finding that the staff don't realize that there are valuable metrics hidden within.
You may also need to look at how they report any metrics already used in-house. Do they use the top-down approach or the bottom-up approach, as described in a 2006 paper on security metrics by S. C. Payne published in the SANS Security Essentials collection? The top-down method starts by determining what they need to report based on the company goals and how they can prove they are meeting them. The bottom-up approach looks at the resources they have available (e.g., logs, applications, and funds) and constructs the report using what they have. Ideally, the top-down approach is the best; however, as a consultant who is not an employee in the client's company, you may often find yourself having to work under contract limitations that force you to cope with what you have without the option of having the client spend more money to collect more data.
Metric reporting
Supposing the bottom-up approach is all you have available to you, what can you use for data and what can you manage to extract from the data? Most of your data in this case will come from log files. Find out how logging is configured; if necessary, as Scott Berinato suggests in a 2005 paper published in CSO Magazine, ask the IT team to reconfigure their current logging to provide the widest range of information you can use in your security analysis.
The key to logging for metrics is to save everything possible within the limits of storage: you cannot go back and get what you did not gather. If necessary, it may be cost effective to buy some inexpensive off-the-shelf high-capacity disk storage units for your work; a 2TB USB/Firewire IOMEGA external drive cost only around $350 in 2009; the 1.5TB unit version cost $200 at that time. Providing the team with an easy-to-use disk may soften the resistance to increased volume of log files.
In addition to collecting expanded log files, you may need to invest in a log analysis tool for the operating system in question that helps you find what you need from logs by appropriate filtering and search capabilities. Searching Google using "log file analysis tools" as the search string brings up a number of articles and data sheets about products worth examining.
For reading about the complex issue of security metrics, see the following references:newsletter in June 2009.Directions in Security Metrics Research" by Wayne Jensen (2009) from NIST (NISTIR 7564)Guide to Security Metrics, A," by Shirley Payne (2006) from the SANS Institute InfoSec Reading Room.Performance Measurement Guide for Information Security" by Elizabeth Chew, Marianne Swanson, Kevin Stine, Nadya Bartol, Anthony Brown, and Will Robinson (2008) from NIST (SP 800-55 Rev 1).Security metrics research" in this newsletter in May 2009.Service management metrics significant for CSIRTs" in this newsletter in February 2008.Security Metrics: Replacing Fear, Uncertainty, and Doubt by Andrew Jaquith (2007) published by Addison-Wesley (ISBN 978-0321349989). AMAZONSecuritymetrics.org "community [W]ebsite for security practitioners"
• "Consensus metrics for information security" in this
• "
• "
• "
• "
• "
•
•
In the next part, Gordon discusses the regulatory environment.
* * *
Merrill, MSIA, currently lives and works in Tennessee. His career has taken him to 48 of the 50 states and to six foreign countries. Gordon's information assurance background has included working for major computer companies such as IBM, managing IT projects for Fortune 250 companies in the risk management field, owning his own business, and working as a private consultant. You may contact him by e-mail.
This series is based on some of the papers Merrill wrote during his MSIA Program at Norwich University from 2007 through 2008. Merrill and I have collaborated closely in rewriting his research for this series.