* Mich Kabay goes through the Center for Internet Security's metrics
On May 20, 2008, the Center for Internet Security (CIS) announced the public release of a set of metrics for information security. The organization is dedicated to helping “organizations reduce the risk of business and e-commerce disruptions resulting from inadequate technical security controls. Click Here to learn more about CIS’s mission.” Their charter was last updated in 2002 and is fully described online.
On May 20, 2008, the Center for Internet Security (CIS) announced the public release of a set of metrics for information security. The organization is dedicated to helping “organizations reduce the risk of business and e-commerce disruptions resulting from inadequate technical security controls. Click here to learn more about CIS’s mission.” Their charter was last updated in 2002 and is fully described online.
At the simplest level, the CIS is dedicated to establishing benchmarks – that is, measurable objectives – based on real-world contributions of security practitioners. Their description summarizes their process as follows:
The center provides Internet security benchmarks based on recognized best practices for deployment, configuration and operation of networked systems. The center’s security-enhancing benchmarks encompass all three factors in Internet-based attacks and disruptions: technology (software and hardware), process (system and network administration) and human (end user and management behavior). The benchmarks are open, that is, publicly available to everyone.
The center’s Internet security benchmarks are intended to:
• Provide managers, business partners and insurance underwriters with a security ‘ruler’, in which each increment on the ruler represents a set of security-enhancing actions. This security ruler will enable an organization to select the level of security deemed appropriate for that enterprise and implement the specific technical actions associated with the security level chosen;
• Include interventions that can be implemented before, during and after attacks to reduce losses; and
• Be subject to customization, where appropriate, for specific industries and risk profiles such as those needed by the healthcare sector to implement the extensive privacy and security requirements of the Health Insurance Portability and Accountability Act of 1996 (HIPAA).
Technical requirements without enforcement mechanisms are rarely effective. To ensure that the benchmarks are more than paper products, the center will develop and deploy:
• Compliance/auditing methodologies, including automated vendor tools certified by the center, to ensure efficient and accurate compliance with the benchmarks;
• Accreditation guidelines for system administrators and auditors to allow them to demonstrate a high level of proficiency in implementing and auditing against the benchmarks, and
• Methods of maintaining confidentiality that encourage CIS members and others to share information that supports keeping the benchmarks up-to-date.
Cyber attacks will continue; therefore the benchmarks will be enhanced and updated to ensure that available benchmarks respond to real losses.
CIS has made a wide range of technical metrics freely available; after a simple registration process, visitors can download any or all of the specific documents (PDF) and tools (executables and scripts) for the following categories (see the menu for details):
• Applications; e.g.,
• Web servers
• Databases
• Virtual machines
• Operating systems
• Windows
• *nix
• Mac
• Apple iPhone
• Routers
• Firewalls
• Wireless networks
The 90-page general overview called “Consensus Metric Definitions v1.0.0” covers the following areas:
• Incident management
• Vulnerability management
• Patch management
• Application security
• Configuration management
• Finance
As one example of the style of this manual, the “Incident Management” discussion (p 10 ff) begins with Table 2 (“Security Incidents Table”) data attributes “that should be populated as completely as possible for each security incident.” The table has the following columns:
• Name
• Type
• De-Identified
• Required
• Description
Table 2 includes the following items:
• Incident ID
• Date of Occurrence
• Date of Discovery
• Discovered By
• Detected by Internal Controls
• Verified By
• Date of Verification
• Date of Containment
• Date of Recovery
• Level of Effort
• G[r]oss Loss Amount
• Business System Downtime
• Scope of Incident
• Affected Systems
• Affected Organizations
• Classifications
• Root Cause
• Priority
• Country of Origination
• Country of Destination
The document then continues with in-depth definitions of specific metrics; for example, the Mean-Time-To-Incident-Discovery (p 13) has the following objective:
“Mean-Time-To-Incident-Discovery (MTTID) characterizes the efficiency of detecting incidents, by measuring the average elapsed time between the initial occurrence of an incident and its subsequent discovery. The MTTID metric also serves as a leading indicator of resilience in organization defenses because it measures detection of attacks from known vectors and unknown ones.”
Its description is as follows:
“Mean-Time-To-Incident-Discovery (MTTID) measures the effectiveness of the organization in detecting security incidents. Generally, the faster an organization can detect an incident, the less damage it is likely to incur. MTTID is the average amount of time, in hours, that elapsed between the Date of Occurrence and the Date of Discovery for a given set of incidents. The calculation can be averaged across a time period, type of incident, business unit, or severity.”
The fundamental question is, “What is the average (mean) number of hours between the occurrence of a security incident and its discovery?”
The targets are described as follows: “MTTID values should trend lower over time. The value of “0 hours” indicates hypothetical instant detection times. There is evidence the metric result may be in a range from weeks to months (2008 Verizon Data Breach Report). Because of the lack of experiential data from the field, no consensus on the range of acceptable goal values for MTTIDs exist.”
The discussion continues with usage, limitations and references.
I believe that all of us in the field should be paying attention to this overview document and to the specific metrics made available by the CIS. By working together to improve these documents through our collective intelligence and experience, we can build and document collective intelligence to benefit all of our organizations and our stakeholders.
