At this past year's RSA Conference, I was the moderator on a panel about "Security Metrics: What matters?" One of the participants was my friend Jody Brazil, President of Firemon. Jody referenced a new survey that Firemon had partnered with the Ponemon Institutue about security metrics and the interaction between security teams and executives.
The full survey was just released last week. Without even getting into what metrics you should measure and what you should report, the survey shows some startling findings around how security admins and the executive team interact (or at least how they perceive each other to interact) or maybe how they don't interact.
First off was the disconnect about how strong the organization's security posture was. Security pros felt that 66% of executives thought that their organizations were either very strong or well above average. At the same time, only about 39% of security pros felt that their organizations were very strong or well above average.
According to the study, this points to a failure of the actual security posture being reported up to the executive team. The Ponemon report says that this factors into why security does not get the resources it needs to do better, namely executives making budgeting decisions think that security is already a strength. Personally, I think anything that can be used to justify more security budget is quickly latched onto by security teams.
Delving into why the executives don't have a realistic viewpoint of the real security posture of the organization, respondents cited five factors that all scored more than 50% as responses.
Interesting that over 70% think communication is at too low a level (I assume on the executive side). Does this mean high-level executives are not engaged? The next most popular choice - only communicating when there is an incident - is a classic issue and in more than just security. Two of the popular answers that information is too technical and negative facts are filtered are two that I have heard time and time again.
Many security pros tell me they have to "dumb down" security metrics so executives can understand them. Others have said that any technical information just shuts executives down from paying attention. My issue is that there are some things that are important and can't be brought down to a second-grade level. We need to convey the real picture and it may take a little domain intelligence. This screams to why you need a security person in the executive room. However, even today most organizations do not have a CISO or equivalent as part of their executive team.
Filtering out negative facts is another staple. No one wants to be the bearer of bad news, though for too long security teams have done just that. As a result, security has gotten a "chicken little" reputation of always screaming the sky is falling. Afer a while, we move from chicken little to the "boy who cried wolf" and no one pays attention. This is certainly borne out in the survey answers.
For me, though, the most surprising responses were on when the executive team meets with the security team:
Over 50% of respondents said they meet with the senior executives only when a serious risk is revealed, or that they don't communicate at all. Eek, that is scary. Scarier still is that only 13% of organizations have regularly scheduled meetings.
The rest of the report is chock full of more great information and insights. I had a chance to speak with Jody Brazil of Firemon and Jerry Skurla, VP of marketing at Firemon, about the survey. Firemon seeks to provide proactive security intelligence to organizations. Deciding what the correct information is for different audiences is something the Firemon team has spent a lot of time and effort on. Overall, Jody says the problem is we have done a lousy job of defining what metrics are really important. That is a threshold question to answer before we can tackle how to best convey that information to executives.
Until security teams can get their heads around which information is important and then tackle how to best show it to the executive team, we are destined to repeat many of the failures of the past. That is too bad. Let's hope for all of our sakes that we begin to answer these questions soon.