How to SOC it to the bad guys

A security operations center is becoming an enterprise must-have.

Eamus Halpin's wake-up call was the Slammer worm. Until it hit, he had relied solely on port blocking to protect his enterprise network from hacks and intrusions. After he saw the network carnage Slammer wreaked around the globe, Halpin knew he had to revamp his company's approach to network security.

"I happened to be with Microsoft at the time at an NDA event in Seattle, and somebody scared me about what could happen to a port blocking-based network hit by Slammer," recalls Halpin, who is chief technical architect at iRevolution, a managed services provider in London. Although iRevolution's network was spared a direct hit by the worm, Halpin knew that had just been luck. "I spent three hours researching the implications of the worm, and my hair went white. We were as open as Swiss cheese," he says.

Although iRevolution had the basics in place - firewalls, anti-virus software, intrusion-detection systems (IDS) - it had no way to combine alerts from these various security tools to build a logical picture of the security health of the network.

"Everything was separately maintained and managed. They didn't speak to each other and didn't give us a business temperature for the enterprise as a whole," Halpin says. "So we could see occasionally that we were being attacked by a particular type of virus through e-mail, but we couldn't really determine how big an issue that was in the great scheme of things."

Halpin decided then and there to do a complete security overhaul. His goal was to build and maintain a world-class security operations center (SOC) for iRevolution's internal network, as well as to help support customers.

Just as network operations centers (NOC) continuously monitor networks to mitigate faults and ensure optimal performance, SOCs continuously monitor and manage a range of security devices and events to maintain and ensure overall network security. Experts say SOCs are becoming more common among companies for a variety of reasons, most notably because security has evolved from a discipline based on point solutions to something far more pervasive and critical to overall network health.

"It used to make sense to have security specialists managing the various firewalls, IDS and so on because security was at a very specific location on your network and had a very specific function," explains Andreas Antonopoulos, senior vice president and founding partner at Nemertes Research. "But security no longer works that way. The perimeter is porous, and instead, security needs to be applied at the application level, at the network level and at the storage level. It's become a feature of your end-to-end application delivery, much like network performance."

Regulatory pressure brought on by the Sarbanes-Oxley Act (SOX), the Health Insurance Portability and Accountability Act and the Gramm-Leach-Bliley Act also drives enterprise SOC development.

"SOX is a good example of a proactive driver for SOCs," says Diana Kelley, executive security adviser at Computer Associates' eTrust division. "You've got to be ready for 404," she says, referring to the section of SOX that explains that executive management needs to take responsibility for establishing and maintaining an adequate internal control structure. "That means you need the correct and effective controls on your business reporting. And once you have them, you need to monitor and maintain them, and a SOC is an effective way to do that."

According to preliminary research by Nemertes, the average U.S. organization plans this year to up its security budget 100%, from 2.4% of the IT budget to 4.8% - an increase Antonopoulos attributes almost entirely to regulatory compliance. "I can tell you that all those companies that are doubling their budget to do regulatory compliance are looking at either building a SOC or re-engineering a SOC to comply with the regulations," he says.

The trend to pull back security monitoring duties previously outsourced to managed security services providers (MSSP), especially in the financial services sector, adds fuel to the fire. An internal SOC allows better control and visibility into the enterprise network, and reduced costs overall, says Jim Tiller, chief security officer and vice president of security services at International Network Services, a network consulting firm.

"MSSPs are having difficulty responding in some cases," Tiller says. "With the regular occurrence of worms and denial-of-service attacks, especially in the financial industry, and the increase in our vulnerability and the sophistication of those threats, the ability to respond is strictly related to how much visibility you have in your network. By pulling the management in, you have more visibility and can facilitate the ability to respond."

Five SOC pitfalls to avoid

1. Technology tunnel vision. Getting caught up in the latest and greatest tools is tempting, but the core of your security operations center (SOC) should be based on sound risk assessment and security policies. Once you’ve hammered those out, you can focus on the products and technologies that will best support them.

2. Silo mentality. Don’t organize your SOC in a silo separate from your network operations. An efficient SOC depends on fully integrating security and network monitoring tools, as well as the staffing associated with them.

3. Staffing mistakes. Don’t use your veteran security staff to do low-level monitoring, and make sure you have the proper checks and balances in place so that no one person holds all the keys to your network kingdom.

4. Inflexible tools sets. Choose tools that will support not only your current security devices, ticketing systems and network monitoring suites, but also those that are easy to customize and offer a variety of templates and wizards. Be aware that even the best tool sets require a good deal of customization and integration.

5. Taking the cheap route. A SOC is no place to skimp. On average, large organizations should plan to invest $1 million or more to implement and maintain a truly enterprise-level SOC. And that investment will most likely grow over time.

Joanne Cummings

Plus, "for large companies, the investment in managed security services is fairly significant and they're seeing long-term cost/benefit with regard to pulling that in-house and managing it themselves," he says.

The hurdles

Although recognizing the need for a SOC is fairly easy, building one is not so straightforward. This is especially true when the security and network operations groups have grown up independently. Security monitoring might be robust, but if it is separate from network operations monitoring that can be a recipe for disaster, experts say.

"Security events don't always appear as security events," Antonopoulos says. For example, if a router stops responding and that's all the information you have, it's difficult to tell if it's a network problem, a systems problem or a security problem. If your network operations group is completely separate from your security operations group, one of two things will happen: "Either both groups will chase the problem separately, or worse, neither will chase it, concluding that it's the other group's problem," he says.

This confusion is exacerbated when it comes time for remediation. "If both organizations are implementing things on the network and monitoring it, you may come to the point where the network people are changing [access control lists], reducing your security, or your security people are applying ACLs that are impacting network performance," he says. "Since you're not integrating this and looking at it from an end-to-end perspective, you end up with problems."

A true SOC integrates security and network event information so the security and operations staffs have an overall view of the event and the effect it's having on the network, and can make informed decisions about how best to react according to predefined security policies. But that's easier said than done.

Where to start

Many organizations first look to purchase a security event management system or alert correlation engine. But experts say that's a tactical mistake. An overall risk assessment, for determining the actual business importance of each network asset, must come first in the SOC project.

"You have to apply your resources to protect the things that are most important to you," says John Summers, global director of managed security services at Unisys. "Some IT execs have a very good handle on their infrastructures. They know what assets are out there and what's running at each IP address, but very few can tag a business priority to their infrastructure elements."

How to staff a SOC

Staffing a security operations center can be almost as challenging as building it or paying for it, users and experts say.The 24/7 monitoring necessary in a SOC presents one of the biggest hurdles. “For companies used to having security personnel working eight hours a day, five days a week, that dramatically increases their overall staffing requirements, since one 24-by-7 seat is equal to roughly five full-time employees,” says John Summers, global director of managed security services at Unisys.Faced with such a prospect, many organizations look to cut corners.For example, some make the mistake of staffing their SOC solely with their best security personnel. “Companies take seasoned security professionals, stick them in front of a screen and ask them to do a six-hour monitoring shift,” says Andreas Antonopoulos, senior vice president and founding partner at Nemertes Research. “You won’t retain those people too long because they will very quickly become bored.”Beyond boring and overworking a valued staffer, this tactic also could create a huge security vulnerability.“If one person is writing your security policy, implementing your policy, monitoring it and then checking for compliance, that person is basically one huge risk,” Antonopoulos says. “There’s no separation of duties, and absolutely no checks and balances.”Instead, a good SOC, like a good, traditional network operations center, should be staffed in tiers, with Tier 1 personnel receiving alerts and doing low-level troubleshooting and Tier 2 and 3 people handling more complex alerts and problems. In the best of all worlds, Tier 1 personnel should provide the first line of response for both the security and network operations sides of the house.That way, your more veteran security professionals can handle the more complex risk-management and policy-writing tasks, while putting lower-level staffers into the SOC for the primary monitoring. Then, when alerts come up and the Tier 1 staffers are unsure how to proceed, they can kick up the problem to a Tier 2- or 3-level person. Only then does your more expert, and expensive, staff get involved. 

Joanne Cummings

Knowing the business importance is key because the purpose of a SOC is to enable not only security event monitoring but also confident responses to those events. "So if this server went down, what would it mean to the business, and is this server more important than this other one? Once you know that, the technology part tends to fall into place," says Summers, who manages Unisys' three major SOCs.

Technology caveats

Choosing a technology platform comes next. The goal is to find a security event management platform that can work with the variety of security devices you have in place, correlate their various alerts, and provide some form of integration with whatever you are using for trouble ticketing and network operations management. Organizations need this depth of visibility into the network to ferret out security breaches, experts say.

"A big financial services company we work with was seeing some poking at different areas of its network around the globe," CA's Kelley says. "Each of the pokes didn't look particularly bad individually, but they were all coming from the same IP address over a period of a couple of days. None of it was enough to trigger an alert on its own, but once the company pulled that information into a centralized console [within its SOC] and saw what this one IP address was doing to its network around the globe, things started to add up."

However, getting to the point where you can have such a global view within your SOC is time-consuming and expensive.

In addition to integrated security event managers from start-ups such as ArcSight, Intellitactics and netForensics, most of the large network management companies - CA, HP and IBM - offer a security event monitoring capability integrated within their platforms. But they all come at a pretty hefty cost.

"In the security space, IDSs generally don't speak the same language to your management system that your firewall does," Nemertes' Antonopoulos says. "If you want to add a rule into your firewall to block something, you can't use the same language you would use to add a rule in a router. As a result, security event monitors require a large integration project to pull all that information, turn it into a common format and correlate it across all those domains."

1 2 Page 1
Page 1 of 2
Now read: Getting grounded in IoT