Vectra Networks correlates odd bits of user behavior that signal an attack in progress

Behavioral analysis is used to identify and correlate incidents that, taken individually, might be ignored.

This column is available in a weekly newsletter called IT Best Practices.  Click here to subscribe.  

There's little doubt anymore that perimeter security, while still absolutely critical, isn't sufficient to fully protect enterprise networks. A determined attacker will always be able to find a way in. What's more, in the case of a malicious employee, the attacker is already inside the gates.

So, if the presumption is that some attackers are already inside, what's needed is a solution that can complement peripheral security by detecting malicious activity and stopping it in its tracks before the damage is done.

There's certainly no shortage of solutions that fall into that category. Many of them look for command and control (C&C) traffic to a compromised server that is running a reconnaissance scan to learn about the internal configuration and assets, spreading malware or moving data. The C&C traffic is critical for orchestrating the malware.

Traffic going to an external C&C server is certainly an important sign that a network is under attack, but it's not the only sign. The security company Vectra Networks tracks a wide variety of C&C traffic, but takes the idea several steps further by monitoring internal network traffic to detect the spread of an attack through a network and the attempt to steal data.

The company has observed that unusual behavior inside the network is a good indication that an active attack is going on. By correlating seemingly odd bits of user activity that individually might be ignored, Vectra formulates a "threat certainty" score that prioritizes events and, ideally, triggers alerts to security operations people before any communications leaves the company. The Vectra solution also identifies how close an event is to breaching prioritized and sensitive assets (as determined by the company), so IT personnel can address the most important alerts first.

If you look at the major attacks that have made headlines recently, they all follow a similar blueprint. The first step is the attackers get some kind of privileged access to the network. For example, in the case of Target Corp., the attackers managed to get keylogger software onto a computer at a third party contractor. This allowed them to capture the user's credentials, which they used to log into the Target portal to get a foothold inside the network. They expanded that foothold internally until they found the assets they wanted to steal and then went about exfiltrating the data.

In the Target case, as well as with many other attacks, that lateral movement inside the network represents an opportunity to stop the attack—if the activity raises a red flag early enough. This is the premise behind the Vectra solution.

The Vectra appliance sits on a tap port or tap infrastructure, or on a span port on one or more aggregation switches, and listens to all the traffic traversing the network. Vectra customers typically put the solution at the intersection between key assets, such as apps or data, and the security zones that represent the attack surface. By being at that intersection, Vectra can hear the traffic between the users and the key assets, as well as traffic that goes from user to user. Vectra uses data science and machine learning to determine which traffic is gray.

Within a couple of days of installation, the machine learning allows Vectra to understand what traffic is normal for a particular network (it listens to gigabits or tens of gigabits of data per second). After that, white traffic is ignored but the gray traffic – which is suspicious – is put through a set of detection algorithms that identify behavior that indicates an attack could be in process. Even if an attack goes on over a period of weeks, Vectra maintains a long-term memory about all of this. Then these individual incidents get correlated to the host under attack.

For example, consider what happens when an attacker does internal reconnaissance of a targeted network. It might start when the attacker gets malware onto an end user's computer, perhaps through a phish email. Once in a trusted position the attacker can start to poke around.

He knows the PC's IP address and now he can test for other network IP addresses. In the course of doing that he might reach out to an IP address that was never assigned. This is certainly an unusual behavior, and Vectra would detect it and begin to score for risk. The score grows higher for threat level or certainty based on the number of IP addresses that Vectra sees the computer reaching out to and the rate at which it reaches out to them. In this same way, Vectra looks for a variety of additional reconnaissance, lateral movement and data theft. Vectra correlates each of these incidents and plots the aggregate score on an XY chart, as shown in Figure 1.

Vectra Networks

Figure 1: Vectra host detections user interface

The dots on the chart represent a single host (solid dots) or multiple hosts (hollow dots) that have suspicious behavior. Clicking on a dot allows a security person to delve into details. As the dots move to the right of the chart and change color from black to yellow to orange to red, the certainty of an attack and severity of a threat are escalated.

When a threat score reaches a threshold set by the user organization, any number of actions can be triggered. For example, an alert can be sent to a SecOps person or instructions can be sent to a SIEM or firewall to block communication with an external IP address. Vectra claims to be able to identify and stop activity in the kill chain in real time.

Another feature of the Vectra solution is community threat analysis. Using its traffic listening technique, Vectra automatically builds communities of hosts and key assets that normally communicate with each other. Unusual communication outside of these communities raises suspicion, especially in the case of insider attacks. It could be legitimate communication, or it could be nefarious. Community threat analysis is just one more tool for raising the red flag as soon as possible in order to stop an attack in progress.

Rob Caputo is a principal consultant at the IT advisory firm CS Technology. He has used Vectra Networks' solution since 2012, first joining a beta program and now as a full fledged customer. Caputo chose this product because of its ability to catch things without having to use a signature. "We have a range of security products in-house and we have seen that Vectra's solution identifies activity much sooner than the other products," says Caputo. "We follow up on the Vectra alerts and remedy the situation, and then a week later we find out it was 'malware X' that our other tools can now scan for and eliminate because a signature was created."

Caputo says the Vectra solution "just sits there and does its job. We don't have to do any maintenance, which makes this product really easy to use." He also says that the detailed alerts enable first-line support staff to handle most issues. "Our frontline support people can jump on something immediately and eradicate the problem before it escalates. The management team here is very keen to make sure we keep everything pretty well locked down, and Vectra helps us do that."

Join the Network World communities on Facebook and LinkedIn to comment on topics that are top of mind.
Must read: Hidden Cause of Slow Internet and how to fix it
Notice to our Readers
We're now using social media to take your comments and feedback. Learn more about this here.