Flood of threat intelligence overwhelming for many firms

The amount of threat information coming in from security systems is overwhelming for many companies

Data science
Credit: Nick Hoffman

Three years after Target missed alerts warning them about a massive data breach, the amount of threat information coming in from security systems is still overwhelming for many companies, according to new reports, due to a lack of expertise and integration issues.

Seventy percent of security pros said that their companies have problems taking actions based on threat intelligence because there is too much of it, or it is too complex, according to a report by Ponemon Research released on Monday. In particular, 69 percent said that their companies lacked staff expertise. As a result, only 46 percent said that incident responders used threat data when deciding how to respond to threats, and only 27 percent said that they were effective in using the data.

"There's too much data to really make sense of if you have a limited resource staff of security operations center analysts or threat analysts," said Travis Farral, director of security strategy at Anomali, which sponsored the report. "It can be overwhelming to sit and figure out which of these 100,000 things to look at first."

It takes a special kind of person to be able to do this, he added.

Travis Farral, director of security strategy at Anomali

"There are starting to be a few training classes out there for this, but the skill set is different from the typical person who does analysis to find out if something happened or not," he said.

According to the report, 52 percent of respondents believe their companies need a qualified threat analyst to maximize the value of threat intelligence.

In addition to lack of expertise, it's also difficult to integrate the various technologies involved.

"You've got logs in different formats, firewalls in one format, endpoint logs that are in a completely different format, and you try to merge in threat intelligence data which is typically specific IPs or domains or hashes of malware," Farral said. "It's not necessarily straightforward to try to bring everything together in one place - and having to go to 50 different browser windows is overwhelming."

In fact, while 62 percent of respondents said that SIEM integration was necessary to maximize the value of threat intelligence data, 64 percent said that the integration of a threat intelligence platform with other security technologies or tools is a difficult and time-consuming task.

Another survey, released this morning, showed that 72 percent of organizations have tools in place to defend against advanced persistent threats, 79 percent scan for malware, 52 percent do penetration testing, and 44 percent do cyber forensics. In addition, 66 percent have a cyber security plan that fully covers all on-premise environments and devices, and another 25 percent have partial coverage, while 61 percent fully cover cloud-based environments and devices and 29 percent have partial coverage.

The high percentage of companies who had sophisticated security tools in place was a surprise, said Vikram Chabra, solution architect at NetEnrich, which sponsored the report.

However, lack of expertise remains an issue, he added.

"Despite the fact that we have the finest tools that can defend against advanced persistent threats, we still need qualified security analysts or engineers to look at the incidents thrown out by the tools, comb out false positives, and take actions," he said.

To help deal with the issue, 66 percent of companies said that they used third-party consultants or managed security service providers to develop or implement their cyber security plans.

Intergration was an issue here as well, Chabra added.

"Your security technology vendor isn't the same as your managed security service provider," he said. "You've got multiple vendors involved -- one vendor managing the security, another managing the technology, and there's a gap there."

Finally, according to a report by security vendor eSentire, despite the large amounts of data flowing in from firewalls and other security systems, a large number of attacks are still slipping through.

"There are many attacks that do not get detected by traditional defenses because the velocity at which the bad guys evolve their weaponry is so much faster than how the good guys can respond," said Mark McArdle, CTO at eSentire.

And it's not just the most clever attacks that get through.

According to a report based on two years of sensor data, 57 percent of attacks that get through firewalls and antivirus systems are unsophisticated, brute-force attacks.

This is due to ongoing, automated activity by attackers running scans looking for unpatched software, default passwords, and misconfigured systems.

"We consider that to be the 'background radiation' of the Internet," McArdle said. "There's nothing you can do to stop that from happening -- it's just one of the realities you accept the minute you connect to the internet."

These probes are constantly looking for ways that attackers can grab a foothold in a system, and there isn't much that companies can do to stop it without also locking out customers, partners, employees, and other legitimate services.

These attacks are often not picked up by SIEMs, he added.

"The SIEM's only source of visibility are the events generated by the firewalls and the antivirus," he said. "And while the SIEM will give excellent views into the attacks that it knows about, it will have nothing to say about new attacks or sophisticated attacks. There's lots of good information in it, but relying on it as the primary means of identifying threats will result in you missing significant activity."

This story, "Flood of threat intelligence overwhelming for many firms" was originally published by CSO.

Join the Network World communities on Facebook and LinkedIn to comment on topics that are top of mind.
Must read: Hidden Cause of Slow Internet and how to fix it
Notice to our Readers
We're now using social media to take your comments and feedback. Learn more about this here.