One of the best ways to understand your enemy – what he’s up to, what his capabilities are and how he can damage you – is to spy on him.
And according to some cybercrime experts, one of the easier and more effective ways to do that is to hang out where the bad guys do – on the Dark Web.
In a recent post on Dark Reading, Jason Polancich, founder and chief architect of SurfWatch Labs, asserted that, “most businesses already have all the tools on hand for starting a low-cost, high-return Dark Web intelligence operations within their own existing IT and cybersecurity teams.”
Such a data mining operation, he wrote, could be up and running in a day.
It is widely known in IT circles that the Dark Web is a thriving cybercrime marketplace offering multiple exploits, hacking for hire, stolen personal data and intellectual property, spam and phishing campaigns, insider threats for hire and more.
It is also a relatively secure place for criminals to operate, thanks to randomness, anonymity and encryption.
[ ALSO ON CSO: How to surf the Dark Web for fun and profit ]
But just because it is difficult to track criminals individually doesn’t mean it is impossible to conduct surveillance on what they are doing. Polancich wrote that the Dark Web is the place to, “find out what may have been stolen or used against you and improve your overall security posture to close the infiltration hole.”
Is it really that easy?
According to Kevin McAleavey, cofounder of the KNOS Project and a malware expert, “easy” may not be the right word. But “possible” definitely is.
“Can anyone do it? You bet,” he said, “but only if you're willing to pay people to sit around and just surf. Most managers consider that ‘wasting time’ and it's often frowned upon, but it works really well.”
He said that was one of the things he did in a previous job – “follow the bad guys back to their cave so I could see what they were working on before they released it. But it was one of the most time-consuming parts of being ahead of the curve rather than under it.”
Nicholas Albright, principal researcher, ThreatStream, agrees. “These networks seem obscure to many, but with a simple tutorial, anyone could be up and running in less time than it takes to watch an episode of ‘Mr. Robot’,” he said.
“The hardest part of monitoring is really learning where to look. Many of the sites on these obscure networks move locations or go offline periodically. However, once an individual has identified a handful of sites, they frequently lead to others.”
He also agrees with McAleavey that it is labor-intensive, and does not always yield useful intelligence. On the “slow” days, “you might not see anything of value,” he said. “Furthermore, this requires an analyst's fingers on keyboard. Deploying a 'tool' to do this job is not effective. Scraper bots are detected and regularly purged.”
Others are a bit more dubious about the average IT department doing effective Dark Web surveillance, even if the budget is there. “The task of collecting raw information itself is non-trivial,” said Dr. Fengmin Gong, cofounder and chief strategy officer at Cyphort. “And distilling the threat intelligence from the raw data is not any easier. So while it is beneficial to do it, it's not a task that can be undertaken by an average IT department effectively.”
That, he said, is because the average IT worker doesn’t have the expertise to do it, “and it’s not easy to get up to speed. It requires understanding of threats and data mining, which is a high hurdle.”
Fred Touchette, security analyst at AppRiver, is less dubious, but said the deeper the analysis goes, the more expertise is required.
“Initial high-level research should be easily executed by any research team that knows its way around implementing Tor (The Onion Router),” he said. “Once one gets a basic understanding of how Tor is implemented and how to use it, the Dark Web is nearly as easy to navigate, albeit much slower than the regular internet.”
“And once research goes beyond passive and into trying to find and possibly purchase samples, things could get pricey,” he said. “Depending on the merchant, sometimes free samples can be obtained, but not always. From here, the same tools and expertise would be required to analyze samples.”
Easy or difficult, most experts agree that enterprises monitoring the Dark Web for threat intelligence is not yet mainstream. “I am aware of technology researchers and developers proposing this as a complementary means to security threat monitoring, but it's not very common as an initiative taken by enterprises themselves,” Gong said.
That may change, however, as more tools become available to make surfing the Dark Web easier.
Juha Nurmi, writing on the Tor Blog, said he has been working since 2010 on developing Ahmia, an open-source search engine for Tor hidden service websites.
And Eric Michaud, founder and CEO of Rift Recon, is also CEO and cofounder of DarkSum, which launched just last week and is promoting a search engine that it calls “Google for the Dark Net.”
Michaud agrees with Gong that effective surveillance of the Dark Net would be beyond the capability of most organizations smaller than Fortune 100. But he said with a search engine like DarkSum that indexes the Dark Net, they can do it. “We make it easy,” he said.
McAleavey said he has already done it. “All it really takes is setting up a couple of machines to crawl the Tor network with a dictionary list of interesting keywords to match up with, and then let it rip,” he said.
“Once the results have been put into the database of what was found and where, human analysts can then fire up a Tor browser and check out what the crawler found. The more keywords you have, the more results you'll get, and the more people you have to rifle through it all, the better the chances of finding the needles in that haystack.”
Of course, indexing the Dark Web is not static. As McAleavey notes, sites on the Tor network, “often change their address every few hours or every few days, so you need to crawl again looking for those sites of interest because they probably moved since the last time you crawled.”
Michaud agreed, but said it is possible to keep up with address changes. While he wouldn’t discuss the techniques his company uses to do it, “we do it really well,” he said.
Whether it is worth the time and expense to conduct Dark Web surveillance is also a matter of debate. Gong contends that while it is helpful as a “layer” of security, it is not easy to do well. “It requires both sophisticated infrastructure and technical skills that are not trivial to establish,” he said, adding that, “it is not very crucial or affordable for an enterprise IT to pull off by itself.”
And he believes there is, “nothing that can replace direct monitoring of your own networks and assets.”
But Michaud said as it becomes easier and cheaper, it will be a necessary part of a security operation. “Enterprises are scared,” he said, “because they know they will be held responsible for data breaches if they aren’t proactive.
“If you’re just being defensive, you’re going to have a bad day.”
This story, "How to uncover the Dark Web" was originally published by CSO.