Digging out new rootkits

These attacker tools have become stealthier than ever. Even so, they may not require specialized protection.

Talk of rootkits , favorite attacker tools for compromising computer systems without detection, has again begun percolating among security experts. The question is, "Why?" Rootkits certainly aren't new - they've been around for more than a decade, first on Unix and now primarily targeting Windows systems. And their purpose hasn't changed much either. They still give attackers root control and a backdoor into the compromised system.

But the latest buzz is how rootkits have become exceptionally stealthy, and now sometimes hide malware and subvert traditional desktop defenses. For example, rootkits found earlier this year by F-Secure's research team on Web servers in Russia were hiding the Masland family of viruses targeted at taking down Web sites operated by Chechen rebels. And the Klog rootkit uses a kernel filter driver to implement keyloggers.

Nobody can say how many computers already are owned by the stealthiest of rootkits. But the number might be surprisingly high, researchers say, given that new rootkits are so hard to detect. Since they have complete control of their hosts, rootkits hide the calls to the operating system that would normally alert security professionals to their presence. Some rootkits even turn security software off altogether.

"Once a rootkit is in your kernel, it has complete and total control of all memory and hardware. Hence, all software, including your defensive technologies, can be subverted," says Greg Hoglund, co-author of Rootkits: Subverting the Windows Kernel, and owner of rootkit.com .

Nipping rootkits in the bud

Some security experts see today's stealthier rootkits as another form of malware to be dealt with through traditional means - primarily policy-enforced endpoint security, network anomaly detection and intrusion prevention.

As a first level of protection against rootkits, corporations need to keep endpoint security patches up-to-date, particularly on the browser, says Alfred Huger, senior director of engineering for Symantec. This is important, he adds, because rootkits install most often on computers and laptops that have touched upon malicious sites. Most of these are malicious sites that use free Web-hosting services, says Websense Security Labs, which in the first half of 2005 tracked more than 2,500 personal sites and blogs hosting malware installers. More than 500 such sites were discovered in the first two weeks of July, Websense reports.

"The most common and alarming thing we've seen with these back doors is commercial-related crime, such as credit card theft and identity theft," Huger says. "They're taking advantage of Internet Explorer and Mozilla client-side vulnerabilities to access the system."

Other security experts believe that dealing with today's rootkits requires a way of looking at what goes on inside a computer that today's protective technologies aren't sophisticated enough to handle. "This is a much bigger problem than anti-virus companies alone can solve," says Alan Paller, research director at the SANS Institute. "By nature, rootkits are hard to find. The even better ones are harder to find."

Digging deeper

Getting onto data center servers for purposes of espionage means more targeted attacks that could require specialized rootkit finders, says Mikko Hypponen, F-Secure's director of anti-virus research. Internal rootkit installations are harder for attackers to achieve, but they're not uncommon, he adds.

Hypponen relates the tale of an Austrian company that reported receiving false positives when using the beta version of F-Secure's BlackLight rootkit finder released in March. "We went over the records with them. BlackLight found 10 highly suspicious file names on their file server - turned out to be a brand-new rootkit never discovered in the wild before that," he says.

A handful of rootkit removal tools detect some of the most used rootkits, including Hacker Defender and FU. These detection technologies compare the Windows Task Manager process list to the internal system task list. A difference in these two is a telltale sign that a rootkit has probably tampered with TaskMan. The same is true for hidden folders in registry keys.

But as with any malware maturation cycle, rootkit developers and defenders are playing a cat-and-mouse game. At the recent Black Hat security conference, for example, developers announced Shadow Walker, a new hiding technique that can be located only if rootkit detectors upgrade their processes to check whether memory or page fault handlers have been replaced, Hypponen says.

That throws the usefulness of specialized rootkit defenders into question, at least for Pete van de Gohm, director of security for Bayer's North American IS division in Pittsburgh. For now, he says, the best protection against stealth rootkits is the layered, in-depth defenses that savvy IT managers already use - not another specialized security tool.

"Know and understand what's normal for your network. Look at your access points, and figure out which is the best way to mitigate blended threats," van de Gohm says. "For example, it's pretty easy to tell when your Web server has become an FTP server if you just watch your traffic."

As "Mudg," a division scientist at BBN Technologies pointed out during a morning security "rant" session at the recent DefCon hacker conference, all networks follow the same laws of physics. For example, servers don't normally initiate calls; they just serve. Desktops don't normally talk to each other. Packets don't travel fragmented and out of order across the network. Obviously, he concludes, "These are all signs of trouble."

6 questions to ask rootkit vendors

  • Which of the following detection algorithms is used?

    -The cross-view approach (detects hidden objects to indicate a rookit has been embedded).

    -Direct rootkit detectors (catches hooked entries into IDT/SDT/IRP).

    -Signature-based approach.
  • Does the rootkit defender combine cross-view detection with heuristics-based file analysis? (So far, no tools do both, but they should.)
  • Can the tool detect the rootkit at time of installation?
  • Does work have to be stopped for a full scan of the machine to detect the rootkit? How long does that take? How often should you scan?
  • Does the technology detect kernel, memory, BIOS and/or user mode rootkits?
  • Can the tool automatically remove the rootkit? Often rootkit removal requires booting into safe mode and then manually cleaning.

(Compiled with the help of Joanna Rutkowska, owner of invisiblethings.org, and Vlad Gorelik, CTO of Sana Security.)

Radcliff (www.debradcliff.com) is a freelance writer specializing in online safety and network security.

Learn more about this topic

Q&A: Holy Father says he writes rootkit for fun, profit

03/16/05

Kaspersky Lab Paper: Rootkits and how to combat them

Network World on Security newsletter

Join the Network World communities on Facebook and LinkedIn to comment on topics that are top of mind.
Related:
Take IDG’s 2020 IT Salary Survey: You’ll provide important data and have a chance to win $500.