New IDS tool fends off false positives

Passive fingerprinting improves intrusion detection.

Intrusion-detection systems are a key component of a multi-layered security  architecture. However, the use of network IDSs has been hampered by less-than-perfect detection and a preponderance of false-positive alerts, which report attacks when no vulnerability exists or no compromise has occurred.

As a result, some security administrators have questioned the value of existing IDS deployments.

In response, a next generation of IDS products that use passive fingerprinting technology is emerging.

Passive fingerprinting identifies information regarding target host systems within a network. The IDS gathers critical data such as operating system, services, and in some cases applications running on the host, and uses this information to reduce false positives.

Cross-referencing

When an IDS detects a packet containing possibly malicious traffic, the IDS cross-references it against the target host vulnerability profile to determine if the host is vulnerable to the potential attack. If the host is deemed not vulnerable, the alert notification is suppressed. For example, the IDS would suppress an alert regarding a Windows Remote Procedure Call-based attack targeted at a Linux server when it determined that the targeted host was not vulnerable.

Passive fingerprinting works by comparing key TCP and IP header information from the sender host with a “signature” database containing specifics of the target host. The most common header identifiers are window size, time-to-live, DF bit, and total length:

•  Window size (wSize) refers to the size of the incoming packet buffer. The operating system typically sets the wSize parameter at the beginning of a TCP session. Most Unix operating systems such as Linux or Solaris keep the wSize parameter static through the TCP session, but Windows operating systems tend to change window size value during the session.

•  Time-to-live (TTL) is another parameter that provides useful insight into the host operating system. Operation systems set different defaults in the IP header value for the number of hops a packet may take before it is dropped. For example, a TTL of 64 would indicate that the host is likely running FreeBSD or Linux, and a TTL at 128 would indicate that the host is probably a Windows box.

•  The don’t fragment (DF) flag is not quite as useful an indicator as the previous two parameters because most operating systems will set the DF flag by default. However, it might be viewed as a “by exception” indicator because only a few operating systems such as OpenBSD actually leave the DF flag unset.

•  Total length is a field that indicates the length of the packet, including IP header and payload. It serves as a valuable identifier because some operating systems can be denoted by the default total length value of the SYN and SYNACK packets. These are the total packet lengths for each primary operating system: Linux (60), Solaris (44) and Windows 2000 (48).

None of these parameters can determine the sender hot profile on their own. But when an IDS collectively assesses these indicators against an operating system’s fingerprint signature database, the analysis often accurately determines the host operating system and services.

For example, the IDS fingerprinting engine examining a packet with a TTL of 64 will narrow down the possible operating systems  to either Linux or OpenBSD because both share identical TTL values. However, validating the wSize value will differentiate between Linux and OpenBSD. The combination of indicative parameters is what offers a reliable “signature” of the operating system in question.

By accounting for whether the targeted host is vulnerable to a given attack before generating a security alert, false positives are dramatically reduced.

The result is a new generation of IDS products that deliver more accurate detection.