Adam Powers CTO of Lancope: Top 5 uses of NetFlow

Picking the mind of an extremely talented technologist is the objective of this new blog series. The following Q&A with Adam Powers of Lancope is the first of a multi-part series on the Top 5 Uses of NetFlow. For the first story, Adam Powers - CTO of Lancope, has answered the questions of yours truly below to discuss NetFlow Event Logging and how it compares to Syslog.

Section I: NetFlow Event Logging

1. What types of messages used in Syslog do you feel will move to NetFlow?

High speed structured events such as firewall logs primarily.

NetFlow Event Logging (NEL) is found in Cisco’s newest high speed firewall technologies such as the new ASR-1000 and the venerable ASA 5000 series.

Let’s take a look at the ASR-1000 as an example.

The ASR offers an integrated, zone-based IOS firewall capable of speeds in excess of 2Gbps.

At these rates, firewall state table session setup and teardowns occur at incredibly high rates.

Customers with even trivial logging requirements will find that traditional syslog incurs a significant load on the ASR-1000’s route processor.

Cisco’s ASR (Aggregated Services Router) is designed to run advanced services at the network’s edge.

It follows that customers would want to use the route processor’s available horsepower to accommodate those advanced services rather than spend its time manufacturing and sending syslog.

When using NetFlow Event Logging in the ASR, firewall logging is pushed down into hardware freeing the Route Processor to perform higher order tasks.

The creation of hardware-based NetFlow v9 is much less expensive than the creation of software-based syslog.

Unfortunately NEL technology is relatively new and support for it is sparse at best.

Customers wanting to reap the benefits of NEL will find that very few if any syslog aggregation and collection technologies simply do not support NEL.

Working with Cisco engineers, Lancope has introduced a simple 1U appliance called the NEL Gateway to address this issue.

The StealthWatch NEL Gateway leverages Lancope’s knowledge of high speed NetFlow processing to create a simple protocol gateway that converts the high speed NEL events into C3PL compliant syslog messages suitable for export to most any traditional syslog collector.

Each 1U NEL Gateway appliance can process up to 40,000 firewall events per second, translating them into equivalent syslog messages and forwarding them to one or more syslog collectors using the original source IP of the sending ASR.

The diagram below provides a overview of how the ASR and the NEL Gateway work together...

Within the ASR-1000, the following firewall event types are exported over NEL and translated into syslog by the NEL Gateway:

The screenshot below shows the status page for the NEL Gateway. Note the counts shown on a per record type basis.

One last important note: Due to the efficiency of NetFlow v9, NEL is up to 18x more efficient in terms of bandwidth generated through logging.

Given an Ethernet MTU of 1500 up to 18 NEL events are packed into a single NEL UDP datagram.

Section II: Flexible NetFlow

2. When does Lancope plan to have flexible NetFlow phase 2 implemented?

Lancope has already begun to support additional key and non-key fields provided in v9.

Our next major improvement to Flexible NetFlow support should arrive in the Q1 timeframe of 09.

This release is being timed in conjunction with some very interesting Flexible NetFlow enhancements we’re not even allowed to discuss as of yet.

The trouble with Flexible NetFlow has been that there were too few real benefits provided by FNF (Flexible NetFlow) over TNF (traditional NetFlow).

That is changing very rapidly thanks to new key/non-key’s being added to FNF.

3. What types of cache messages (normal, immediate, permanent) do you plan to accept initially and when?

The StealthWatch NetFlow collector has a minimum set of fields required to operate.

As long as the Flexible NetFlow exporter sends at least the minimum set the cache mechanism and cache timers can be configured any way the customer desires.

The minimum required fields for the StealthWatch flow collector include:

FIRST_SWITCHED
LAST_SWITCHED
IN_BYTES
IN_PKTS
SRC_INT_ID
DST_INT_ID
PROTOCOL
L4_SRC_PORT
L4_DST_PORT
SRC_ADDR_IPV4/IPv6       (IPv6 support added in 5.8 release)
DST_ADDR_IPV4/IPv6

For those customers wanting to reduce the amount of bandwidth consumed by NetFlow to an absolute minimum, creating a flow template with these the set above will result in a very efficient NetFlow export.

Immediate and permanent cache settings can be used with the current release of StealthWatch so long as they include the fields listed above.

Automated cache configuration via SNMP PUTs are a bit further out.

The jury is still out on how useful automation of cache creation will be in an real-world environment.

4. Will you be archiving these different caches where applicable and rolling up the data into 10, 30, 1 hour etc. intervals?

Definitely.

This is already in the product.

The StealthWatch System provides a retention management system that allows the operator to specify how long they would like to keep various statistics derived through NetFlow analysis.

StealthWatch tracks interface utilization statistics from 1 minute up to 1 day resolution.

Statistics are "rolled up" over time in accordance with the retention policy configured within the system.

The screenshot below gives you an idea of how this retention manager works...

5. Will you be triggering and sending the SNMP sets necessary to setup the different caches above?

Lancope is researching the usefulness of automated cache creation but has held off on a firm date commitment for this kind of functionality.

Instead we’re focusing our engineering efforts on the newer key/non-key fields provided by FNF.

We’re interested in how we can use them and what additional fields we might persuade Cisco to add in the future.

As FNF gets more real field use we’ll get a better understanding for how automated cache creation should work.

For more information, feel free to contact me offline at: apowers@lancope.com

The Adam Powers Lancope NetFlow Blog Series:

Part Two

Use NetFlow for visibility into network blind spots: Virtualization, 10Gb+ Ethernet, MPLS and Multipoint VPN

What questions would YOU like Adam Powers to answer?

http://www.BradReese.Com

Search 26,983 current Cisco Job openings worldwide.

	JDSU 4th Quarter 2008: Expect strong outperformance for its advanced optical technologies (AOT) segment
	CCIE water cooler gossip: CCIE agent musical chair game?
	CCIE water cooler gossip: CCIE agent rules of the road?
	Brocade 3rd Quarter 2008 results: Cisco long-term threat to Brocade's Fibre Channel switching franchise
	CCIE water cooler gossip: Is a rosy future in the forecast for sitting CCIEs?
	Brad Reese on Cisco Story Archives