In the previous article we covered how NetFlow, IPFIX and the variety of other flow export protocols can be used to give us some application traffic visibility. However, nothing gives as much detail as raw packet decode. A new protocol called AppFlow may hold some promise of giving administrators the data they need while still working in a hybrid topology environment. AppFlow, which if compatible with IPFIX, provides those application protocol details and as more vendors embrace this protocol it will become more popular. AppFlow may strike just the right balance between providing application-layer details with the performance and flexibility of a flow export protocol.
AppFlow is a flow export protocol that not-only provides details about the TCP connection that has been made, but it also includes details about the application connection. AppFlow provides information on the TCP connection (e.g. source/destination IP address, source/destination TCP port number, flow volume, timestamp) which is similar to NetFlow and IPFIX. In fact, AppFlow is compatible with IPFIX. IPFIX allows vendors to use a Vendor ID to create their own proprietary information to be exported. The vendor can put anything they want into that export and IPFIX supports variable length fields (NetFlow does not). AppFlow is just a form of IPFIX that is focused on application data. Furthermore, AppFlow uses a different name so it gives people the impression that it is completely different than IPFIX.
Application-layer data is also exported in the AppFlow flow data. AppFlow also provides information on the HTTP information and other application performance data. AppFlow flow records include information like Round Trip Time (RTT) and protocol latency. AppFlow records also include information about the HTTP URL, HTTP request methods and response-status codes. AppFlow also uses Transaction ID, Connection ID and Custom IDs.
There are two components of the AppFlow architecture: a generator and a consumer. Think of the generator like a router that might have traditionally sent NetFlow/IPFIX data. The consumer can be thought of as being similar to the NetFlow/IPFIX collector or analyzer computer/service. The AppFlow generator gathers up information about application flows and then sends them to the consumer for aggregation, display, analysis, and archival. The AppFlow generator/consumer terminology is similar to the NetFlow/IPFIX exporter/collector architecture. Different names, but the same concept and function.
One of the other benefits of AppFlow is that it can work on cloud-based systems. It does not require a physical tap, SPAN, port mirror, or physical probe. AppFlow also does not require an agent to be installed on a physical or virtual server. Therefore, AppFlow may be just what administrators are looking for to give them visibility into their hybrid cloud environments.
A key benefit to using AppFlow is that any standard NetFlow v9 or IPFIX collector can also be adapted to parse AppFlow data. AppFlow transmits the flow data using UDP port 4739 with the IPFIX protocol format. So long as the collector has the ability to parse the additional AppFlow information and analyze the data within, the AppFlow data can be analyzed by an IT administrator. In fact, AppFlow is also sometimes referred to as IPFIX Extended.
Initially there was a version 1 of the AppFlow specification, but as of May 2012 the AppFlow version 2 is now the current edition. The AppFlow protocol is documented and maintained by the AppFlow organization on their web site appflow.org.
Implementations of AppFlow:
AppFlow may be somewhat new, but it is being supported by an increasing number of vendors and gaining acceptance with customers. Also, vendors that support IPFIX today could work with AppFlow. Following is a list of the dominant vendors that are supporting AppFlow in their products.
Citrix is one of the vendors that has been the most vocal and proactive about AppFlow and has been advocating the use of the protocol. There is broad support for AppFlow on Citrix NetScaler line of products. A Citrix NetScaler Application Delivery Controller (ADC) can be an AppFlow generator. AppFlow is also supported on the Citrix NetScaler 1000V ADC that is integrated with the Cisco Nexus 1000V.
Splunk is the “Cookie Monster” of IT and machine data. Splunk can be a consumer of AppFlow data and analyze the trends and pull intelligent results out of the volumes of data. There is a Splunk App for collecting AppFlow data from Citrix NetScalers. There is also a Splunk App for Citrix CloudBridge which would be useful in a cloud-based topology where a packet capture would be difficult if not impossible to perform.
Other companies that make software that can be AppFlow consumers are:
As mentioned before, any IPFIX-capable system could work with AppFlow. AppFlow and IPFIX are very much interoperable and compatible.
There are many flow-based network monitoring protocols to choose from. Having too many choices can be confusing to consumers and lead to “analysis paralysis”. It is helpful to know the differences and similarities between these flow-export protocols. However, most of these protocols only send information about the higher-level characteristics and less-granular details of a connection. Nothing beats raw packet captures, so long as the data is not encrypted, but those are not typically feasible or always available. Packet captures are also not a solution for longer-duration analysis.
If you want to get more detailed flow information without having to break out the protocol analyzer, you should explore AppFlow and how IPFIX can be extended. Ask your vendor if they support AppFlow and/or IPFIX and if not, when it is going to be prioritized on their product road-map. AppFlow strikes the right balance between having a protocol that can work in many topologies, have good performance and low impact, and still give you the application visibility you require.