Riverbed wins 7-vendor WAN optimization test

Challengers Ipanema and Exinda score high marks for innovation

1 2 3 4 Page 3
Page 3 of 4

We looked briefly at Exinda’s analysis tool, NRC (an OEM product from Vineyard Networks), but they told us that the tool will be replaced by a new central management and analysis platform at year’s end. This leaves Exinda x800-series at a disadvantage, between long-term flow analysis platforms, compared to Riverbed Steelhead, Blue Coat Mach5, and Silver Peak VX-series.

Along the way, we tripped across a nifty feature in Ipanema and Exinda products (and available as part of Riverbed Cascade): application performance monitoring. With application monitoring, network optimization appliances can become first steps at answering the question: “Is this a network or a server problem?” For example, Ipanema ip|engine appliances and Salsa management tools collect network delay and jitter, packet loss rates, server response time, and TCP retransmission statistics for each defined application. This fits beautifully in Ipanema’s model, which is all application-based, and allows the Ipanema Salsa management system to report on overall user experience. Since one of the main reasons to install network optimization is to improve end-user experiences, application performance monitoring struck us as a great complementary feature. 


When WAN branch offices have multiple links for reliability, network managers understandably want to use every bit of outbound bandwidth they’re paying for to improve performance. Outbound load sharing is a feature common in branch office firewalls, but the load sharing is usually fairly simplistic. Recently, the term “WAN Path Selection” has been used to talk about a more sophisticated type of outbound load balancing. 

With WAN Path Selection, the outbound path for a network flow is selected based on configured-in application performance goals and constantly measured link information. For example, if video conferencing is a business critical application requiring low jitter and high throughput, WAN Path Selection might pick whichever link happens to have the lowest jitter and highest throughput at the moment. And, while the videoconference is going on, WAN Path Selection might also keep other flows from running over the same circuit.

Because the Network Optimization devices we tested all include some type of traffic management controls, WAN Path Selection is an obvious additional feature to help improve overall network performance. 

WAN Path Selection is such an interesting feature that when we started our testing, only one product supported the option (Ipanema ip|engine) and by the time we were finished writing, both Cisco and Riverbed had added it to the products we were testing. Since only Ipanema ip|engine included WAN Path Selection when we started, we did not do rigorous testing. Network managers should also note that the comparative pricing for Ipanema’s product provided in this test does not include the additional license required for Dynamic WAN Selection (Ipanema’s name for the feature), at $8,700 for the central site and $725 for each branch office.

While WAN Path Selection sounds like an interesting idea, when we looked deeply at how the network optimization vendors were implementing it, we found some significant gotchas. The biggest issue is that WAN Path Selection can only really be properly done in the edge firewall device, not a network optimization device that sits behind the firewall. The conflicting goals of the optimization and the firewall are to blame here. 

Normally, the network optimization device sits inside the firewall (towards the LAN side of the branch office). This is because the firewall will have VPN tunnels to the rest of the network, and the optimization device can only work effectively on unencrypted traffic. However, for the network optimization device to perform WAN Path Selection directly, it has to be outside the firewall (towards the WAN side of the branch office) and in-line with each link leaving the branch office. 

Working around this fundamental problem has smart people thinking overtime about potential solutions. The Ipanema ip|engine has a Rube Goldbergian scheme to perform WAN Path Selection by indicating a preferred route to the upstream firewall or router using some of the DiffServ bits in the IP header or hard-coding MAC addresses of upstream routers -- solutions that can cause more problems than they solve. Riverbed’s engineers suggested a different approach of running the traffic through the network optimization box twice, straddling the firewall. Inside the firewall their device would optimize the traffic and mark priorities. Outside the firewall, the traffic would pass through again, and the Steelhead could look at the markings left over from the first pass to shape the traffic and control WAN routing.

In the short term, until companies such as Ipanema can actually fully replace edge firewall/router devices, the only real contender in this space is Cisco with a fully integrated router, firewall, and network optimization device in one chassis. Since ISR-integrated version of Cisco’s WAAS has a similar technology (Cisco’s name for it is Cisco Performance Routing, which they acronymize as PfR), the integrated ISR can make a WAN path selection routing decision, apply traffic management, and then pass the network flows through the WAAS engine for optimization.

Some network managers with different network topologies from the one we used may find that Ipanema ip|engine and Riverbed Steelhead work great out-of-the-box. However, WAN Path Selection introduces a very different paradigm for these devices, because all but Cisco ISR-integrated WAAS currently operate as a “bump in the wire.” Considerable engineering and expertise has gone into making the bump very innocuous. When a device suddenly starts looking like a router more than a bridge, that’s a very different kind of design and very different behavior that is incompatible with most current deployments. You’re throwing out a lot of experience and knowledge and venturing into unknown territory.

For this reason, we’d advise extreme caution when looking at this feature. Fundamentally, IP routing is incompatible with the idea of keeping a single flow running across a particular network path, especially when asymmetric routing is possible. This doesn’t mean that WAN Path Selection can’t work, just that the places where it can work may be very limited or extremely restricted network environments.

WAN MANAGEMENT: Cisco, Ipanema, Riverbed offer strongest tools

We prepared a set of testing criteria for enterprise suitability but found little difference in products except when it came to management. Areas such as high availability and clustering were all nearly identical.

When we looked at SNMP support we also found some variation. For example, Citrix left out SNMP v3 support and Ipanema doesn’t approve of SNMP for device management, sending you to the global management system instead. Our testing shows that Riverbed Steelhead has the most sophisticated SNMP management available in its devices. But while these differences indicate a relative lack of maturity in some products, it’s hard to get too excited about these differences.

Instead, we chose to focus on centralized management and reporting. We found a dramatic spectrum from almost nonexistent (Exinda) to nearly unusable (Blue Coat and Citrix) to chaotic (Silver Peak) to comprehensive and well-thought-out (Cisco, Ipanema, and Riverbed). 

In many cases, we also found a strong distinction between performance monitoring and network management, often with completely different products needed to solve the two problems. We found that all products have a very low management requirement: get them set up, and as long as your environment doesn’t change, you really don’t have to touch them.

Certainly the strongest management tools came from Cisco and Ipanema, with Riverbed a very close second place. Cisco and Ipanema both take a systemic approach to management of network optimization, meaning that you rarely have to dive into individual device configurations. Instead, you configure a general set of policies that apply across the entire deployment and the management systems take care of keeping configurations in synchronization. 

Ipanema’s Salsa management tool is definitely the most sophisticated in the network optimization market, automatically handling a wide variety of configuration tasks focused on application performance. With Salsa, the network manager lays out applications on the network, bandwidth required for each, and, well, that’s about it. Everything else is automatic. However, Salsa is missing other types of management that we’d expect at this level. For example, software updates are unreasonably complicated, especially for a company making rapid changes to their software. Salsa goes way, way beyond what everyone else does in this space, but then it hits the wall so severely that it makes a dent where other areas, such as element management and reporting, are concerned. 

Cisco’s management tools also take a desirable systems approach to setting up network optimization, although this approach is being fragmented by Cisco’s new ISR-integrated approach. With standalone Cisco WAVE boxes, a single management tool, the WAAS Central Manager, is all you want and all you need. For a company that has consistently had problems with network management, the WAAS Central Manager is amazingly good and stands out as one of the strongest parts of the product and one of the most elegant management systems in this space. 

However, when the ISR-integrated WAAS is used, Cisco now requires two management systems: the legacy WAAS Central Manager for basic WAAS acceleration operations, and Cisco PRIME for management of the IOS wrapper that handles other parts of network optimization, such as traffic management and visibility. Cisco PRIME lacks the smooth polish of the WAAS standalone manager, and because it is early in PRIME’s life cycle, still has a number of glitches. Network managers selecting Cisco PRIME should plan on losing time to minor bugs and an aggressive upgrade cycle the first few years.

Riverbed’s Central Management Console is more element-focused than either Ipanema’s Salsa or Cisco’s WAAS Central Manager tool, but is a very strong, stable, and capable management system. With full coverage across most product features in a single tool, Riverbed has the strongest and most consistent management system. The lack of a “full network” view of policy and traffic management will create extra work for network managers, but the stability, solidity, and comprehensive view of each device makes up for this deficit. Any network manager choosing Riverbed Steelhead for their network optimization will be impressed by the quality of the management system, including ease of use and consistency.

Compared with Riverbed, Ipanema and Cisco, the rest of the network optimization products we tested look amateurish in their management systems. Silver Peak decided that it was unhappy with its existing Global Management System (GMS) management tool and re-wrote it, but not completely, leaving network managers to run two separate clients (one in Java) with very different interfaces in order to get any real work done.  

Citrix didn’t even bother to write a real management system for its network optimization devices -- it had one left over from its load balancer and grafted the network optimization configuration on top of the existing tool, leaving much of the management system and reporting engine dangling in space without really restructuring to handle the network optimization case. Similarly, Blue Coat re-purposed its proxy management tool with a bevy of obscure details and confusing sub-options to handle the network optimization product. 

Exinda’s management, like Silver Peak’s, is in transition except that Exinda is nowhere near as far along as Silver Peak. Instead, Exinda has a cobbled-together a set of third party, cloud, and on-premises solutions which together don’t cover the bases. Network managers choosing Exinda should be prepared for a lot of command line configuration until Exinda’s new management system comes out. And from the preview that Exinda showed us, the new management system is not going to be complete anytime soon. Reporting is in good shape in the beta we saw, but configuration is non-existent. 

EASE OF USE: Riverbed rules

As an early master of the field of WAN compression and de-duplication, Riverbed set the stage for the marketplace with products that are both flexible and easy to use. Riverbed Steelhead devices don’t require complicated configuration to define network topologies. Instead, if you drop a Steelhead in-line with an existing network connection, it runs with almost no programming and no continuous management or updating. 

With Riverbed Steelhead setting the bar, everyone has fallen into place and done a pretty good job. The laggards here are Silver Peak and Cisco.

With Silver Peak VX-series, the connection between two devices is more explicitly managed because a tunnel is always established between devices. This means that you have to put VX-series outside of any firewall function, because a firewall on the WAN side of the VX-series cannot apply normal controls. Silver Peak VX-series will automatically create tunnels if you want, but then the tunnels have to be managed if you want to use traffic management features, creating an additional and unnecessary burden, especially in a large WAN. Silver Peak’s heritage as a data center-to-data center company, where the number of devices in the network is tiny, is showing through here with this design defect.

1 2 3 4 Page 3
Page 3 of 4
The 10 most powerful companies in enterprise networking 2022