Q&A: Cisco engineer on LANs, NAC, network design

James Henry Carmouche might have already built and tested the network you will soon be deploying. As technical marketing engineer in Cisco’s Enterprise Systems Engineering group, Carmouche figures out the best way to put together the products Cisco makes, then builds and validates network reference designs for campus LANs, VPNs, network admission control and convergence. The CCIE-certified engineer also consults with customers on how to implement Cisco reference designs, and wrote the book “IPSec Virtual Private Network Fundamentals.” He took time out this week to discuss the trends he’s seeing in network designs in between his “Meet the Engineer” one-on-one sessions at Cisco’s Networkers show this week.

James Henry Carmouche might have already built and tested the network you will soon be deploying. As technical marketing engineer in Cisco’s Enterprise Systems Engineering group, Carmouche figures out the best way to put together the products Cisco makes, then builds and validates network reference designs for campus LANs, VPNs, Network Admission Control and convergence. The CCIE-certified engineer also consults with customers on how to implement Cisco reference designs, and wrote the book “IPSec Virtual Private Network Fundamentals.” He took time out this week to discuss the trends he’s seeing in network designs in between his “Meet the Engineer” one-on-one sessions at Cisco’s Networkers show this week.

What design issues or problems are customers bringing to you for help?

It varies so wildly. In the “Meet the Engineer” sessions, there’s been so many [different issues]. From an IPSec standpoint… as the network becomes more diverse in what it can offer, in terms of voice and multicasting and video and all different other types of business networking applications, the encrypted infrastructure and encrypted technologies need to be able to support that. So we’re constantly looking at innovations into IPSec and cryptography to enable that support seamlessly and scalably.

It comes down to helping customers understand what are the issues with IPSec. What is the number of tunnels [they] can support. Is it the bandwidth you get through? It really comes down to packets per second – encrypted packets per second. There are all different kinds of [things] that impact [the performance] of encrypted packets per second – how it’s switched in the hardware, in the data-forwarding plane. [This] impacts the scalability of the design.

It’s a continuous drive, as you converge applications on [encrypted networks], to understand the scalability and functionality of the VPN. So that’s where we play, to keep that awareness up, to speed the rate of adoption with customers.

Does this involve deploying new gear, or reconfiguring or adjusting switches and routers that are already deployed?

As bandwidth increases, the theory is also that the switching capacity – or the equipment that supports that bandwidth – is going to increase. What we’ve seen with convergence is smaller packet sizes. We’ve seen situations in which a small bandwidth pipe is now receiving small packets across it, so in other words the bandwidth doesn’t increase, but the number of packets that are going through are increasing, because we’re sending smaller packets. So you have to be able to switch that traffic very fast, and predictability with low jitter and all that good stuff.

What other reference designs are you working on to integrate infrastructure and security?

My most recent project was with the Network Admission Control initiative. That’s a shift in gears for me from the IPSec world to NAC. Over the past few years, I’ve been trying to understand the impact of integrating Network Admission Control into a recommended best-practice campus architecture. The team I work for has put out several designs that pertain to convergence. High-speed convergence in the campus is very important, with scalability and security services in terms of hardening the infrastructure.

We have Catalyst integrating security features that prevent Layer 2 attacks. NAC is another initiative that we have that essentially allows the network to understand the software compliance level of the end station that attaches to it.

What are the challenges in deploying NAC? Are there issues with performance, or added complexity?

The first thing is that it requires a certain degree of intelligence on the end station right now. So we have something that’s called the Cisco Trust Agent that we embed there. In terms of campus design, there are varying levels of complexity. In a lot of areas, there is not a lot of impact to the [network] administrator. [NAC] is a feature that you can turn on in the switch or in the infrastructure that will help prevent against certain attack vectors.

In other areas [the challenge is] integrating security technologies in, but [not] at the price of convergence [meaning, resiliency and failover capabilities]… or inserting overhead [into the network].

[The danger is in] inserting security products into the data plane… that essentially switch packets slower, in order to make the network more secure… We’re doing a lot of [testing] to make sure that the convergence is optimal – that we can design the network in a way that still achieves high availability, but with increased security. That’s the biggest challenge. Maybe challenge isn’t the best choice of words; [but] it’s something we have to validate and prove to our customers. That’s kind of what my job is.

How are customers approaching NAC? Are they asking for the technology?

We have reference designs in our group. The job is to build reference designs to make services available to customers at a high degree of availably. So the conversation starts with, essentially: I have a security problem, so I want to enforce software compliance on the network. Some customers have a widely varying populace of end stations that attach – guest users, contract users attaching to the network, all with different requirements… [they] want to be able lock that down. So, [customers] will voice that problem.

Then [the discussion] turns into, OK, so now how do we integrate it. How do we integrate the appropriate technology to solve the problem, without impacting the availability of the existing business-critical services.

How much of a challenge is it to deploy NAC in a heterogeneous environment?

We have different solutions. We have what’s called the NAC appliance. That came to us by an acquisition. That supports a more heterogeneous environment. There’s also 802.1x; we support that IEEE standard. That’s not NAC, it’s an IEEE standard.

So NAC is one tool for securing the campus; what are some other tools?

It’s integration with the Catalyst integrated security feature set. 802.1x, obviously. It’s a part of NAC L2 [the technology that integrates security into switching] that relies on the functionality of 802.1x. It’s the integration of all those things. Catalyst integrated security consists of a few key features: DHCP snooping; Dynamic ARP inspection; IP Source Guard, and some others. We’re looking into integrating that, and ensuring that when you turn these things on together, that you’re not impacting the availability of the network. So, you can secure the network, and it’s as available as it was before.

How does the commoditization of LAN and campus switching technology affect the design of networks? Could you implement a lot of the network reference designs with any LAN equipment?

My view is that [with] commoditization… there is just additional pressures to innovate constantly. There are additional vendors that are doing Layer 2. There are standards that standardize a lot of the technologies there… A lot of the view out there right now is that [you] have things like spanning tree, for a Layer 2 LAN… 802.1x.

But there are other things, such as extending routing protocols to the closet, where you can get higher convergence, or comparable convergence with better manageability. Those are things that are differentiators. NAC is a differentiator. Catalyst integrated security is a differentiator. These are new innovations that we keep looking at to try and solve business problems. As long as customers have business problems, and have a need for campus networks and LAN switching, then there will always be opportunities to innovate and grow.

In that case, with commoditization, unless the standards bodies can keep track and essentially handle all innovation, which is kind of hard to do, there’s always going to be space to innovate. And that’s what we’re trying to do [to stay in front of the standards].

A lot of the stuff [we come up with] eventually makes it into the standards bodies. But being out on the forefront kind of helps us stave off the commoditization of [LAN technology] … [We try to] to build designs that are not necessarily cookie-cutter designs, but designs that are manageable, deployable, and that solve business problems.

Is the wider adoption of 10G Ethernet changing the way people are thinking about architecting their LANs? For example, some vendors and analysts have talked about eliminating the aggregation layer between wiring closets and the core and using 10 Gigabit Ethernet for that link.

That sounds like a collapsed core design. The three-tier model [with a core, aggregation layer and wiring closet or access layer] is still something that’s popular for services integration – services such as QoS, control-plane policing, and [security services such as] scavenger QoS, which essentially is a means by which to aggressively draw traffic that is anomalous [and potentially threatening].

So, you have ability to separate those things out from your core. If you have a core distribution and access layers, traditionally you have Layer 2 access, you have the distribution blocks that are consolidated right there. Then you have the core, which is essentially focused on high-speed packet processing that’s shared across many different distribution layers.

Converging all of those services tends to complicate the core/distribution model. Having those things out in the distribution layer is just a little bit cleaner. It kind of allows you to design the network in a modular way. In the campus, all the reference designs have three-tier models.

What are some new trends you’re seeing, or new ideas you are pushing in terms of campus LAN design?

One of those twists or trends is a migration towards getting back to routing protocols in the access layer. You still have the Layer 2 to 3 boundary, it just exists at the access layer instead of the distribution layer. That allows you to distribute services out to the edge, instead of consolidating them into one area.

Now you have a three-tier design that gives you comparable convergence with what you had before. With proper tuning and configuring, [it] probably has a slight advantage towards getting [traffic] to converge quicker [in case of a device or link failure].

Running OSPF [Open Shortest Path First] and EIGRP [Enhanced Interior Gateway Routing Protocol] from the [LAN] access layer into the network is something that folks are investigating. Today it’s been what’s mostly referred to as multilayer design. It’s hierarchical and modular, with the core, distribution and access layers. But traditionally in the multilayer design, it’s been Layer 3 from core to distribution, and then from distribution to access it’s been Layer 2.

That introduces things like spanning tree. We can converge quickly with things like spanning tree and [Rapid-Per-VLAN-Spanning Tree, or Rapid-PVST]. But with proper tuning of the routing protocols, we can get even faster convergence. [This] is one of the most important things in the design of the campus – the ability to fail over quickly. So, if I have a phone call, I’m not getting dropped, or getting jitter. We want to make sure that any delay attributable to a failover situation won’t cause poor [service] quality.

What are some other technologies or protocols that may not be used in the access layer that now that you’re advocating to network designers?

When you push routing protocols to the edge, there are services that come along with it. The control plane moves down. There’s an impact to QoS in the way that’s designed. Control plane policing is essentially policing traffic ingress into the control plane. In situations where the control plane is unavailable [and a switch or router is not responding], the administrators cannot get to the box. Being able to police traffic on ingress into the control plane allows the routing protocols to stay up – things like SSH and management protocols [also] stay up, so you can actually get to the box and recover it.

Are there any complications or downsides in pushing Layer 3 protocols to the access layer?

It actually makes things a little bit easier, because routing protocols are a well-adopted technology… The debugging suite is mature. The diagnostic output is mature. So you have these control troubleshooting and diagnostics available.

Before, you were running a Layer 2 network in the wiring closet, you had spanning tree to contend with and you had to designing the loops properly. Spanning tree doesn’t have the level of diagnostics that a Layer 3 feature set does. Having that out in the closet makes it easier. You also get rid of your default gateway redundancy schemes.

In a multi-layer model, where you have an access layer talking to two distribution layers, [each access-layer switch] is all one subnet. So, default gateways are up [in the access layer], being shared [among access-layer switches]. But in a routing protocol situation, I’m laying the routing protocol over everything. I don’t need to configure things like [Hot Standby Router Protocol] or spanning tree anymore, because there are no more loops. The Layer 2 boundary is contained to that single access-layer switch.

This sounds technically superior to having traditional Layer 2 switches in the access layer, but does this justify the cost that might be involved in upgrading an entire access layer to Layer 3 software feature sets?

I can’t answer in terms of exact price of the IP feature set. But what I can say is that we’ve made an attempt to include the functionality needed to support this [Layer 3 design] in the IP-based feature set. So that it makes it attractive to a customer that’s running Layer 3 in the wiring closet. Keeping the number of routes down in the closet, running stub functionality in the wiring closet is there in the IP-based feature set. [A stub router is a Cisco IOS feature that will only advertise the availability of a limited set of configured routes, rather than the entire routing table]. These things can help reduce the complexity of running Layer 3 in wiring closets, which could help lower costs.