Devices—How is traffic separation maintained internally to a device? What are the primitives used for Layer 2, Layer 3, or Layer 4 traffic?
Data path—How is traffic separation enforced across a network path? What tools are available to maintain the separation across a network?
Control plane—Because data-path virtualization essentially builds an overlay topology, what changes are needed for routing protocols to function correctly?
This chapter does not cover architectures, topologies, or designs. The purpose of this chapter is to make sure you understand the technology before using it. (The world would be a better place if this principle were universally applied).
The general format for each section of this chapter is to start with a discussion of the protocol technology, highlighting any important details, with some limited configuration examples.
Network Device Virtualization
One of the characteristics of a VN is that it provides what are essentially private communication paths between members of a group over a shared infrastructure. This creates two requirements for the network infrastructure:
Traffic from one group is never mixed with another—For sending and receiving traffic over shared links, tunnels (many borrowed from existing virtual private network [VPN] solutions) can guarantee data separation. Network devices need to enforce group separation in their internal memory (for example, during routing table lookups, access lists processing, or NetFlow statistics gathering).
Each VN has a separate address space—This requirement is derived from the fact that VNs offer the same characteristics as a physical network. Address space and forwarding within it are two of the most basic aspects of any network.
Note - In this section, we start with a narrow definition of device virtualization as creating a separate address space within a network device. However, you can think of this as a special case of a more general definition of a virtualized device, which is the ability for an administrator to allocate device resources to different uses. The first device resource we consider is address space, but as we proceed through both the chapter and book, we include different layers of device policy control mechanisms, such as quality of service (QoS) and security rules. Therefore, the final picture of a virtualized device will be much closer to the general definition.
The first problem to solve is how to virtualize the forwarding plane in a way that meets the requirements for address and traffic flow separation. Depending on the type of device, the virtual separation can go by the following names:
Virtual LAN (VLAN)
Virtual routing and forwarding (VRF)
Virtual forwarding instance (VFI)
Virtual firewall context
Layer 2: VLANs
VLANs are a good example of a piece of the virtualization puzzle that has been around for quite some time. A VLAN is a logical grouping of ports on a switch that form a single broadcast domain. Ports in a VLAN can communicate only with other ports in the same VLAN. How a given switch does this is implementation dependent, but a common solution is for the switch to tag each frame with a VLAN number as it arrives on a port. When a frame is sent to other ports, the output hardware copies the packet only if it is configured with the VLAN number carried in the frame.
On an Ethernet switch, there is typically a single MAC table, which maps ports to MAC addresses. To support VLANs (and simple Layer 2 virtualization), the MAC table has a field for the VLAN number on which the station was discovered, as demonstrated in Example 4-1.
Example 4-1 Switch MAC Table
Switch# show mac-address-table ... Non-static Address Table: Destination Address Address Type VLAN Destination Port ------------------- ------------ ---- -------------------- 0010.0de0.e289 Dynamic 1 FastEthernet0/1 0010.7b00.1540 Dynamic 2 FastEthernet0/5 0010.7b00.1545 Dynamic 2 FastEthernet0/5 0060.5cf4.0076 Dynamic 1 FastEthernet0/1 0060.5cf4.0077 Dynamic 1 FastEthernet0/1 0060.5cf4.1315 Dynamic 1 FastEthernet0/1 0060.70cb.f301 Dynamic 1 FastEthernet0/1 00e0.1e42.9978 Dynamic 1 FastEthernet0/1 00e0.1e9f.3900 Dynamic 1 FastEthernet0/1
Note - Note that the output in Example 4-1 was taken from http://www.cisco.com/en/US/products/sw/iosswrel/ps5207/products_command_reference_chapter09186a0080417db7.html#wp1021274.
The summary effect of the VLANs is to partition the switch into logical Layer 2 domains. Each domain has its own address space and packets from one domain are kept separate from those of another.
Layer 3: VRF Instances
VRFs are to Layer 3 as VLANs are to Layer 2 and delimit the domain of an IP network within a router. The Cisco website has a more formal definition:
VRF—A VPN Routing/Forwarding instance. A VRF consists of an IP routing table, a derived forwarding table, a set of interfaces that use the forwarding table, and a set of rules and routing protocols that determine what goes into the forwarding table.
Unlike the VLAN scenario, where an extra column in the MAC table is adequate, a VRF partitions a router by creating multiple routing tables and multiple forwarding instances. Dedicated interfaces are bound to each VRF.
Figure 4-1 shows a simple logical representation of a router with two VRFs: RED and GREEN. The RED table can forward packets between interfaces E1/0, E1/2, and S2/0.102. The GREEN table, on the other hand, forwards between interfaces E4/2, S2/0.103, and S2/1.103. An interface cannot be in multiple VRFs at the same time.
Multiple VRFs on a Router
You can see in Figure 4-1 how a VRFs provide separate layer paths between routed interfaces. RED packets can never end up on a GREEN interface.
Note - There is a way to share routes between VRFs, but that is beyond the scope of this introduction. Interested readers should consult the references in the appendix for specialized texts on Multiprotocol Label Switching VPNs (MPLS VPNs).
It is easy to see VRFs on a router. Example 4-2 shows the RED and GREEN VRFs, with their interfaces. Example 4-2 shows overlapping IP addresses on the Serial interfaces. Example 4-3 gives an interface-centric view of the same data.
Example 4-2 Displaying VRFs
Router# show ip vrf Name Default RD Interfaces RED 100:1 Ethernet1/0, Ethernet1/2, Serial2/0.102 GREEN 100:2 Serial2/0.103, Serial2/1.103, Ethernet4/2 Example 4-3 Displaying interface to VRF mapping Router# show ip vrf interfaces Interface IP-Address VRF Protocol Ethernet1/0 130.22.0.33 RED up Ethernet1/2 130.77.0.33 RED up Serial2/0.102 130.77.0.33 RED up Serial2/0.103 130.77.0.33 GREEN up Serial2/1.102 130.77.0.33 GREEN up Ethernet4/2 130.22.0.33 GREEN up router#
FIBs and RIBs
Before looking at the routing information for a VRF, we need to introduce the routing table's two main data structures, which are used to find the egress interface for a given packet: the Forwarding Information Base (FIB) and the Routing Information Base (RIB). Long gone are the days when a router maintained a single routing table on which it did linear, longest-prefix searches against destination IP addresses.
The FIB is a database of information used to forward packets. When a packet is received on a routed interface, the router looks up the destination address in the FIB to find the next hop for the packet.
The FIB structure is particularly efficient for resolving longest-prefix matches, and Cisco IOS resolves all route redirections so that a single lookup can yield the entry for the next hop of a packet. Cisco literature often mentions an adjacency concept when presenting the FIB. An adjacency is any node in the network that is reachable with a single Layer 2 hop. It so happens that Cisco IOS also maintains a data structure of adjacencies, which contains, among other things, interface and MAC layer rewrite information for all possible next hops. FIB entries point to the adjacency table, and in the remainder of this chapter, we group the two together and refer simply to the FIB. Hardware-based forwarding paths use the FIB concept, as does Cisco Express Forwarding (CEF).
Because it contains both Layer 2 and Layer 3 information, the FIB can be updated by several sources, such as routing protocol and Address Resolution Protocol (ARP) updates.
The RIB is the memory structure that contains classic routing data. The RIB can contain recursive routes. If a packet destination is not in the FIB, the router "punts" the packet to a slow processing path and resolves the destination next hop using the RIB.
When you enable VRFs, there are multiple instances of information in the FIB and RIB. You can see routing information with show ip route vrf name command, as shown in Example 4-3. Except for the first line, which identifies the VRF, there is no difference from the regular show ip route output.
Example 4-3 VRF Routing Table Information
R104#show ip route vrf RED
Routing Table: RED
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route
Gateway of last resort is not set
20.0.0.0/24 is subnetted, 1 subnets
O 20.0.0.0 [110/11121] via 40.0.0.1, 00:00:02, Tunnel0
40.0.0.0/24 is subnetted, 1 subnets
C 40.0.0.0 is directly connected, Tunnel0
30.0.0.0/24 is subnetted, 1 subnets
C 30.0.0.0 is directly connected, Ethernet0/0
The adjacencies for any routing entry in the VRF must resolve to different interfaces, even if the IP addresses on the interface are identical. Example 4-4 has a FIB entry for 40.0.0.2 from the global routing table, and Example 4-5 has the same information for the RED VRF. These examples are deliberately simple, but you can use the show ip cef forwarding vrf name detail command to peruse the FIB in all its glory.
Example 4-4 FIB Output for Global Routing Table
R104#show ip cef Prefix Next Hop Interface 0.0.0.0/0 drop Null0 (default route handler entry) 0.0.0.0/32 receive 40.0.0.0/24 attached Loopback1 40.0.0.0/32 receive 40.0.0.2/32 receive 40.0.0.255/32 receive
Example 4-5 FIB Output for VRF RED
R104#show ip cef vrf RED Prefix Next Hop Interface 0.0.0.0/0 drop Null0 (default route handler entry) 0.0.0.0/32 receive 40.0.0.0/24 attached Tunnel0 40.0.0.0/32 receive 40.0.0.2/32 receive
The term global is used to refer to the routing instances and tables that are not in a VRF, as in "global routing table" or "global address space."
When discussing VRFs, it is common to hear questions about resource allocation, such as "how are router resources allocated between VRFs?" This is a natural question, even if it is not logical! There are no special rules to prioritize data from one VRF to be processed quicker than data from other VRFs. A VRF exists in the memory plane of a device, not in the scheduling plane, which is what per-VRF traffic prioritization would require.
Note - Creating a VRF does not automatically consume a significant amount of memory. However, the expectation is that you will add routes to the VRFs and in this case be careful that the sum of entries in the virtual routing tables does not exceed the capacity of the device. In truth, this is not much different from standard best practices governing route table size.
Traffic processing happens according to the same rules as on a device with no VRFs:
Traffic enters the router.
The ingress policy is applied.
Routing and forwarding lookup occurs.
The egress policy is applied.
Traffic is forwarded.
Obviously, the ingress and egress policies can include QoS statements that prioritize traffic to or from a particular interface or address, but the fact that a packet belongs to a particular VRF has no impact on those policies. It simply alters what happens in Step 3 of the preceding list.
It is far more common to want to bind an interface or packet flow to a particular VRF based on policy criteria. For example, all interfaces from a certain user domain are bound to a single company VRF, or packets with a 10.0.0.0/8 source address are bound to a guest VRF. We look at this in great detail in some of the design chapters.
Virtual and Logical Routers
A VRF is not the same thing as a completely virtualized device, even if they are sometimes confused as such (it is true that some marketing literature encourages such confusion). They simply allow routers to support multiple address spaces. This is some distance from a fully virtualized device, where resources can be more or less arbitrarily allocated to tasks.
Virtualized devices do exist, however, and, to cut through the fog of confusion, it is helpful to have a taxonomy of terms to start with: