An Internet service provider's simple MPLS VPN design
Understanding the CEF and MPLS relationship
CEF considerations when troubleshooting MPLS VPN across various platforms
CEF and MPLS VPN load-sharing considerations
Understanding CEF operation in a Multiprotocol Label Switching (MPLS) environment is important for design stability and troubleshooting. This chapter analyzes CEF in MPLS Virtual Private Network (VPN) applications across Cisco platforms. Service providers and enterprises are increasingly using MPLS to provide connection-oriented services using an IP infrastructure. They also deploy MPLS VPNs for reasons including scalability, security, and flexibility. Because of increased MPLS VPN usage, you need to understand CEF's role, especially when troubleshooting.
CEF is the fundamental switching path for MPLS. Without CEF, MPLS forwarding does not occur. MPLS forwarding relies heavily on the IP routing table and the CEF architecture. Therefore, MPLS VPN relies on CEF because MPLS VPN depends on MPLS for successful operation.
This chapter assumes a basic understanding of MPLS VPNs. Although issues can occur with MPLS VPN setup, label distribution protocol (LDP), and operation, many times the issues relate to label mismatches stored in CEF or some other CEF inconsistency. Sometimes administrators believe the issue is with CEF, but it is with routing design. This chapter discusses some of these issues across various Cisco platforms.
An Internet Service Provider's Simple MPLS VPN Design
An ISP network typically consists of edge routers (often called provider edge, or PE, routers) that connect to customer edge (CE) routers and other service provider devices. In Figure 7-1, the ISP's PE routers receive customer routes (VPNv4 routes) through external BGP (eBGP) or some other routing protocol and send them through internal BGP (iBGP) to all other PE routers. Most networks use route reflectors to avoid the full mesh of iBGP connections, so all iBGP speakers have a session to the route reflectors. This is the basic structure of an ISP network for MPLS VPN implementation. Table 7-1 defines the type of devices in an MPLS VPN network and their awareness of MPLS VPN.
Typical ISP MPLS VPN Implementation
Table 7-1 MPLS VPN Device Type and Awareness
| Type of Device | Description | MPLS VPN Awareness |
|---|---|---|
| Provider (P) router | Part of provider network that interfaces with PE and other P routers | No |
| Provider edge (PE) router | Part of the provider network that interfaces with the CE router and P and PE routers | Yes |
| Customer edge (CE) router | Part of the customer network that interfaces with the PE | No |
MPLS VPN uses a shared infrastructure but provides security for customers' traffic. If site Routers CE1 and CE3 belong to the same customer and VPN, their traffic is segmented from other customers' traffic and marked with a label. The PE routers maintain a separate routing table for each customer VPN called a virtual routing and forwarding (VRF) table.
While the VRF tables provide the isolation between customers, the data from these routing tables is exchanged between PE routers to enable data transfer between sites attached to different PE routers. The core routers in the provider network (P routers) that provide transport across the provider's backbone are not aware of customer routes.
The transit across the backbone is performed by using the labels that had been exchanged for routes in the global IP routing table. So, in the most common form, two labels are added to customer traffic. The core routers do not care about the customers' IP packet for forwarding, but see a labeled packet targeted toward an egress PE router. The egress PE router performs label switching on the second label (VPN label that was previously assigned), and either forwards the IP packet toward the CE router or performs another IP lookup in the VRF. The end-to-end path between CE1 and CE3 is an MPLS Label Switch Path (LSP) tunnel. Figure 7-2 illustrates a typical LSP between two sites.
Typical LSP Between Two Sites
Understanding the CEF and MPLS VPN Relationship
An MPLS VPN depends on CEF functionality. MPLS creates its own database for lookups called the Label Forwarding Information Base (LFIB), but it uses the CEF FIB as a source of this information. If CEF cannot resolve a route, the label cannot switch to a destination either. The label process path of an MPLS VPN packet between customer sites is label imposition, label swapping, and then label disposition.
Label imposition occurs when the PE router forwards based on the IP header and adds an MPLS label to the packet upon entering an MPLS network. In the direction of label imposition, the router switches packets based on a CEF table lookup to find the next hop and adds the appropriate label information stored in the FIB for the destination.
When a router performs label swapping in the core on an MPLS packet, the router does an MPLS table lookup. The router derives this MPLS table (LFIB) from information in the CEF table and the Label Information Base (LIB). Path changes that occur to reach a destination can result in changes in the FIB and LFIB. Multiple paths known through different routers can result in one label for each path. The router derives label load-sharing information from the CEF load-sharing information. For load sharing, the router looks inside the packet to determine whether it is an IP version 4 (IPv4) packet. If it is an IPv4 packet, CEF uses the IP addresses for hashing. For example, in a P-P scenario, the P router looks below the MPLS labels and inside the IP header to use the IP addresses to allocate the correct hash bucket for load sharing but not to make a forwarding decision. The P router makes the forwarding decision based on the topmost label.
Label disposition occurs when the PE router receives an MPLS packet, makes a forwarding decision based on the MPLS label, removes the label, and sends an IP packet. The PE router uses the LFIB for path determination for a packet in this direction.
Table 7-2 summarizes which structures to look at when troubleshooting. As stated previously, a special iBGP session facilitates the advertisement of VPNv4 prefixes and their labels between PE routers. At the advertising PE, BGP allocates labels for the VPN prefixes learned locally and installs them in the LFIB, which is the MPLS forwarding table. From a PE device perspective, locally learned prefixes are prefixes learned from the connected customer edge device by the PE. At the receiving PE, if BGP accepts the remotely learned VPN prefixes with labels based on the route-target importation, BGP installs the VPN prefixes in the VRF FIB, which is the CEF table. From the PE perspective, remotely learned prefixes are prefixes learned from another PE through eBGP. An IP lookup occurs only once at the ingress PE. In short, if the router receives an MPLS packet, lookup occurs in the LFIB. If the router receives an IP packet, lookup occurs in the FIB.
Table 7-2 Switching Structure Used by Function
| Structure | Command | Where to Use | Device Function |
|---|---|---|---|
| LFIB | show mpls forwarding <destination> | P routers | Label-to-label switching |
| LFIB | show mpls forwarding vrf <name> <destination> detail | Advertising PE/egress PE routers for locally learned prefixes | Label-to-IP switching (disposition) |
| VRF FIB | show ip cef vrf <name> <destination> detail | Receiving PE/ingress PE for remotely learned prefixes | IP-to-label switching (imposition) |
| FIB | show ip cef <destination> | CE | Normal routing/CEF switching |
It is best to relate to an example to show how CEF is important in each stage of the label process path. The following cases show how to verify proper operation of an MPLS VPN network and how to analyze the LFIB and VRF CEF tables across an LSP.
Case 1: Label Disposition
In Figure 7-3, an MPLS VPN tunnel (LSP) exists between PE1 and PE2. CE1 is advertising its LAN prefix 10.1.0.0/16 through eBGP to PE1. PE1 in turns advertises this local prefix through iBGP to other PE routers. PE1 must perform label disposition on traffic coming from the MPLS cloud toward 10.1.0.0/16. Therefore, to verify that PE1 has the information on how to reach 10.1.1.1 properly, you need to look at the LFIB on PE1. First, examine what label BGP assigns for this VPN prefix 10.1.0.0/16.
MPLS VPN Tunnel
In Example 7-1, PE1's BGP process sets the VPN label to 20.
Example 7-1 Verifying BGP Label Assignment on Egress PE for 10.1.0.0/16
PE1#show ip bgp vpn vrf red label | include 10.1.0.0 10.1.0.0/16 10.3.1.2 20/nolabel PE1#show ip bgp vpn vrf red 10.1.1.1 BGP routing table entry for 1:1:10.1.0.0/16, version 25550 Paths: (1 available, best #1, table red) Advertised to non peer-group peers: 192.168.4.4 65001 10.3.1.2 from 10.3.1.2 (10.3.1.6) Origin IGP, metric 0, localpref 100, valid, external, best Extended Community: RT:1:1, mpls labels in/out 20/nolabel
Now, check the LFIB to make sure that PE1 sees the local, expected label to receive as 20. In Example 7-2, notice that the LFIB sees the correct local label/tag as 20.
Example 7-2 Verifying the LFIB on Egress PE for 10.1.0.0/16
PE1#show mpls forwarding vrf red 10.1.1.1 Local Outgoing Prefix Bytes tag Outgoing Next Hop tag tag or VC or Tunnel Id switched interface 20 Untagged 10.1.0.0/16[V] 0 Se2/0 point2point
Note - The PE router typically installs a locally learned route from a CE in the VRF routing table, VRF CEF table, and LFIB. However, the PE usually only checks the LFIB when passing traffic to a CE because traffic destined for the CE is typically an incoming MPLS packet. A PE router can also receive an IP packet destined for a CE if two CE routers are in the same VRF, connected to the same PE router, and sending traffic to each other. So, if an incoming packet to a PE is an IP packet from a VPN site, look in the VRF CEF table. If an incoming packet is an MPLS packet, look in the LFIB.
Case 2: Label Imposition
For CE2 to reach CE1, PE2 must perform label imposition on an IP packet received from CE2. At the other end of the VPN tunnel, PE2 should also see the correct VPN label of 20 to reach 10.1.1.1. Look at the BGP table and the VRF FIB to confirm this, as shown in Example 7-3.
Example 7-3 Verifying BGP and VRF FIB on Ingress PE for 10.1.0.0/16
PE2#show ip bgp vpn vrf red 10.1.1.1
BGP routing table entry for 1:1:10.1.0.0/16, version 8
Paths: (1 available, best #1, table red)
Advertised to non peer-group peers:
10.4.1.2
65001
192.168.2.2 (metric 21) from 192.168.2.2 (192.168.2.2)
Origin IGP, metric 0, localpref 100, valid, internal, best
Extended Community: RT:1:1,
mpls labels in/out nolabel/20
PE2#show ip cef vrf red 10.1.1.1
10.1.0.0/16, version 7, epoch 0, cached adjacency 172.16.3.1
0 packets, 0 bytes
tag information set
local tag: VPN-route-head
fast tag rewrite with Et1/0, 172.16.3.1, tags imposed: {16 20}
via 192.168.2.2, 0 dependencies, recursive
next hop 172.16.3.1, Ethernet1/0 via 192.168.2.2/32
valid cached adjacency
tag rewrite with Et1/0, 172.16.3.1, tags imposed: {16 20}
In Example 7-3, the out label on PE2 to reach 10.1.1.1 is 20. This is correct. Two labels are found in the FIB output. The second label/bottom label is the BGP label. The first label/top label is the IGP label. The core P devices use the IGP label to reach the egress PE. Therefore, the IGP label must also be correct to reach PE1 from PE2 across the MPLS cloud. The IGP label is 16 in this output. The next case verifies whether this IGP label is correct. PE2 uses PE1's loopback address to reach 10.1.1.1 because it is the BGP next hop, as seen in Example 7-4.
Example 7-4 Verifying the VPN Path to Reach 10.1.1.1 from the Ingress PE
PE2#show ip route vrf red 10.1.1.1 Routing entry for 10.1.0.0/16 Known via "bgp 65100", distance 200, metric 0 Tag 65001, type internal Last update from 192.168.2.2 3d13h ago Routing Descriptor Blocks: * 192.168.2.2 (Default-IP-Routing-Table), from 192.1 Route metric is 0, traffic share count is 1 AS Hops 1 PE2#show ip route 192.168.2.2 Routing entry for 192.168.2.2/32 Known via "ospf 100", distance 110, metric 21, type intra area Last update from 172.16.3.1 on Ethernet1/0, 01:38:42 ago Routing Descriptor Blocks: * 172.16.3.1, from 192.168.2.2, 01:38:42 ago, via Ethernet1/0 Route metric is 21, traffic share count is 1
Case 3: Label Swapping
On the P router, the local tag to reach 192.168.2.2, PE1's loopback, is 16, as seen in Example 7-5. Therefore, for traffic destined for 192.168.2.2, the P router expects an incoming label of 16.
Example 7-5 Verifying the LFIB on the P Router
P#show mpls forwarding 192.168.2.2 Local Outgoing Prefix Bytes tag Outgoing Next Hop tag tag or VC or Tunnel Id switched interface 16 Pop tag 192.168.2.2/32 4350468 Et0/0 172.16.2.1
Therefore, the IGP label that PE2 sees is correct in Example 7-3. When the P router receives traffic toward 10.1.1.1 with a label of 16, it pops the top label and the packet passes to PE1 with only the bottom label of 20, the VPN label.