Chapter 7: Understanding CEF in an MPLS VPN Environment

Cisco Press

  • An Internet service provider's simple MPLS VPN design

  • Understanding the CEF and MPLS relationship

  • CEF considerations when troubleshooting MPLS VPN across various platforms

  • CEF and MPLS VPN load-sharing considerations

Understanding CEF operation in a Multiprotocol Label Switching (MPLS) environment is important for design stability and troubleshooting. This chapter analyzes CEF in MPLS Virtual Private Network (VPN) applications across Cisco platforms. Service providers and enterprises are increasingly using MPLS to provide connection-oriented services using an IP infrastructure. They also deploy MPLS VPNs for reasons including scalability, security, and flexibility. Because of increased MPLS VPN usage, you need to understand CEF's role, especially when troubleshooting.

CEF is the fundamental switching path for MPLS. Without CEF, MPLS forwarding does not occur. MPLS forwarding relies heavily on the IP routing table and the CEF architecture. Therefore, MPLS VPN relies on CEF because MPLS VPN depends on MPLS for successful operation.

This chapter assumes a basic understanding of MPLS VPNs. Although issues can occur with MPLS VPN setup, label distribution protocol (LDP), and operation, many times the issues relate to label mismatches stored in CEF or some other CEF inconsistency. Sometimes administrators believe the issue is with CEF, but it is with routing design. This chapter discusses some of these issues across various Cisco platforms.

An Internet Service Provider's Simple MPLS VPN Design

An ISP network typically consists of edge routers (often called provider edge, or PE, routers) that connect to customer edge (CE) routers and other service provider devices. In Figure 7-1, the ISP's PE routers receive customer routes (VPNv4 routes) through external BGP (eBGP) or some other routing protocol and send them through internal BGP (iBGP) to all other PE routers. Most networks use route reflectors to avoid the full mesh of iBGP connections, so all iBGP speakers have a session to the route reflectors. This is the basic structure of an ISP network for MPLS VPN implementation. Table 7-1 defines the type of devices in an MPLS VPN network and their awareness of MPLS VPN.

Figure 7-1

Figure 7-1

Typical ISP MPLS VPN Implementation

Table 7-1 MPLS VPN Device Type and Awareness

Type of DeviceDescriptionMPLS VPN Awareness
Provider (P) routerPart of provider network that interfaces with PE and other P routersNo
Provider edge (PE) routerPart of the provider network that interfaces with the CE router and P and PE routersYes
Customer edge (CE) routerPart of the customer network that interfaces with the PENo

MPLS VPN uses a shared infrastructure but provides security for customers' traffic. If site Routers CE1 and CE3 belong to the same customer and VPN, their traffic is segmented from other customers' traffic and marked with a label. The PE routers maintain a separate routing table for each customer VPN called a virtual routing and forwarding (VRF) table.

While the VRF tables provide the isolation between customers, the data from these routing tables is exchanged between PE routers to enable data transfer between sites attached to different PE routers. The core routers in the provider network (P routers) that provide transport across the provider's backbone are not aware of customer routes.

The transit across the backbone is performed by using the labels that had been exchanged for routes in the global IP routing table. So, in the most common form, two labels are added to customer traffic. The core routers do not care about the customers' IP packet for forwarding, but see a labeled packet targeted toward an egress PE router. The egress PE router performs label switching on the second label (VPN label that was previously assigned), and either forwards the IP packet toward the CE router or performs another IP lookup in the VRF. The end-to-end path between CE1 and CE3 is an MPLS Label Switch Path (LSP) tunnel. Figure 7-2 illustrates a typical LSP between two sites.

Figure 7-2

Typical LSP Between Two Sites

Understanding the CEF and MPLS VPN Relationship

An MPLS VPN depends on CEF functionality. MPLS creates its own database for lookups called the Label Forwarding Information Base (LFIB), but it uses the CEF FIB as a source of this information. If CEF cannot resolve a route, the label cannot switch to a destination either. The label process path of an MPLS VPN packet between customer sites is label imposition, label swapping, and then label disposition.

Label imposition occurs when the PE router forwards based on the IP header and adds an MPLS label to the packet upon entering an MPLS network. In the direction of label imposition, the router switches packets based on a CEF table lookup to find the next hop and adds the appropriate label information stored in the FIB for the destination.

When a router performs label swapping in the core on an MPLS packet, the router does an MPLS table lookup. The router derives this MPLS table (LFIB) from information in the CEF table and the Label Information Base (LIB). Path changes that occur to reach a destination can result in changes in the FIB and LFIB. Multiple paths known through different routers can result in one label for each path. The router derives label load-sharing information from the CEF load-sharing information. For load sharing, the router looks inside the packet to determine whether it is an IP version 4 (IPv4) packet. If it is an IPv4 packet, CEF uses the IP addresses for hashing. For example, in a P-P scenario, the P router looks below the MPLS labels and inside the IP header to use the IP addresses to allocate the correct hash bucket for load sharing but not to make a forwarding decision. The P router makes the forwarding decision based on the topmost label.

Label disposition occurs when the PE router receives an MPLS packet, makes a forwarding decision based on the MPLS label, removes the label, and sends an IP packet. The PE router uses the LFIB for path determination for a packet in this direction.

Table 7-2 summarizes which structures to look at when troubleshooting. As stated previously, a special iBGP session facilitates the advertisement of VPNv4 prefixes and their labels between PE routers. At the advertising PE, BGP allocates labels for the VPN prefixes learned locally and installs them in the LFIB, which is the MPLS forwarding table. From a PE device perspective, locally learned prefixes are prefixes learned from the connected customer edge device by the PE. At the receiving PE, if BGP accepts the remotely learned VPN prefixes with labels based on the route-target importation, BGP installs the VPN prefixes in the VRF FIB, which is the CEF table. From the PE perspective, remotely learned prefixes are prefixes learned from another PE through eBGP. An IP lookup occurs only once at the ingress PE. In short, if the router receives an MPLS packet, lookup occurs in the LFIB. If the router receives an IP packet, lookup occurs in the FIB.

Table 7-2 Switching Structure Used by Function

StructureCommandWhere to UseDevice Function
LFIBshow mpls forwarding <destination>P routersLabel-to-label switching
LFIBshow mpls forwarding vrf <name> <destination> detailAdvertising PE/egress PE routers for locally learned prefixesLabel-to-IP switching (disposition)
VRF FIBshow ip cef vrf <name> <destination> detailReceiving PE/ingress PE for remotely learned prefixesIP-to-label switching (imposition)
FIBshow ip cef <destination>CENormal routing/CEF switching

It is best to relate to an example to show how CEF is important in each stage of the label process path. The following cases show how to verify proper operation of an MPLS VPN network and how to analyze the LFIB and VRF CEF tables across an LSP.

Case 1: Label Disposition

In Figure 7-3, an MPLS VPN tunnel (LSP) exists between PE1 and PE2. CE1 is advertising its LAN prefix through eBGP to PE1. PE1 in turns advertises this local prefix through iBGP to other PE routers. PE1 must perform label disposition on traffic coming from the MPLS cloud toward Therefore, to verify that PE1 has the information on how to reach properly, you need to look at the LFIB on PE1. First, examine what label BGP assigns for this VPN prefix

Figure 7-3


In Example 7-1, PE1's BGP process sets the VPN label to 20.

Example 7-1 Verifying BGP Label Assignment on Egress PE for

PE1#show ip bgp vpn vrf red label | include    20/nolabel
PE1#show ip bgp vpn vrf red
BGP routing table entry for 1:1:, version 25550
Paths: (1 available, best #1, table red)
 Advertised to non peer-group peers: 
 65001 from (
   Origin IGP, metric 0, localpref 100, valid, external, best
   Extended Community: RT:1:1,
   mpls labels in/out 20/nolabel

Now, check the LFIB to make sure that PE1 sees the local, expected label to receive as 20. In Example 7-2, notice that the LFIB sees the correct local label/tag as 20.

Example 7-2 Verifying the LFIB on Egress PE for

PE1#show mpls forwarding vrf red
Local Outgoing  Prefix      Bytes tag Outgoing  Next Hop  
tag  tag or VC  or Tunnel Id   switched  interface       
20   Untagged[V]  0     Se2/0   point2point

Note - The PE router typically installs a locally learned route from a CE in the VRF routing table, VRF CEF table, and LFIB. However, the PE usually only checks the LFIB when passing traffic to a CE because traffic destined for the CE is typically an incoming MPLS packet. A PE router can also receive an IP packet destined for a CE if two CE routers are in the same VRF, connected to the same PE router, and sending traffic to each other. So, if an incoming packet to a PE is an IP packet from a VPN site, look in the VRF CEF table. If an incoming packet is an MPLS packet, look in the LFIB.

Case 2: Label Imposition

For CE2 to reach CE1, PE2 must perform label imposition on an IP packet received from CE2. At the other end of the VPN tunnel, PE2 should also see the correct VPN label of 20 to reach Look at the BGP table and the VRF FIB to confirm this, as shown in Example 7-3.

Example 7-3 Verifying BGP and VRF FIB on Ingress PE for

PE2#show ip bgp vpn vrf red
BGP routing table entry for 1:1:, version 8
Paths: (1 available, best #1, table red)
 Advertised to non peer-group peers: 
 65001 (metric 21) from (
   Origin IGP, metric 0, localpref 100, valid, internal, best
   Extended Community: RT:1:1,
   mpls labels in/out nolabel/20
PE2#show ip cef vrf red, version 7, epoch 0, cached adjacency
0 packets, 0 bytes
 tag information set
  local tag: VPN-route-head
  fast tag rewrite with Et1/0,, tags imposed: {16 20}
 via, 0 dependencies, recursive
  next hop, Ethernet1/0 via
  valid cached adjacency
  tag rewrite with Et1/0,, tags imposed: {16 20}

In Example 7-3, the out label on PE2 to reach is 20. This is correct. Two labels are found in the FIB output. The second label/bottom label is the BGP label. The first label/top label is the IGP label. The core P devices use the IGP label to reach the egress PE. Therefore, the IGP label must also be correct to reach PE1 from PE2 across the MPLS cloud. The IGP label is 16 in this output. The next case verifies whether this IGP label is correct. PE2 uses PE1's loopback address to reach because it is the BGP next hop, as seen in Example 7-4.

Example 7-4 Verifying the VPN Path to Reach from the Ingress PE

PE2#show ip route vrf red
Routing entry for
 Known via "bgp 65100", distance 200, metric 0
 Tag 65001, type internal
 Last update from 3d13h ago
 Routing Descriptor Blocks:
 * (Default-IP-Routing-Table), from 192.1
   Route metric is 0, traffic share count is 1
   AS Hops 1
PE2#show ip route
Routing entry for
 Known via "ospf 100", distance 110, metric 21, type intra area
 Last update from on Ethernet1/0, 01:38:42 ago
 Routing Descriptor Blocks:
 *, from, 01:38:42 ago, via Ethernet1/0
   Route metric is 21, traffic share count is 1

Case 3: Label Swapping

On the P router, the local tag to reach, PE1's loopback, is 16, as seen in Example 7-5. Therefore, for traffic destined for, the P router expects an incoming label of 16.

Example 7-5 Verifying the LFIB on the P Router

P#show mpls forwarding
Local Outgoing  Prefix      Bytes tag Outgoing  Next Hop  
tag  tag or VC  or Tunnel Id   switched  interface       
16   Pop tag  4350468  Et0/0

Therefore, the IGP label that PE2 sees is correct in Example 7-3. When the P router receives traffic toward with a label of 16, it pops the top label and the packet passes to PE1 with only the bottom label of 20, the VPN label.

1 2 3 4 5 6 Page 1
Page 1 of 6
The 10 most powerful companies in enterprise networking 2022