MPLS and BGP Multipath, Prefix Independent Convergence (PIC) and Best External

The aim of this post is to explain in detail some key Border Gateway Protocol (BGP) features in a Multiprotocol Label Switching (MPLS) network topology.

Contents

  • Network Diagrams
    • Physical Topology
    • IP Addressing and Customer IGP/BGP Topology
    • MPLS, BGP VPNv4, VRF and Redistribution Topology
  • BGP Multipath – BGP VPNv4 Full Mesh
    • BGP Multipath – iBGP
    • BGP Multipath – eiBGP
  • BGP Multipath – BGP VPNv4 Route Reflector (RR)
    • BGP Multipath – iBGP
  • BGP Prefix Independent Convergence (PIC)
  • BGP Best Path External
  • APPENDIX

Introduction

The configuration options covered in this document are mainly related to two BGP features:

BGP Multipath Load Sharing for EBGP and iBGP which allows the configuration of multipath load balancing with both external BGP (EBGP) and internal BGP (iBGP) paths in BGP networks that are configured to use MPLS Virtual Private Networks (MPLS VPNs). This feature provides improved load balancing deployment and service offering capabilities and it is useful for multi-homed autonomous systems and Provider Edge (PE) routers that import both EBGP and iBGP paths from multihomed and stub networks.

BGP PIC Edge for IP and MPLS VPN feature improves BGP convergence after a network failure. This convergence is applicable to both core and edge failures and can be used in both IP and MPLS networks. This feature creates and stores a backup/alternative path in the Routing Information Base (RIB), Forwarding Information Base (FIB), and Cisco Express Forwarding (CEF) so that when a failure is detected, the backup/alternative path can immediately take over, thus enabling fast failover.

Notice that this post assumes you understand basic MPLS, Label Distribution Protocol (LDP), Virtual Routing and Forwarding (VRF) and PE to Customer Edge (CE) routing configuration.

Network Diagrams

Physical Topology

Figure 1

IP Addressing and Customer IGP/BGP Topology

Figure 2

MPLS, BGP VPNv4, VRF and Redistribution Topology

Figure 3

BGP Multipath – BGP VPNv4 Full Mesh

In this section we are going to analyse step by step the advertisement of BGP VPNv4 prefixes between PEs and the requirements needed in order to achieve load sharing or multipathing when there are multiple paths available for the same final destination.

Figure 4 shows the first network design we are going to use for our study; a Full Mesh iBGP VPNv4 between PE routers (R1 to R6).

Figure 4

R1 – R6:

router bgp 200
 no bgp default ipv4-unicast
 bgp log-neighbor-changes
 neighbor 1.1.1.1 remote-as 200
 neighbor 1.1.1.1 update-source Loopback10
 neighbor 2.2.2.2 remote-as 200
 neighbor 2.2.2.2 update-source Loopback10
 neighbor 3.3.3.3 remote-as 200
 neighbor 3.3.3.3 update-source Loopback10
 neighbor 4.4.4.4 remote-as 200
 neighbor 4.4.4.4 update-source Loopback10
 neighbor 5.5.5.5 remote-as 200
 neighbor 5.5.5.5 update-source Loopback10
 neighbor 6.6.6.6 remote-as 200
 neighbor 6.6.6.6 update-source Loopback10
!
 address-family vpnv4
  neighbor 1.1.1.1 activate
  neighbor 2.2.2.2 activate
  neighbor 3.3.3.3 activate
  neighbor 4.4.4.4 activate
  neighbor 5.5.5.5 activate
  neighbor 6.6.6.6 activate  
 exit-address-family

Customer A and Customer B have two sites: Site A which is dual-homed to a MPLS Service Provider (SP) backbone and Site B that is single-homed to the same MPLS SP. The purpose of this network design is to guarantee connectivity between Customer Sites across the MPLS core network.

Now we are going to focus exactly on how R15, which is located at Customer A Site B, routes traffic to R11’s Loopback0 (172.16.1.11/32), which is located at Customer A Site A.

R1 (PE) installs the OSPF route in the Routing Table for VRF CA that is advertised from R11 (CE).

R1#show ip route vrf CA 172.16.1.11
Routing entry for 172.16.1.11/32
  Known via “ospf 125”, distance 110, metric 11, type inter area
  Redistributing via bgp 200
  Advertised by bgp 200
  Last update from 10.1.111.11 on FastEthernet0/1, 00:01:59 ago
  Routing Descriptor Blocks:
  * 10.1.111.11, from 11.11.11.11, 00:01:59 ago, via FastEthernet0/1
      Route metric is 11, traffic share count is 1

R2 (PE) has also the OSPF route in the Routing Table for VRF CA to the destination 172.16.1.11/32 from R12 (CE).

R2#show ip route vrf CA 172.16.1.11
Routing entry for 172.16.1.11/32
  Known via “ospf 125”, distance 110, metric 21, type inter area
  Redistributing via bgp 200
  Advertised by bgp 200
  Last update from 10.1.122.12 on FastEthernet0/1, 00:03:44 ago
  Routing Descriptor Blocks:
  * 10.1.122.12, from 11.11.11.11, 00:03:44 ago, via FastEthernet0/1
      Route metric is 21, traffic share count is 1

R1 and R2, the PEs, perform mutual redistribution between the Customer IGP (OSPF) and BGP VPNv4.

R1:

router ospf 125 vrf CA
 log-adjacency-changes
 redistribute bgp 200 subnets
 network 10.1.111.0 0.0.0.255 area 0

 

router bgp 200
address-family ipv4 vrf CA
  redistribute ospf 125 vrf CA
  no synchronization
 exit-address-family

R2:

router ospf 125 vrf CA
 log-adjacency-changes
 redistribute bgp 200 subnets
 network 10.1.122.0 0.0.0.255 area 0

 

router bgp 200
address-family ipv4 vrf CA
  redistribute ospf 125 vrf CA
  no synchronization
 exit-address-family

Therefore the OSPF routes to 172.16.1.11/32 also appear as BGP VPNv4 routes and can be advertised to the rest of BGP VPNv4 neighbours. In our current topology there is a full mesh between iBGP VPNv4 peers in the MPLS core, so each peer will receive the advertisement from R1 and R2.

A BGP VPNv4 route is composed of the Router Distinguisher (RD) plus the prefix and the length. So the BGP VPNv4 prefix for the IP Prefix 172.16.1.11/32 is 1:2:172.16.1.11/32. The RD is configured under the VRF of R1 and R2 and in this specific scenario they share the same RD for VRF CA (1:2).

R1 and R2:

vrf definition CA
 rd 1:2
 route-target export 125:125
 route-target import 125:125
 address-family ipv4
 exit-address-family

Because this is a dual-homed design, R1 has two BGP VPNv4 routes for 1:2:172.16.1.11/32, one locally originated and with the next hop value of 10.1.111.11 (R11’s Fa0/0) and one from R2, the other PE serving the same Customer Site, with next hop 2.2.2.2 (R2’s Loopback).

R1#show bgp vpnv4 unicast all
   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 1:2 (default for vrf CA)
* i10.1.1.0/24      2.2.2.2                 20    100      0 ?
*>                  10.1.111.11             20         32768 ?
* i10.1.111.0/24    2.2.2.2                 30    100      0 ?
*>                  0.0.0.0                  0         32768 ?
* i10.1.122.0/24    2.2.2.2                  0    100      0 ?
*>                  10.1.111.11             30         32768 ?
*>i10.1.155.0/24    5.5.5.5                  0    100      0 ?
* i172.16.1.11/32   2.2.2.2                 21    100      0 ?
*>                  10.1.111.11             11         32768 ?
* i172.16.1.12/32   2.2.2.2                 11    100      0 ?
*>                  10.1.111.11             21         32768 ?
*>i172.16.1.15/32   5.5.5.5                 11    100      0 ?
Route Distinguisher: 5:5
*>i10.1.155.0/24    5.5.5.5                  0    100      0 ?
*>i172.16.1.15/32   5.5.5.5                 11    100      0 ?

R1 best route for 1:2:172.16.1.11/32 is the one that R1 locally originates (0.0.0.0) and with next-hop 10.1.111.11 (R11’s Fa0/0). Because R1 does the redistribution from OSPF into BGP VPNv4 is going to automatically assign a BGP weight value of 32768 and this wins the BGP Path Selection algorithm. 32768 > 0 (default weight for the BGP VPNv4 route advertised by R2).

R1#show bgp vpnv4 unicast all 172.16.1.11
BGP routing table entry for 1:2:172.16.1.11/32, version 12
Paths: (2 available, best #2, table CA)
  Advertised to update-groups:
     1
  Local
    2.2.2.2 (metric 21) from 2.2.2.2 (2.2.2.2)
      Origin incomplete, metric 21, localpref 100, valid, internal
      Extended Community: RT:125:125 OSPF DOMAIN ID:0x0005:0x0000007D0200
        OSPF RT:0.0.0.0:3:0 OSPF ROUTER ID:10.1.122.2:0
      mpls labels in/out 120/220
  Local
    10.1.111.11 from 0.0.0.0 (1.1.1.1)
      Origin incomplete, metric 11, localpref 100, weight 32768, valid, sourced, best
      Extended Community: RT:125:125 OSPF DOMAIN ID:0x0005:0x0000007D0200
        OSPF RT:0.0.0.0:3:0 OSPF ROUTER ID:10.1.111.1:0
      mpls labels in/out 120/nolabel

The same applies to R2 for 1:2:172.16.1.11/32 BGP VPNv4 prefix. The route locally originated from R2 is preferred over the route advertised by R1 because of the high weight value.

R2#show bgp vpnv4 uni all
   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 1:2 (default for vrf CA)
* i10.1.1.0/24      1.1.1.1                 20    100      0 ?
*>                  10.1.122.12             20         32768 ?
* i10.1.111.0/24    1.1.1.1                  0    100      0 ?
*>                  10.1.122.12             30         32768 ?
* i10.1.122.0/24    1.1.1.1                 30    100      0 ?
*>                  0.0.0.0                  0         32768 ?
*>i10.1.155.0/24    5.5.5.5                  0    100      0 ?
* i172.16.1.11/32   1.1.1.1                 11    100      0 ?
*>                  10.1.122.12             21         32768 ?
* i172.16.1.12/32   1.1.1.1                 21    100      0 ?
*>                  10.1.122.12             11         32768 ?
*>i172.16.1.15/32   5.5.5.5                 11    100      0 ?
Route Distinguisher: 5:5
*>i10.1.155.0/24    5.5.5.5                  0    100      0 ?
*>i172.16.1.15/32   5.5.5.5                 11    100      0 ?

 

R2#show bgp vpnv4 unicast all 172.16.1.11
BGP routing table entry for 1:2:172.16.1.11/32, version 12
Paths: (2 available, best #2, table CA)
  Advertised to update-groups:
     1
  Local
    1.1.1.1 (metric 21) from 1.1.1.1 (1.1.1.1)
      Origin incomplete, metric 11, localpref 100, valid, internal
      Extended Community: RT:125:125 OSPF DOMAIN ID:0x0005:0x0000007D0200
        OSPF RT:0.0.0.0:3:0 OSPF ROUTER ID:10.1.111.1:0
      mpls labels in/out 220/120
  Local
    10.1.122.12 from 0.0.0.0 (2.2.2.2)
      Origin incomplete, metric 21, localpref 100, weight 32768, valid, sourced, best
      Extended Community: RT:125:125 OSPF DOMAIN ID:0x0005:0x0000007D0200
        OSPF RT:0.0.0.0:3:0 OSPF ROUTER ID:10.1.122.2:0
      mpls labels in/out 220/nolabel

From R5’s point of view (PE for Customer A Site B), we can see that receives both (R1 and R2) BGP VPNv4 routes for 1:2:172.16.1.11/32.

R5#show bgp vpnv4 unicast all
   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 1:2
*>i10.1.1.0/24      1.1.1.1                 20    100      0 ?
* i                 2.2.2.2                 20    100      0 ?
*>i10.1.111.0/24    1.1.1.1                  0    100      0 ?
* i                 2.2.2.2                 30    100      0 ?
* i10.1.122.0/24    1.1.1.1                 30    100      0 ?
*>i                 2.2.2.2                  0    100      0 ?
*>i172.16.1.11/32   1.1.1.1                 11    100      0 ?
* i                 2.2.2.2                 21    100      0 ?
* i172.16.1.12/32   1.1.1.1                 21    100      0 ?
*>i                 2.2.2.2                 11    100      0 ?
Route Distinguisher: 5:5 (default for vrf CA)
*>i10.1.1.0/24      1.1.1.1                 20    100      0 ?
*>i10.1.111.0/24    1.1.1.1                  0    100      0 ?
*>i10.1.122.0/24    2.2.2.2                  0    100      0 ?
*> 10.1.155.0/24    0.0.0.0                  0         32768 ?
*>i172.16.1.11/32   1.1.1.1                 11    100      0 ?
*>i172.16.1.12/32   2.2.2.2                 11    100      0 ?
*> 172.16.1.15/32   10.1.155.15             11         32768 ?

The BGP VPNv4 route preferred for the final destination is through R1. The BGP Best Selection Path is now based on lower Metric (MED). R1’s MED is 11 and is lower than R2’s MED which is 21.

R5#show bgp vpnv4 unicast all 172.16.1.11/32
BGP routing table entry for 1:2:172.16.1.11/32, version 11
Paths: (2 available, best #1, no table)
  Not advertised to any peer
  Local
    1.1.1.1 (metric 32) from 1.1.1.1 (1.1.1.1)
      Origin incomplete, metric 11, localpref 100, valid, internal, best
      Extended Community: RT:125:125 OSPF DOMAIN ID:0x0005:0x0000007D0200
        OSPF RT:0.0.0.0:3:0 OSPF ROUTER ID:10.1.111.1:0
      mpls labels in/out nolabel/120
  Local
    2.2.2.2 (metric 32) from 2.2.2.2 (2.2.2.2)
      Origin incomplete, metric 21, localpref 100, valid, internal
      Extended Community: RT:125:125 OSPF DOMAIN ID:0x0005:0x0000007D0200
        OSPF RT:0.0.0.0:3:0 OSPF ROUTER ID:10.1.122.2:0
      mpls labels in/out nolabel/220
BGP routing table entry for 5:5:172.16.1.11/32, version 15
Paths: (1 available, best #1, table CA)
  Not advertised to any peer
  Local, imported path from 1:2:172.16.1.11/32
    1.1.1.1 (metric 32) from 1.1.1.1 (1.1.1.1)
      Origin incomplete, metric 11, localpref 100, valid, internal, best
      Extended Community: RT:125:125 OSPF DOMAIN ID:0x0005:0x0000007D0200
        OSPF RT:0.0.0.0:3:0 OSPF ROUTER ID:10.1.111.1:0
      mpls labels in/out nolabel/120

As we can see in the output above, the best route is also imported as BGP VPNv4 route with RD 5:5 (5:5:172.16.1.11/32), the RD configured under VRF CA on R5. R5 is doing mutual redistribution between BGP VPNv4 and the Customer IGP (OSPF).

R5:

vrf definition CA
 rd 5:5
 route-target export 125:125
 route-target import 125:125
address-family ipv4
 exit-address-family

 

router ospf 125 vrf CA
 log-adjacency-changes
 redistribute bgp 200 subnets
 network 10.1.155.0 0.0.0.255 area 0

router bgp 200
address-family ipv4 vrf CA
  redistribute ospf 125 vrf CA
  no synchronization
 exit-address-family

Notice that BGP default behaviour is choosing only one best path to the destination based on the BGP Best Path Selection Algorithm. If we do a traceroute to the destination from R5 we can verify that is taking the path through R1 and ignoring the alternative path through R2.

R5#show ip route vrf CA
Gateway of last resort is not set
     172.16.0.0/32 is subnetted, 3 subnets
B       172.16.1.12 [200/11] via 2.2.2.2, 01:20:21
O IA    172.16.1.15 [110/11] via 10.1.155.15, 01:21:13, FastEthernet0/1
B       172.16.1.11 [200/11] via 1.1.1.1, 01:20:21
     10.0.0.0/24 is subnetted, 4 subnets
B       10.1.1.0 [200/20] via 1.1.1.1, 01:20:21
B       10.1.111.0 [200/0] via 1.1.1.1, 01:20:21
B       10.1.122.0 [200/0] via 2.2.2.2, 01:20:21
C       10.1.155.0 is directly connected, FastEthernet0/1

 

R5#show ip route vrf CA 172.16.1.11
Routing entry for 172.16.1.11/32
  Known via “bgp 200”, distance 200, metric 11, type internal
  Redistributing via ospf 125
  Advertised by ospf 125 subnets
  Last update from 1.1.1.1 01:20:29 ago
  Routing Descriptor Blocks:
  * 1.1.1.1 (Default-IP-Routing-Table), from 1.1.1.1, 01:20:29 ago
      Route metric is 11, traffic share count is 1
      AS Hops 0

R5#traceroute vrf CA 172.16.1.11
  1 10.1.59.9 [MPLS: Labels 908/120 Exp 0] 72 msec 44 msec 64 msec
  2 10.1.109.10 [MPLS: Labels 10006/120 Exp 0] 68 msec 64 msec 68 msec
  3 10.1.107.7 [MPLS: Labels 700/120 Exp 0] 64 msec 60 msec 60 msec
  4 10.1.111.1 [MPLS: Label 120 Exp 0] 32 msec 60 msec 44 msec
  5 10.1.111.11 56 msec 56 msec 76 msec

R5 advertises its best route through R1 to the customer. Finally we can verify that the route gets installed in R15’s routing table as an OSPF Inter Area (IA) route. A traceroute command shows end to end reachability.

R15#show ip route ospf
     172.16.0.0/32 is subnetted, 3 subnets
O IA    172.16.1.12 [110/21] via 10.1.155.5, 01:24:31, FastEthernet0/0
O IA    172.16.1.11 [110/21] via 10.1.155.5, 01:24:31, FastEthernet0/0
     10.0.0.0/24 is subnetted, 4 subnets
O IA    10.1.1.0 [110/30] via 10.1.155.5, 01:24:31, FastEthernet0/0
O IA    10.1.111.0 [110/11] via 10.1.155.5, 01:24:31, FastEthernet0/0
O IA    10.1.122.0 [110/11] via 10.1.155.5, 01:24:31, FastEthernet0/0

 

R15#show ip route 172.16.1.11
Routing entry for 172.16.1.11/32
  Known via “ospf 125”, distance 110, metric 21, type inter area
  Last update from 10.1.155.5 on FastEthernet0/0, 01:25:05 ago
  Routing Descriptor Blocks:
  * 10.1.155.5, from 10.1.155.5, 01:25:05 ago, via FastEthernet0/0
      Route metric is 21, traffic share count is 1

R15#traceroute 172.16.1.11
  1 10.1.155.5 16 msec 12 msec 8 msec
  2 10.1.59.9 [MPLS: Labels 908/120 Exp 0] 76 msec 68 msec 76 msec
  3 10.1.109.10 [MPLS: Labels 10006/120 Exp 0] 64 msec 96 msec 80 msec
  4 10.1.107.7 [MPLS: Labels 700/120 Exp 0] 52 msec 60 msec 68 msec
  5 10.1.111.1 [MPLS: Label 120 Exp 0] 64 msec 64 msec 64 msec
  6 10.1.111.11 64 msec 76 msec 48 msec

BGP Multipath – iBGP

The option of having multiple paths to the destination and use only the best path might not be the desirable choice. Instead using some or all of the alternative paths for load sharing might be more efficient.

Figure 5 shows the idea of installing both routes, through R1 and R2, for forwarding.

Figure 5

In order to configure BGP Multipath, the first 9 points of the BGP Best Selection Path algorithm must be identical for all the routes available.

  1. Prefer the path with the highest WEIGHT.
  2. Prefer the path with the highest LOCAL_PREF.
  3. Prefer the path that was locally originated via a network or aggregate BGP subcommand or through redistribution from an IGP.
  4. Prefer the path with the shortest AS_PATH.
  5. Prefer the path with the lowest origin type.
  6. Prefer the path with the lowest multi-exit discriminator (MED).
  7. Prefer eBGP over iBGP paths.
    1. If bestpath is selected, go to Step 9 (multipath).
  8. Prefer the path with the lowest IGP metric to the BGP next hop.
    1. Continue, even if bestpath is already selected.
  9. Determine if multiple paths require installation in the routing table for BGP Multipath.

As we saw and following with our example, R5 has two iBGP VPNv4 routes to 1:2:172.16.1.11/32. However, only the best route (advertised by R1) is imported into the VRF CA as iBGP route. The alternative path gets discarded and it is not used for sending traffic. In order to use both routes for forwarding traffic and we enable BGP Multipath feature.

Then we configure BGP Multipath on R5. Because R5 is who has two possible iBGP VPNv4 routes to the destination.

R5(config)#router bgp 200
R5(config-router)#address-family ipv4 vrf CA
R5(config-router-af)#maximum-paths ?
  <1-16>  Number of paths
  eibgp   Both eBGP and iBGP paths as multipath
  ibgp    iBGP-multipath
  import  Maximum import paths
R5(config-router-af)#maximum-paths ibgp ?
  <1-16>        Number of paths
  unequal-cost  Perform Un-Equal cost multipath selection
R5(config-router-af)#maximum-paths ibgp 2 import 2

NOTE: In this specific version the import option is required to be able to install redundant paths that can be configured as backup multipaths for a VRF instance. This keyword can be configured only under a VRF in address family configuration mode.

However, nothing has changed so far. R5 still prefers the route through R1 and it is not using the alternative path advertised by R2.

R5#show bgp vpnv4 unicast all 172.16.1.11
BGP routing table entry for 1:2:172.16.1.11/32, version 5
Paths: (2 available, best #1, no table)
  Not advertised to any peer
  Local
    1.1.1.1 (metric 32) from 1.1.1.1 (1.1.1.1)
      Origin incomplete, metric 11, localpref 100, valid, internal, best
      Extended Community: RT:125:125 OSPF DOMAIN ID:0x0005:0x0000007D0200
        OSPF RT:0.0.0.0:3:0 OSPF ROUTER ID:10.1.111.1:0
      mpls labels in/out nolabel/120
  Local
    2.2.2.2 (metric 32) from 2.2.2.2 (2.2.2.2)
      Origin incomplete, metric 11, localpref 100, valid, internal
      Extended Community: RT:125:125 OSPF DOMAIN ID:0x0005:0x0000007D0200
        OSPF RT:0.0.0.0:3:0 OSPF ROUTER ID:10.1.122.2:0
      mpls labels in/out nolabel/220
BGP routing table entry for 5:5:172.16.1.11/32, version 10
Paths: (1 available, best #1, table CA)
Multipath: iBGP
  Not advertised to any peer
  Local, imported path from 1:2:172.16.1.11/32
    1.1.1.1 (metric 32) from 1.1.1.1 (1.1.1.1)
      Origin incomplete, metric 11, localpref 100, valid, internal, best
      Extended Community: RT:125:125 OSPF DOMAIN ID:0x0005:0x0000007D0200
        OSPF RT:0.0.0.0:3:0 OSPF ROUTER ID:10.1.111.1:0
      mpls labels in/out nolabel/120

Now it is time to compare the BGP attributes in order to comply with the BGP requirements for Multipath.

1. Prefer the path with the highest WEIGHT: 0 for both routes.

2. Prefer the path with the highest LOCAL_PREF: 100 for both routes.

3. Prefer the path that was locally originated via a network or aggregate BGP subcommand or through redistribution from an IGP: neither path is locally originated (sourced attribute does not appear).

4. Prefer the path with the shortest AS_PATH: iBGP paths originated in the same AS (Local).

5. Prefer the path with the lowest origin type: incomplete because the routes were redistributed.

6. Prefer the path with the lowest multi-exit discriminator (MED): MEDs are different. R1 reports a MED value of 11 to R5 and R2 reports a MED value of 21. In order to achieve BGP Multipath these values must be equal.

R2(config)#route-map SET_MED
R2(config-route-map)#set metric 11

 

R2(config)#router bgp 200
R2(config-router)#address-family vpnv4 unicast
R2(config-router-af)#neighbor 5.5.5.5 route-map SET_MED out

R5#show bgp vpnv4 unicast all 172.16.1.11
BGP routing table entry for 1:2:172.16.1.11/32, version 11
Paths: (2 available, best #1, no table)
  Not advertised to any peer
  Local
    1.1.1.1 (metric 32) from 1.1.1.1 (1.1.1.1)
      Origin incomplete, metric 11, localpref 100, valid, internal, best
      Extended Community: RT:125:125 OSPF DOMAIN ID:0x0005:0x0000007D0200
        OSPF RT:0.0.0.0:3:0 OSPF ROUTER ID:10.1.111.1:0
      mpls labels in/out nolabel/120
  Local
    2.2.2.2 (metric 32) from 2.2.2.2 (2.2.2.2)
      Origin incomplete, metric 11, localpref 100, valid, internal
      Extended Community: RT:125:125 OSPF DOMAIN ID:0x0005:0x0000007D0200
        OSPF RT:0.0.0.0:3:0 OSPF ROUTER ID:10.1.122.2:0
      mpls labels in/out nolabel/220

7. Prefer eBGP over iBGP paths: both paths are iBGP (internal).

8. Prefer the path with the lowest IGP metric to the BGP next hop: both routes have the same OSPF cost to the next-hop (32).

R5#show ip route 1.1.1.1
Routing entry for 1.1.1.1/32
  Known via “ospf 1”, distance 110, metric 32, type intra area
  Last update from 10.1.59.9 on FastEthernet0/0, 02:16:05 ago
  Routing Descriptor Blocks:
  * 10.1.59.9, from 1.1.1.1, 02:16:05 ago, via FastEthernet0/0
      Route metric is 32, traffic share count is 1

 

R5#show ip route 2.2.2.2
Routing entry for 2.2.2.2/32
  Known via “ospf 1”, distance 110, metric 32, type intra area
  Last update from 10.1.59.9 on FastEthernet0/0, 02:16:11 ago
  Routing Descriptor Blocks:
  * 10.1.59.9, from 2.2.2.2, 02:16:11 ago, via FastEthernet0/0
      Route metric is 32, traffic share count is 1

9. Determine if multiple paths require installation in the routing table for BGP Multipath: There are two iBGP VPNv4 paths for the final destination 172.16.1.11/32.

Once all the BGP attributes required for Multipath are equal, we can check if multipath has been enabled.

R5#show bgp vpnv4 unicast all 172.16.1.11
BGP routing table entry for 1:2:172.16.1.11/32, version 5
Paths: (2 available, best #1, no table)
  Not advertised to any peer
  Local
    1.1.1.1 (metric 32) from 1.1.1.1 (1.1.1.1)
      Origin incomplete, metric 11, localpref 100, valid, internal, best
      Extended Community: RT:125:125 OSPF DOMAIN ID:0x0005:0x0000007D0200
        OSPF RT:0.0.0.0:3:0 OSPF ROUTER ID:10.1.111.1:0
      mpls labels in/out nolabel/120
  Local
    2.2.2.2 (metric 32) from 2.2.2.2 (2.2.2.2)
      Origin incomplete, metric 11, localpref 100, valid, internal
      Extended Community: RT:125:125 OSPF DOMAIN ID:0x0005:0x0000007D0200
        OSPF RT:0.0.0.0:3:0 OSPF ROUTER ID:10.1.122.2:0
      mpls labels in/out nolabel/220
BGP routing table entry for 5:5:172.16.1.11/32, version 19
Paths: (2 available, best #2, table CA)
Multipath: iBGP
Flag: 0x800
  Not advertised to any peer
  Local, imported path from 1:2:172.16.1.11/32
    2.2.2.2 (metric 32) from 2.2.2.2 (2.2.2.2)
      Origin incomplete, metric 11, localpref 100, valid, internal, multipath
      Extended Community: RT:125:125 OSPF DOMAIN ID:0x0005:0x0000007D0200
        OSPF RT:0.0.0.0:3:0 OSPF ROUTER ID:10.1.122.2:0
      mpls labels in/out nolabel/220
  Local, imported path from 1:2:172.16.1.11/32
    1.1.1.1 (metric 32) from 1.1.1.1 (1.1.1.1)
      Origin incomplete, metric 11, localpref 100, valid, internal, multipath, best
      Extended Community: RT:125:125 OSPF DOMAIN ID:0x0005:0x0000007D0200
        OSPF RT:0.0.0.0:3:0 OSPF ROUTER ID:10.1.111.1:0
      mpls labels in/out nolabel/120

Notice that BGP Multipath has been configured under the BGP address-family IPV4 unicast vrf CA on R5; this is why we see the multipath attributes under the BGP VPNv4 routes with RD 5:5  (R5’s RD). Finally the routes are installed in the Routing Table for R5’s VRF CA.

R5#show ip route vrf CA
Gateway of last resort is not set
     172.16.0.0/32 is subnetted, 3 subnets
B       172.16.1.12 [200/11] via 2.2.2.2, 00:03:00
O IA    172.16.1.15 [110/11] via 10.1.155.15, 00:10:19, FastEthernet0/1
B       172.16.1.11 [200/11] via 2.2.2.2, 00:03:00
                    [200/11] via 1.1.1.1, 00:03:00
     10.0.0.0/24 is subnetted, 4 subnets
B       10.1.1.0 [200/11] via 2.2.2.2, 00:03:00
B       10.1.111.0 [200/0] via 1.1.1.1, 00:03:00
B       10.1.122.0 [200/11] via 2.2.2.2, 00:03:00
C       10.1.155.0 is directly connected, FastEthernet0/1
R5#show ip cef vrf CA 172.16.1.11 255.255.255.255
172.16.1.11/32, version 19, epoch 0, per-destination sharing
0 packets, 0 bytes
  tag information set
    local tag: VPN-route-head
  via 2.2.2.2, 0 dependencies, recursive
    traffic share 1
    next hop 10.1.59.9, FastEthernet0/0 via 2.2.2.2/32
    valid adjacency
    tag rewrite with Fa0/0, 10.1.59.9, tags imposed: {909 220}
  via 1.1.1.1, 0 dependencies, recursive
    traffic share 1
    next hop 10.1.59.9, FastEthernet0/0 via 1.1.1.1/32
    valid adjacency
    tag rewrite with Fa0/0, 10.1.59.9, tags imposed: {908 120}
  0 packets, 0 bytes switched through the prefix
  tmstats: external 0 packets, 0 bytes
           internal 0 packets, 0 bytes

If we do a traceroute to the final destination we can see that load sharing has been enabled.

R5#traceroute vrf CA 172.16.1.11
  1 10.1.59.9 [MPLS: Labels 909/220 Exp 0] 88 msec 64 msec 44 msec
  2 10.1.109.10 [MPLS: Labels 10006/120 Exp 0] 72 msec 56 msec 56 msec
  3 10.1.107.7 [MPLS: Labels 701/220 Exp 0] 60 msec 56 msec 64 msec
  4 10.1.111.1 [MPLS: Label 120 Exp 0] 24 msec
    10.1.122.2 [MPLS: Label 220 Exp 0] 68 msec
    10.1.111.1 [MPLS: Label 120 Exp 0] 44 msec
  5 10.1.122.12 44 msec
    10.1.111.11 64 msec
    10.1.122.12 44 msec

R15 sees its normal IA OSPF route to the destination with next-hop R5’s Fa0/1 IP address.

R15#show ip route ospf
     172.16.0.0/32 is subnetted, 3 subnets
O IA    172.16.1.12 [110/21] via 10.1.155.5, 00:17:34, FastEthernet0/0
O IA    172.16.1.11 [110/21] via 10.1.155.5, 00:17:34, FastEthernet0/0
     10.0.0.0/24 is subnetted, 4 subnets
O IA    10.1.1.0 [110/21] via 10.1.155.5, 00:17:34, FastEthernet0/0
O IA    10.1.111.0 [110/11] via 10.1.155.5, 00:17:34, FastEthernet0/0
O IA    10.1.122.0 [110/21] via 10.1.155.5, 00:17:34, FastEthernet0/0

BGP Multipath – eiBGP

Figure 6 illustrates the routing decision to R14’s loopback, 172.16.1.14/32.

Figure 6

Customer B Site A is running EBGP between PEs (AS 200) and CEs (AS 346). R13 and R14 are iBGP neighbours (AS 346) and are also EBGP peers with R3 and R14 (AS 200).

We study now how R4 routes traffic to 172.16.1.14/32 (R14’s loopback). As you can see in the output below, R4 prefers its EBGP route towards the destination (external) although it also receives an alternative iBGP route from R3.

R4#show bgp vpnv4 unicast all 172.16.1.14
BGP routing table entry for 3:4:172.16.1.14/32, version 20
Paths: (2 available, best #1, table CB)
  Advertised to update-groups:
     1
  346
    10.1.144.14 from 10.1.144.14 (14.14.14.14)
      Origin IGP, metric 0, localpref 100, valid, external, best
      Extended Community: RT:346:346
      mpls labels in/out 419/nolabel
  346
    3.3.3.3 (metric 21) from 3.3.3.3 (3.3.3.3)
      Origin IGP, metric 0, localpref 100, valid, internal
      Extended Community: RT:346:346
      mpls labels in/out 419/318

Therefore having R4 able to see both routes we proceed to configure Multipath. Notice this time one route is EBGP (best) and the other route is iBGP so we need to use eibgp option in order to enable load sharing.

R4(config)#router bgp 200
R4(config-router)#address-family ipv4 vrf CB
R4(config-router-af)#maximum-paths eibgp 2 import 2

Next step is verifying the Multipath requirements specified by BGP algorithm.

1. Prefer the path with the highest WEIGHT: both routes have the default value of 0.

2. Prefer the path with the highest LOCAL_PREF: both routes have the default value of 100.

3. Prefer the path that was locally originated via a network or aggregate BGP subcommand or through redistribution from an IGP: none of them is locally originated.

4. Prefer the path with the shortest AS_PATH: both routes have the same value, 346.

5. Prefer the path with the lowest origin type: both routes have IGP origin type.

6. Prefer the path with the lowest multi-exit discriminator (MED): both routes have a metric of 0.

7. Prefer eBGP over iBGP paths: the route through R14 is preferred as it is an EBGP route. Route through R3 is an iBGP route. However, the algorithm says: If bestpath is selected, go to Step 9 (multipath).

8. Prefer the path with the lowest IGP metric to the BGP next hop: this step is bypassed by the algorithm, so having different IGP metrics is not going to affect eibgp Multipath.

R4#show ip route vrf CB 10.1.144.14
Routing entry for 10.1.144.0/24
  Known via “connected”, distance 0, metric 0 (connected, via interface)
  Routing Descriptor Blocks:
  * directly connected, via FastEthernet0/1
      Route metric is 0, traffic share count is 1
R4#show ip route 3.3.3.3
Routing entry for 3.3.3.3/32
  Known via “ospf 1”, distance 110, metric 21, type intra area
  Last update from 10.1.48.8 on FastEthernet0/0, 00:18:33 ago
  Routing Descriptor Blocks:
  * 10.1.48.8, from 3.3.3.3, 00:18:33 ago, via FastEthernet0/0
      Route metric is 21, traffic share count is 1

9. Determine if multiple paths require installation in the routing table for BGP Multipath: we have two possible paths for eibgp Multipath through R14 and through R3.

R4#show bgp vpnv4 unicast all 172.16.1.14
BGP routing table entry for 3:4:172.16.1.14/32, version 34
Paths: (2 available, best #1, table CB)
Multipath: eiBGP
Flag: 0x820
  Advertised to update-groups:
     1
  346
    10.1.144.14 from 10.1.144.14 (14.14.14.14)
      Origin IGP, metric 0, localpref 100, valid, external, multipath, best
      Extended Community: RT:346:346
      mpls labels in/out 420/nolabel
  346
    3.3.3.3 (metric 21) from 3.3.3.3 (3.3.3.3)
      Origin IGP, metric 0, localpref 100, valid, internal, multipath
      Extended Community: RT:346:346
      mpls labels in/out 420/318
R4#show ip route vrf CB
Gateway of last resort is not set
     172.16.0.0/32 is subnetted, 3 subnets
B       172.16.1.16 [200/0] via 6.6.6.6, 00:15:31
B       172.16.1.13 [20/0] via 10.1.144.14, 00:11:22
                    [20/0] via 3.3.3.3, 00:11:22
B       172.16.1.14 [20/0] via 10.1.144.14, 00:11:22
                    [20/0] via 3.3.3.3, 00:11:22
     10.0.0.0/24 is subnetted, 1 subnets
C       10.1.144.0 is directly connected, FastEthernet0/1

BGP Multipath – BGP VPNv4 Route Reflector (RR)

Figure 7

In this topology R10 is configured as BGP VPNv4 Route Reflector (RR). BGP RRs avoid the need of a full mesh between BGP peers but hide topology information in return. In this scenario the PEs peer with R10 and send their BGP VPNv4 routes, then R10 runs BGP Best Path Selection algorithm and advertises by default only its best BGP VPNv4 route for each destination to each PEs.

R1 – R6:

router bgp 200
 no neighbor 1.1.1.1 remote-as 200
 no neighbor 2.2.2.2 remote-as 200
 no neighbor 3.3.3.3 remote-as 200
 no neighbor 4.4.4.4 remote-as 200
 no neighbor 5.5.5.5 remote-as 200
 no neighbor 6.6.6.6 remote-as 200
 neighbor 10.10.10.10 remote-as 200
 neighbor 10.10.10.10 update-source Loopback0
 !
 address-family vpnv4
  neighbor 10.10.10.10 activate

R10:

router bgp 200
 no bgp default ipv4-unicast
 bgp log-neighbor-changes
 neighbor 1.1.1.1 remote-as 200
 neighbor 1.1.1.1 update-source Loopback00
 neighbor 2.2.2.2 remote-as 200
 neighbor 2.2.2.2 update-source Loopback0
 neighbor 3.3.3.3 remote-as 200
 neighbor 3.3.3.3 update-source Loopback0
 neighbor 4.4.4.4 remote-as 200
 neighbor 4.4.4.4 update-source Loopback0
 neighbor 5.5.5.5 remote-as 200
 neighbor 5.5.5.5 update-source Loopback0
 neighbor 6.6.6.6 remote-as 200
 neighbor 6.6.6.6 update-source Loopback0
 address-family vpnv4
  neighbor 1.1.1.1 activate
  neighbor 1.1.1.1 route-reflector-client
  neighbor 2.2.2.2 activate
  neighbor 2.2.2.2 route-reflector-client
  neighbor 3.3.3.3 activate
  neighbor 3.3.3.3 route-reflector-client
  neighbor 4.4.4.4 activate
  neighbor 4.4.4.4 route-reflector-client
  neighbor 5.5.5.5 activate
  neighbor 5.5.5.5 route-reflector-client
  neighbor 6.6.6.6 activate
  neighbor 6.6.6.6 route-reflector-client
 exit-address-family

R10, the Route Reflector (RR), receives all the BGP VPNv4 Updates from the PE routers with all the information about the customer routes. If we focus again on R11’s Loopback, 172.16.1.11/32, we can see that R10 receives the two available paths through R1 and R2 and R1, as we saw, wins the BGP Best Path Selection algorithm due to a lower metric (11<21).

R10#show bgp vpnv4 uni all
   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 1:2
* i10.1.1.0/24      2.2.2.2                 20    100      0 ?
*>i                 1.1.1.1                 20    100      0 ?
* i10.1.111.0/24    2.2.2.2                 30    100      0 ?
*>i                 1.1.1.1                  0    100      0 ?
*>i10.1.122.0/24    2.2.2.2                  0    100      0 ?
* i                 1.1.1.1                 30    100      0 ?
*>i172.16.1.11/32   1.1.1.1                 11    100      0 ?
* i                 2.2.2.2                 21    100      0 ?
*>i172.16.1.12/32   2.2.2.2                 11    100      0 ?
* i                 1.1.1.1                 21    100      0 ?
Route Distinguisher: 3:4
* i172.16.1.13/32   4.4.4.4                  0    100      0 346 i
*>i                 3.3.3.3                  0    100      0 346 i
* i172.16.1.14/32   4.4.4.4                  0    100      0 346 i
*>i                 3.3.3.3                  0    100      0 346 i
Route Distinguisher: 5:5
*>i10.1.155.0/24    5.5.5.5                  0    100      0 ?
*>i172.16.1.15/32   5.5.5.5                 11    100      0 ?
Route Distinguisher: 6:6
*>i172.16.1.16/32   6.6.6.6                  0    100      0 346 i

Figure 8

If we check now what routes R5 receives from the RR, we can verify in the output below that only has one route to get to the final destination and this route is through R1 which is the best route of R10 for 1:2:172.16.1.11/32. This route is then imported to VRF CA thanks to redistribution and advertised to R15.

R5#show bgp vpnv4 unicast all 172.16.1.11/32
BGP routing table entry for 1:2:172.16.1.11/32, version 43
Paths: (1 available, best #1, no table)
  Not advertised to any peer
  Local
    1.1.1.1 (metric 32) from 10.10.10.10 (10.10.10.10)
      Origin incomplete, metric 11, localpref 100, valid, internal, best
      Extended Community: RT:125:125 OSPF DOMAIN ID:0x0005:0x0000007D0200
        OSPF RT:0.0.0.0:3:0 OSPF ROUTER ID:10.1.111.1:0
      Originator: 1.1.1.1, Cluster list: 10.10.10.10
      mpls labels in/out nolabel/120
BGP routing table entry for 5:5:172.16.1.11/32, version 44
Paths: (1 available, best #1, table CA)
Multipath: iBGP
  Not advertised to any peer
  Local, imported path from 1:2:172.16.1.11/32
    1.1.1.1 (metric 32) from 10.10.10.10 (10.10.10.10)
      Origin incomplete, metric 11, localpref 100, valid, internal, best
      Extended Community: RT:125:125 OSPF DOMAIN ID:0x0005:0x0000007D0200
        OSPF RT:0.0.0.0:3:0 OSPF ROUTER ID:10.1.111.1:0
      Originator: 1.1.1.1, Cluster list: 10.10.10.10
      mpls labels in/out nolabel/120

Why R10, the RR, is not advertising both routes to R5? Because by default BGP VPNv4 RR only chooses one best path for the same VPNv4 prefix: RD:Prefix/Length. In our topology, this means that R10 is receiving 1:2:172.16.1.11/32 from R1 and R2 and R10 is not able to differentiate this VPNv4 prefix (it is the same prefix coming from two different PE neighbors). Therefore, it is only able to choose one as the best path.

There are two solutions to this possible scenario:

  • Having a BGP VPNv4 Full Mesh between all PEs as I explained in previous sections.
  • Making the BGP VPNv4 routes look different changing the RD.
    • R1: RD: 1:2
    • R2: RD: 2:2

BGP Multipath – iBGP

Figure 9

In order to be able to do multipathing on R5, first of all we need to be sure that R5 is receiving both routes. So I decided to implement the change of RD: change R2’s RD to 2:2.

R2(config)#no vrf definition CA
% IP addresses from all interfaces in VRF CA have been removed
R2(config)# vrf definition CA
R2(config-vrf)#rd 2:2
R2(config-vrf)#route-target export 125:125
R2(config-vrf)#route-target import 125:125
R2(config-vrf)#address-family ipv4
R2(config-vrf-af)#exit-address-family

 

R2(config-vrf)#int Fa0/1
R2(config-if)#vrf forwarding CA
R2(config-if)#ip address 10.1.111.1 255.255.255.0

Now R10 sees the routes as different BGP VPNv4 prefixes, 1:2:172.16.1.11/32 and 2:2:172.16.1.11/32, and can run BGP Best Selection Path algorithm independently resulting in both routes being winners of the BGP Best Selection Path algorithm.

R10#show bgp vpnv4 uni all
   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 1:2
*>i10.1.1.0/24      1.1.1.1                 20    100      0 ?
*>i10.1.111.0/24    1.1.1.1                  0    100      0 ?
*>i10.1.122.0/24    1.1.1.1                 30    100      0 ?
*>i172.16.1.11/32   1.1.1.1                 11    100      0 ?
*>i172.16.1.12/32   1.1.1.1                 21    100      0 ?
Route Distinguisher: 2:2
*>i10.1.1.0/24      2.2.2.2                 20    100      0 ?
*>i10.1.111.0/24    2.2.2.2                 30    100      0 ?
*>i10.1.122.0/24    2.2.2.2                  0    100      0 ?
*>i172.16.1.11/32   2.2.2.2                 21    100      0 ?
*>i172.16.1.12/32   2.2.2.2                 11    100      0 ?
[…]

Then we check on R5 whether or not it sees both routes reflected by R10. R5 is able to install both BGP VPNv4 routes in its BGP table but only one is selected as best route (through R1 because of the lower metric).

R5#show bgp vpnv4 unicast all 172.16.1.11/32
BGP routing table entry for 1:2:172.16.1.11/32, version 66
Paths: (1 available, best #1, no table)
  Not advertised to any peer
  Local
    1.1.1.1 (metric 32) from 10.10.10.10 (10.10.10.10)
      Origin incomplete, metric 11, localpref 100, valid, internal, best
      Extended Community: RT:125:125 OSPF DOMAIN ID:0x0005:0x0000007D0200
        OSPF RT:0.0.0.0:3:0 OSPF ROUTER ID:10.1.111.1:0
      Originator: 1.1.1.1, Cluster list: 10.10.10.10
      mpls labels in/out nolabel/120
BGP routing table entry for 2:2:172.16.1.11/32, version 73
Paths: (1 available, best #1, no table)
  Not advertised to any peer
  Local
    2.2.2.2 (metric 32) from 10.10.10.10 (10.10.10.10)
      Origin incomplete, metric 21, localpref 100, valid, internal, best
      Extended Community: RT:125:125 OSPF DOMAIN ID:0x0005:0x0000007D0200
        OSPF RT:0.0.0.0:3:0 OSPF ROUTER ID:10.1.122.2:0
      Originator: 2.2.2.2, Cluster list: 10.10.10.10
      mpls labels in/out nolabel/224
BGP routing table entry for 5:5:172.16.1.11/32, version 77
Paths: (2 available, best #2, table CA)
Multipath: iBGP
  Not advertised to any peer
  Local, imported path from 2:2:172.16.1.11/32
    2.2.2.2 (metric 32) from 10.10.10.10 (10.10.10.10)
      Origin incomplete, metric 21, localpref 100, valid, internal
      Extended Community: RT:125:125 OSPF DOMAIN ID:0x0005:0x0000007D0200
        OSPF RT:0.0.0.0:3:0 OSPF ROUTER ID:10.1.122.2:0
      Originator: 2.2.2.2, Cluster list: 10.10.10.10
      mpls labels in/out nolabel/224
  Local, imported path from 1:2:172.16.1.11/32
    1.1.1.1 (metric 32) from 10.10.10.10 (10.10.10.10)
      Origin incomplete, metric 11, localpref 100, valid, internal, best
      Extended Community: RT:125:125 OSPF DOMAIN ID:0x0005:0x0000007D0200
        OSPF RT:0.0.0.0:3:0 OSPF ROUTER ID:10.1.111.1:0
      Originator: 1.1.1.1, Cluster list: 10.10.10.10
      mpls labels in/out nolabel/120

Up to this point R5 can see both routes, so we can enable Multipath on R5. Remember that, as we saw before, in this scenario we need to have the same MED for this to actually work.

R2(config)#route-map SET_MED permit 10
R2(config-route-map)# set metric 11

 

R2(config)#router bgp 200
R2(config-router)# address-family vpnv4
R2(config-router-af)#  neighbor 10.10.10.10 route-map SET_MED out

R5(config)#router bgp 200
R5(config-router)#address-family ipv4 unicast vrf CA
R5(config-router-af)#maximum-paths ibgp 2 import 2

Finally, R5 has BGP Multipath enabled as we can check on the output below.

R5#show bgp vpnv4 unicast all 172.16.1.11/32
[…]
BGP routing table entry for 5:5:172.16.1.11/32, version 86
Paths: (2 available, best #2, table CA)
Multipath: iBGP
Flag: 0x800
  Not advertised to any peer
  Local, imported path from 2:2:172.16.1.11/32
    2.2.2.2 (metric 32) from 10.10.10.10 (10.10.10.10)
      Origin incomplete, metric 11, localpref 100, valid, internal, multipath
      Extended Community: RT:125:125 OSPF DOMAIN ID:0x0005:0x0000007D0200
        OSPF RT:0.0.0.0:3:0 OSPF ROUTER ID:10.1.122.2:0
      Originator: 2.2.2.2, Cluster list: 10.10.10.10
      mpls labels in/out nolabel/224
  Local, imported path from 1:2:172.16.1.11/32
    1.1.1.1 (metric 32) from 10.10.10.10 (10.10.10.10)
      Origin incomplete, metric 11, localpref 100, valid, internal, multipath, best
      Extended Community: RT:125:125 OSPF DOMAIN ID:0x0005:0x0000007D0200
        OSPF RT:0.0.0.0:3:0 OSPF ROUTER ID:10.1.111.1:0
      Originator: 1.1.1.1, Cluster list: 10.10.10.10
      mpls labels in/out nolabel/120

We can actually confirm the traffic path based on the CEF table information and running a traceroute command.

R5#show ip cef vrf CA 172.16.1.11 255.255.255.255
172.16.1.11/32, version 35, epoch 0, per-destination sharing
0 packets, 0 bytes
  tag information set
    local tag: VPN-route-head
  via 2.2.2.2, 0 dependencies, recursive
    traffic share 1
    next hop 10.1.59.9, FastEthernet0/0 via 2.2.2.2/32
    valid adjacency
    tag rewrite with Fa0/0, 10.1.59.9, tags imposed: {909 224}
  via 1.1.1.1, 0 dependencies, recursive
    traffic share 1
    next hop 10.1.59.9, FastEthernet0/0 via 1.1.1.1/32
    valid adjacency
    tag rewrite with Fa0/0, 10.1.59.9, tags imposed: {908 120}
  0 packets, 0 bytes switched through the prefix
  tmstats: external 0 packets, 0 bytes
           internal 0 packets, 0 bytes

 

R5#traceroute vrf CA 172.16.1.11
  1 10.1.59.9 [MPLS: Labels 909/224 Exp 0] 44 msec 64 msec 68 msec
  2 10.1.109.10 [MPLS: Labels 10006/120 Exp 0] 44 msec 68 msec 52 msec
  3 10.1.107.7 [MPLS: Labels 701/224 Exp 0] 56 msec 44 msec 40 msec
  4 10.1.111.1 [MPLS: Label 120 Exp 0] 48 msec
    10.1.122.2 [MPLS: Label 224 Exp 0] 64 msec
    10.1.111.1 [MPLS: Label 120 Exp 0] 44 msec
  5 10.1.122.12 44 msec
    10.1.111.11 68 msec
    10.1.122.12 64 msec

BGP Prefix Independent Convergence (PIC)

BGP PIC is similar to Fast Reroute for EIGRP or OSPF. This feature creates and stores a backup path in the RIB, FIB and CEF so when a failure is detected, the backup path can immediately take over, thus enabling fast failover.

Figure 10

In order to enable PIC, first of all, we need to have the primary and the backup route in the BGP table. In this example the goal is to configure R1 to enable BGP PIC for the prefix 172.16.1.11/32 (R11’s loopback).

Because R1 and R2 have two different RDs configured under the VRF, R10 is able to reflect both BGP VPNv4 routes to the PEs.

R10(config-router-af)# do show bgp vpnv4 uni all
     Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 1:2
 *>i 10.1.1.0/24      1.1.1.1                 11    100      0 ?
 *>i 10.1.111.0/24    1.1.1.1                  0    100      0 ?
 *>i 10.1.122.0/24    1.1.1.1                 21    100      0 ?
 *>i 172.16.1.11/32   1.1.1.1                  2    100      0 ?
 *>i 172.16.1.12/32   1.1.1.1                 12    100      0 ?
Route Distinguisher: 2:2
 *>i 10.1.1.0/24      2.2.2.2                 11    100      0 ?
 *>i 10.1.111.0/24    2.2.2.2                 11    100      0 ?
 *>i 10.1.122.0/24    2.2.2.2                 11    100      0 ?
 *>i 172.16.1.11/32   2.2.2.2                 11    100      0 ?
 *>i 172.16.1.12/32   2.2.2.2                 11    100      0 ?
[…]
R1#show bgp vpnv4 unicast all
   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 1:2 (default for vrf CA)
* i10.1.1.0/24      2.2.2.2                 11    100      0 ?
*>                  10.1.111.11             20         32768 ?
* i10.1.111.0/24    2.2.2.2                 11    100      0 ?
*>                  0.0.0.0                  0         32768 ?
* i10.1.122.0/24    2.2.2.2                 11    100      0 ?
*>                  10.1.111.11             30         32768 ?
*>i10.1.155.0/24    5.5.5.5                  0    100      0 ?
* i172.16.1.11/32   2.2.2.2                 11    100      0 ?
*>                  10.1.111.11             11         32768 ?
* i172.16.1.12/32   2.2.2.2                 11    100      0 ?
*>                  10.1.111.11             21         32768 ?
*>i172.16.1.15/32   5.5.5.5                 11    100      0 ?
[…]

We enable BGP PIC on R1.

R1(config)#router bgp 200
R1(config-router)#address-family vpnv4 unicast
R1(config-router-af)#bgp additional-paths ?
  install  Additional paths to install into RIB
  select   Selection criteria to pick the paths
R1(config-router-af)#bgp additional-paths install

 

R1#show bgp vpnv4 unicast all
BGP table version is 29, local router ID is 1.1.1.1
Status codes: s suppressed, d damped, h history, * valid, > best, i – internal,
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
              x best-external, a additional-path, c RIB-compressed,
Origin codes: i – IGP, e – EGP, ? – incomplete
RPKI validation codes: V valid, I invalid, N Not found     Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 1:2 (default for vrf CA)
 *bi 10.1.1.0/24      2.2.2.2                 11    100      0 ?
 *>                   10.1.111.11             11         32768 ?
 *bi 10.1.111.0/24    2.2.2.2                 11    100      0 ?
 *>                   0.0.0.0                  0         32768 ?
 *bi 10.1.122.0/24    2.2.2.2                 11    100      0 ?
 *>                   10.1.111.11             21         32768 ?
 *>i 10.1.155.0/24    5.5.5.5                  0    100      0 ?
 *bi 172.16.1.11/32   2.2.2.2                 11    100      0 ?
 *>                   10.1.111.11              2         32768 ?
 *bi 172.16.1.12/32   2.2.2.2                 11    100      0 ?
 *>                   10.1.111.11             12         32768 ?
 *>i 172.16.1.15/32   5.5.5.5                 11    100      0 ?
[…]

The best route and the backup/repair route go into the RIB and the FIB and the best route is used, as normal, for forwarding.

R1#show bgp vpnv4 unicast all 172.16.1.11/32
BGP routing table entry for 1:2:172.16.1.11/32, version 20
Paths: (2 available, best #2, table CA)
  Additional-path-install
  Advertised to update-groups:
     1
  Refresh Epoch 1
  Local, imported path from 2:2:172.16.1.11/32 (global)
    2.2.2.2 (metric 12) from 10.10.10.10 (10.10.10.10)
      Origin incomplete, metric 11, localpref 100, valid, internal, backup/repair
      Extended Community: RT:125:125 OSPF DOMAIN ID:0x0005:0x0000007D0200
        OSPF RT:0.0.0.0:3:0 OSPF ROUTER ID:10.1.122.2:0
      Originator: 2.2.2.2, Cluster list: 10.10.10.10 , recursive-via-host
      mpls labels in/out 120/224
      rx pathid: 0, tx pathid: 0
  Refresh Epoch 1
  Local
    10.1.111.11 from 0.0.0.0 (1.1.1.1)
      Origin incomplete, metric 2, localpref 100, weight 32768, valid, sourced, best
      Extended Community: RT:125:125 OSPF DOMAIN ID:0x0005:0x0000007D0200
        OSPF RT:0.0.0.0:3:0 OSPF ROUTER ID:10.1.111.1:0
      mpls labels in/out 120/nolabel
      rx pathid: 0, tx pathid: 0x0

However, having only a repair route is not enough to improve BGP Convergence Time. We still need to run Bidirectional Forwarding Detection (BFD) for faster failure detection in order to switch to the repair/backup path. This way, we do not have to wait for the withdraw/update messages before electing the next backup path.

BGP Best Path External

In this section, we try to achieve the same result that we saw in the previous section configuring BGP PIC but using the same RD. In addition, we are going to modify BGP Local Preference (LP) attribute to show the Best Path External feature.

Figure 11 shows the new scenario.

Figure 11

R10 receives the BGP VPNv4 prefix about R14’s loopback (3:4:172.16.1.14/32) from R3 and R4.

R10#show bgp vpnv4 uni all
[…]
   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 3:4
*>i172.16.1.13/32   3.3.3.3                  0    100      0 346 i
* i                 4.4.4.4                  0    100      0 346 i
*>i172.16.1.14/32   3.3.3.3                  0    100      0 346 i
* i                 4.4.4.4                  0    100      0 346 i
[…]

The route advertised by R3 is preferred because of the lowest Router ID value (3.3.3.3 < 4.4.4.4) and it is advertised to R4.

R4#show bgp vpnv4 unicast all
   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 3:4 (default for vrf CB)
* i172.16.1.13/32   3.3.3.3                  0    100      0 346 i
*>                  10.1.144.14                            0 346 i
* i172.16.1.14/32   3.3.3.3                  0    100      0 346 i
*>                  10.1.144.14              0             0 346 i
*>i172.16.1.16/32   6.6.6.6                  0    100      0 346 i
Route Distinguisher: 6:6
*>i172.16.1.16/32   6.6.6.6                  0    100      0 346 i

However, R4 still prefers its EBGP route to the final destination through R14.

R4#show bgp vpnv4 unicast all 172.16.1.14
BGP routing table entry for 3:4:172.16.1.14/32, version 23
Paths: (2 available, best #2, table CB)
Multipath: eiBGP
  Advertised to update-groups:
     2
  346
    3.3.3.3 (metric 21) from 10.10.10.10 (10.10.10.10)
      Origin IGP, metric 0, localpref 100, valid, internal, multipath
      Extended Community: RT:346:346
      Originator: 3.3.3.3, Cluster list: 10.10.10.10
      mpls labels in/out 418/318
  346
    10.1.144.14 from 10.1.144.14 (14.14.14.14)
      Origin IGP, metric 0, localpref 100, valid, external, multipath, best
      Extended Community: RT:346:346
      mpls labels in/out 418/nolabel

Now we change the BGP VPNv4 LP to a high value on R3 for 3:4:172.16.1.14/32.

Figure 12

R3(config)#route-map SET_LOCAL_PREFERENCE
R3(config-route-map)#set local-preference 200

 

R3(config)#router bgp 200
R3(config-router)#address-family vpnv4 uni
R3(config-router-af)#neighbor 10.10.10.10 route-map SET_LOCAL_PREFERENCE out

If we check R10’s BGP VPNv4 table, the RR only sees the path through R3. The reason is R10 chooses R3 as the best next-hop for the destination and advertises it to R4. R4 runs BGP Best Path Selection algorithm and also prefers the route through R3 because of the higher LP (200>100) and R4 stops advertising its path to the RR.

R10#show bgp vpnv4 uni all
[…]
   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 3:4
*>i172.16.1.13/32   3.3.3.3                  0    200      0 346 i
*>i172.16.1.14/32   3.3.3.3                  0    200      0 346 i
[…]

Now R4 chooses to go through the MPLS network instead of using its directly connected link to R14. R4 losing its alternative EBGP path via R14 might not be desirable to speed convergence. For this specific scenario we use BGP best external feature.

R4(config)#router bgp 200
R4(config-router)#address-family vpnv4 unicast
R4(config-router-af)#bgp advertise-best-external

If we check the BGP VPNv4 table on R4 we can see the status “x” which shows that best external is enabled.

R4#show bgp vpnv4 unicast all
BGP table version is 19, local router ID is 4.4.4.4
Status codes: s suppressed, d damped, h history, * valid, > best, i – internal,
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
              x best-external, a additional-path, c RIB-compressed,
Origin codes: i – IGP, e – EGP, ? – incomplete
RPKI validation codes: V valid, I invalid, N Not found     Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 3:4 (default for vrf CB)
 *>i 172.16.1.13/32   3.3.3.3                  0    200      0 346 i
 *b x                 10.1.144.14                            0 346 i
 *>i 172.16.1.14/32   3.3.3.3                  0    200      0 346 i
 *b x                 10.1.144.14              0             0 346 i
 *>i 172.16.1.16/32   6.6.6.6                  0    100      0 346 i
Route Distinguisher: 6:6
 *>i 172.16.1.16/32   6.6.6.6                  0    100      0 346 i

Now the alternate path can be installed as a backup/repair path on R4 and can also be advertised to the RR.

R4#show bgp vpnv4 unicast all 172.16.1.14
BGP routing table entry for 3:4:172.16.1.14/32, version 17
Paths: (2 available, best #1, table CB)
  Advertise-best-external
  Advertised to update-groups:
     1          3
  Refresh Epoch 1
  346
    3.3.3.3 (metric 12) from 10.10.10.10 (10.10.10.10)
      Origin IGP, metric 0, localpref 200, valid, internal, best
      Extended Community: RT:346:346
      Originator: 3.3.3.3, Cluster list: 10.10.10.10 , recursive-via-host
      mpls labels in/out 420/319
      rx pathid: 0, tx pathid: 0x0
  Refresh Epoch 1
  346
    10.1.144.14 from 10.1.144.14 (14.14.14.14)
      Origin IGP, metric 0, localpref 100, valid, external, backup/repair, advertise-best-external
      Extended Community: RT:346:346 , recursive-via-connected
      mpls labels in/out 420/nolabel
      rx pathid: 0, tx pathid: 0

 

R4#show ip cef vrf CB 172.16.1.14/32 detail
172.16.1.14/32, epoch 0, flags rib defined all labels
  local label info: other/420
  recursive via 3.3.3.3 label 319
    nexthop 10.1.48.8 FastEthernet0/0 label 800
  recursive via 10.1.144.14, repair
    attached to FastEthernet2/0

R10 knows about both prefixes now. However, R10 is only advertising the primary path or best path to R3. So at this point nothing has changed for R3 yet as the best route is through R3.

R10#show bgp vpnv4 unicast all
   Network          Next Hop            Metric LocPrf Weight Path
[…]
Route Distinguisher: 3:4
* i172.16.1.13/32   4.4.4.4                  0    100      0 346 i
*>i                 3.3.3.3                  0    200      0 346 i
* i172.16.1.14/32   4.4.4.4                  0    100      0 346 i
*>i                 3.3.3.3                  0    200      0 346 i
[…]

 

R10#show bgp vpnv4 uni all neighbors 3.3.3.3 advertised-routes
BGP table version is 78, local router ID is 10.10.10.10
Status codes: s suppressed, d damped, h history, * valid, > best, i – internal,
              r RIB-failure, S Stale
Origin codes: i – IGP, e – EGP, ? – incomplete   Network          Next Hop            Metric LocPrf Weight Path
[…]
Route Distinguisher: 3:4
*>i172.16.1.13/32   3.3.3.3                  0    200      0 346 i
*>i172.16.1.14/32   3.3.3.3                  0    200      0 346 i
[…]

At this point remember that the final objective is trying to achieve what we did previously configuring PIC with different RDs: being able to install both routes into RIB and FIB for fast failover. These are the steps we need to follow:

  • Run bgp additional-paths select backup command to trigger the calculation of the backup path at R10, which is the path via R4.
  • Run bgp additional-paths install command to install the backup path in the forwarding plane. (Do not configure this command if RR is in the control plane).
  • Run neighbor advertise diverse-path backup command to trigger the advertisement of the backup path to R3. R3 will learn the best path, (which is the path via R13 because of the higher LP), and it will learn the backup path from R10 (RR).
R10(config)#router bgp 200
R10(config-router)#address-family vpnv4 unicast
R10(config-router-af)#neighbor 3.3.3.3 advertise diverse-path backup
R10(config-router-af)#bgp additional-paths select best-external backup
R3(config)#router bgp 200
R3(config-router)#address-family vpnv4 unicast
R3(config-router-af)#bgp additional-paths select best-external backup
R3(config-router-af)#bgp additional-paths install

We can verify that R10 is advertising the additional path through R4 to R3.

R10(config-router-af)# do show bgp vpnv4 uni all neigh 3.3.3.3 advertised-routes
BGP table version is 15, local router ID is 10.10.10.10
Status codes: s suppressed, d damped, h history, * valid, > best, i – internal,
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
              x best-external, a additional-path, c RIB-compressed,
Origin codes: i – IGP, e – EGP, ? – incomplete
RPKI validation codes: V valid, I invalid, N Not found     Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 3:4
 *bia172.16.1.13/32   4.4.4.4                  0    100      0 346 i
 *bia172.16.1.14/32   4.4.4.4                  0    100      0 346 i

We check R3’S BGP VPNv4 table and we can see the backup route has been advertised and installed as a repair path.

R3#show bgp vpnv4 uni all
BGP table version is 17, local router ID is 3.3.3.3
Status codes: s suppressed, d damped, h history, * valid, > best, i – internal,
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
              x best-external, a additional-path, c RIB-compressed,
Origin codes: i – IGP, e – EGP, ? – incomplete
RPKI validation codes: V valid, I invalid, N Not found     Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 3:4 (default for vrf CB)
 *bi 172.16.1.13/32   4.4.4.4                  0    100      0 346 i
 *>                   10.1.133.13              0             0 346 i
 *bi 172.16.1.14/32   4.4.4.4                  0    100      0 346 i
 *>                   10.1.133.13                            0 346 i

 

R3#show bgp vpnv4 uni all 172.16.1.14
BGP routing table entry for 3:4:172.16.1.14/32, version 17
Paths: (2 available, best #2, table CB)
  Advertised to update-groups:
     4
  Refresh Epoch 2
  346
    4.4.4.4 (metric 12) from 10.10.10.10 (10.10.10.10)
      Origin IGP, metric 0, localpref 100, valid, internal, backup/repair
      Extended Community: RT:346:346
      Originator: 4.4.4.4, Cluster list: 10.10.10.10
      mpls labels in/out 319/420
      rx pathid: 0, tx pathid: 0
  Refresh Epoch 1
  346
    10.1.133.13 from 10.1.133.13 (13.13.13.13)
      Origin IGP, localpref 100, valid, external, best
      Extended Community: RT:346:346
      mpls labels in/out 319/nolabel
      rx pathid: 0, tx pathid: 0x0

R3#show ip cef vrf CB 172.16.1.14 255.255.255.255 detail
172.16.1.14/32, epoch 0, flags rib defined all labels
  local label info: other/313
  recursive via 10.1.133.13
    attached to FastEthernet2/0
  recursive via 4.4.4.4 label 412, repair
    nexthop 10.1.38.8 FastEthernet0/0 label 808

References

https://www.cisco.com/c/en/us/td/docs/ios/12_2sx/feature/guide/fsxeibmp.html
https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/iproute_bgp/configuration/xe-3s/irg-xe-3s-book/irg-bgp-mp-pic.html
https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/iproute_bgp/configuration/15-s/irg-15-s-book/irg_diverse_path.html
https://www.cisco.com/c/en/us/td/docs/ios/iproute_bgp/command/reference/irg_book/irg_bgp1.html

Appendix

COMMAND DESCRIPTION Options
maximum-paths ibgp number-of-paths To control the maximum number of parallel internal Border Gateway Protocol (iBGP) routes that can be installed in a routing table, use the maximum-paths ibgp command in router or address family configuration mode. To restore the default value, use the no form of this command. import: (Optional) Specifies the number of redundant paths that can be configured as backup multipaths for a virtual routing and forwarding (VRF) instance. This keyword can be configured only under a VRF in address family configuration mode.

 

unequal-cost: Specifies the number of unequal-cost routes to install in the routing table. This keyword can be configured only under a VRF instance in address family configuration mode.

maximum-paths eibgp number-of-paths To enable multipath load sharing among external Border Gateway Protocol (eBGP) and internal BGP (iBGP) routes, use the maximum-paths eibgp command in address family configuration mode. To disable multipath load sharing for eBGP and iBGP routes, use the no form of this command. import: (Optional) Specifies the number of redundant paths that can be configured as backup multipaths for a virtual routing and forwarding (VRF) instance. This keyword can be configured only under a VRF in address family configuration mode.
bgp additional-paths install To enable BGP to calculate a backup path for a given address family and to install it into the Routing Information Base (RIB) and Cisco Express Forwarding, use the bgp additional-paths install command in address family configuration or router configuration mode. To remove the backup paths, use the no form of this command.  
bgp additional-paths select To have the system calculate a second BGP bestpath, use the bgp additional-paths select command in address family configuration mode. To remove this mechanism for calculating a second bestpath, use the no form of the command. best-external: (Optional) Calculates a second bestpath from among those received from external neighbors. Configure this keyword on a PE or RR. This keyword enables the BGP Best External feature on an RR.

 

backup: (Optional) Calculates a second bestpath as a backup path.

bgp advertise-best-external To enable BGP to calculate an external route as the best backup path for a given address family and to install it into the Routing Information base (RIB) and Cisco Express Forwarding, and to advertise the best external path to its neighbors, use the bgp advertise-best-external command in address family or router configuration mode. To remove the external backup path, use the no form of this command.  
neighbor X.X.X.X advertise diverse-path backup To specify that an additional path (a backup path or multipath or both) is advertised to a peer in addition to the bestpath, use the neighbor advertise diverse-path command in address family configuration mode. To remove the designation, use the no form of the command. backup: (Optional) Advertises the backup path. If backup is specifed, but there is no backup path, the best path is advertised.

 

mpath: (Optional) Advertises the multipath, which is the second best path. If mpath is specifed, but there is no multipath, the best path is advertised. If both backup and mpath are specified, the multipath is advertised.

Leave a Reply

Your email address will not be published. Required fields are marked *