I ran into a problem while doing an INE mock lab this morning…. it basically kicked my ass, so I figured I’d post about it!
The overall scenario is that there are two BGP domains, AS 1000 and AS 2000. Within each AS, there is some standard IGP routing, and IPv4 BGP — including eBGP between the two domains. There is also some route reflection and some other fun stuff, but that’s mostly irrelevant for the purposes of this post. Below is the INE topology drawing.
After the tasks that setup the basics, the lab rolls into some inter-AS VPN. Essentially, the routers in AS 1000 and 2000 also have loopbacks that are in the same VRF that AS 3000 lives in. The initial VPNv4 task basically is asking to configure the domains so that loopbacks in this VRF are reachable from both of the domains.
So the first thing to consider is that BGP labels will need to be sent between the domains. Thats pretty simple, just send-label in IOS or labeled-unicast in IOS-XR. In addition to that, IOS will require the “mpls bgp forwarding” command on the interfaces between the domains in order to send the labels. For IOS-XR, since the neighbor is on a physical interface, and it’s not a /32 (obviously), a static host route to the neighbor pointing out the connected interface is required. This is because IOS-XR will not install any labels into the forwarding table that have a next hop of something other than a /32.
After this, we need to ensure that each domain has reachability to the ‘PE’ routers loopbacks. This is to ensure that we have a label switched path the whole way through to the end. We also need to make sure that however we learn about the ‘PE’ (which is basically every router since they all have a loopback in the ‘customer’ VRF) loopbacks, and that we get some labels for those. There is an important piece here that basically says that however we learn about that prefix (/32 for the PE), we must also get a label from the same mechanism. IF we were to learn about those /32s via BGP, we would need a BGP label. If we learn the PE loopbacks via IGP, we need to have a label for that via IGP/LDP.
This leads us to the point of the post! In the course of the lab, I was advertising the loopback of each of the PE devices into BGP on each router individually — i.e. on R5 I advertised 10.0.0.5/32 (loopback0) into BGP locally, and advertised R2s loopback locally, etc. This totally worked — R5 and XR1 both had these prefixes in BGP and while things were configured for normal IPv4 unicast (not labeled) they were advertised across to AS 2000.
Things got a little dicey for me though when I moved the eBGP to labeled unicast. R5 was sending prefixes and labels across to AS 2000, but when shutting down that peering session to test that the inter-AS VPNv4 setup was working across XR1/R3 as well, I was met with crushing defeat!!
Thankfully somebody on the IEOC forums (INEs forum) had this same problem, and Mr. Brian McGahan was there to save the day… here’s what he said:
Only the originator of the BGP route can allocate the label. This means that whoever you have the network statement or the redistribute statement on you need to do the allocation there. In your case if you don’t originate the network on XR1 you’d have to go to R5 and then send-label to XR1, and on XR1 send label back to R5. That’s why in most designs you just have your edge routers originate the BGP networks on behalf of the IGP network, because then you have a single point of control for them. You can do it either way but it’s good to know that the problem exists in the first place.
So basically IOS-XR, which was configured for ‘allocate-label all’ in order to send BGP labels across to AS 2000, was NOT actually sending any labels!! This was due to the way I was getting the loopback prefixes piped into BGP. Killing the advertisements on the other routers, and then advertising them into BGP on XR1 instead allowed XR1 to send the labels across.
So lesson learned! I’m pretty glad that I ‘messed’ up and was able to come across this because I could totally see Cisco doing something like this on the lab — guess I’ll find out in a few weeks when I sit my first attempt!! 😀