ACI Fabric Access Policy Best Practices

I’ve been meaning to write this post for quite a while now as I think this is one of the most important pieces to a solid ACI deployment — having well-defined fabric access policies. If you screw this up early in your ACI deployment you can always fix it but it can be a big pain in the ass. Over the course of the last two years(ish) and a bunch of ACI deployments this is the best strategy I’ve seen; big thanks to my friends @highspeedsnow and @therealaciguy for coming up with the guts of this “design” (or whatever you wanna call it).

Before we dive in, lets review some basics by going over the relevant bits we will discussing:

Interface Policy Groups:

  • An Interface Policy Group is basically a folder that contains L1/L2 traits that you would like to apply to a port or group of ports.
  • Interface Policy Groups come in three main flavors: access, port-channel, and virtual port-channel (vPC).
  • Access Policy Groups can be reused as many times as you want (this will hopefully be more clear as we keep going if it’s not already), whereas port-channel and vPC Policy Groups cannot be reused. PC/vPC Policy Groups aren’t reusable as they are kind of the “interface port-channel” config from traditional NX-OS — what I mean by that is interfaces get associated the Policy Group, and then become part of the port-channel — obviously if we want to have unique port-channels we use unique port-channel IDs in a traditional switch, its much the same in ACI.
  • By themselves, Interface Policy Groups do pretty much nothing! We’ll need to apply the Interface Policy Group to an Interface Selector, don’t worry it all seems kinda crazy and like lots of work at first, but its all quite organized and logical.

Interface Profiles:

  • An Interface Profile by itself kinda doesn’t really do anything either. It’s really a container for Interface Selectors.
  • Interface Profiles ultimately get tied to Switch Profiles — I swear this will all start to make sense shortly 🙂

Interface Selectors:

  • Interface Selectors tie an Interface Policy Group to a port or ports.
  • An Interface Selector is a child object of an Interface Profile.
  • Note that the Interface Selector “selects” a port (via port ID), but not a switch — i.e. Interface Policy Group “Carls-Access-Port” gets associated to port 1/1, but that doesn’t tell us which Switch…

Switch Profiles:

  • Switch Profiles tie all the stuff we’ve talked about together and actually associates things to a specific switch or switches.

Okay so now that we’ve gotten that out-of-the-way, lets jump right into how I like to design this and why. We’ll go in the same order outlined above.

Interface Policy Groups:
Screen Shot 2016-07-24 at 3.26.29 PM

  • As outlined above, only access-port Interface Policy Groups are reusable, so I always create a “Standard-Acess” Policy Group for my access-ports.
  • You’re probably used to having port-channels/vPCs be identified by a number — don’t do that! The name field for the Interface Policy Group accepts numbers so you could easily just name your PC/vPC “1”, “2”, “3”, etc., but its a PITA later as ACI will automatically allocate a PC ID and it will almost certainly not be what you entered!
  • Do yourself a favor and name the PC/vPC policy groups logically. In the above picture I have “F5-PR-A” and “F5-PR-B” — these represent F5 Prod A and B (nice and easy right?). Often times we end up naming the Interface Policy Groups after the hostname of the server they are connecting to.
  • You’ll need Interface Policy Groups for your L3 Outs as well, so I always just name those “L3-Out-XXXX” again you can see I have some L3 Outs to ASAs and a “Subifs” Policy Group as well for the primary L3 Out.
  • Note that in 2.0+ you will see a folder for both “Leaf Policy Groups” and “Spine Policy Groups” – if you are on code prior to 2.0 you will simply have the parent “Policy Groups” folder.

Interface Profiles:
Screen Shot 2016-07-24 at 3.51.05 PM

  • So far nothing too exciting — really just some common sense naming stuff, but this is where it starts to get a little more interesting.
  • My first few times through I was building out Interface Profiles that were specific to a function — my logic was if I have an Interface Profile for lets say my F5 Load Balancers, and I eventually expanded so much I had to add more F5s I could simply deploy that Interface Profile again to the next set of Leaf switches that would connect the new additional F5s. Good logic I guess, but I don’t advocate it!
  • My current strategy is to create an Interface Profile per Leaf switch and per PAIR of Leaf switches (vPC pairs) — this should be pretty obvious in the pic above.

Interface Selectors:
Screen Shot 2016-07-24 at 3.51.21 PM

  • This is perhaps the most important piece to my overall strategy — I always create an Interface Selector PER PORT. So even though we could do a range of ports I always always always do a single port at a time.
  • At first glance that doesn’t make a ton of sense right? Why create more work (building individual Interface Selectors) when we can simply have a range of lets say 10 ports that are all access-ports that all have the same Interface Policy Group attached to them? The answer is simple and pretty obvious (maybe not unless you’ve been burned by this before which is why I’m writing this!) — if I want to make a change to port 1/3, but I’ve previously created an Interface Selector that spans from port 1 to port 10, I would have to delete that Interface Selector (causing some faults and things to happen), create a new Interface Selector for port 1/3, and then another two Interface Selectors for ports 1/1-1/2 and 1/4-1/10. Not cool!
  • Another less drastic example would be if I had a port-channel that was 8 total ports and I created an Interface Selector for ports 1/1-1/4 in my Interface Profile L101-L102 — what happens if I decide that an 80G vPC to my QA A10 is overkill and I want to reclaim some of those ports? You guessed it, gotta delete, rebuild the previous Interface Selectors for the port-channel, then a new one for whatever else I want to do.
  • So the moral of the story here is to be granular as it will save you a lot of hassle in the future.
  • Finally, I always just name these nice and simply “Port-1”, “Port-40”, etc. — keeps it easy to read, and my thinking is that the EPG and/or Interface Policy Group should be the real descriptor (or hey, just use the description field that is available pretty much everywhere in ACI!).

Switch Profiles:
Screen Shot 2016-07-24 at 3.51.35 PM

  • Okay bringing it all together!! So as you can see the Interface Profiles I created mapped to individual switches and vPC pairs of switches (in name only, because they aren’t actually “hooked” to a switch yet), and as you can see I’ve created Switch Profiles that have exactly the same naming convention.
  • And of course, as the name implies, this is where we actually tie all of the interface level stuff to an actual switch. Logically, each Switch Profile is associated to the switches in the name — i.e. Switch Profile “L101-L102” is associated to the vPC switch pair of nodes 101 and 102.

In closing, some of this may seem like extra work, and I suppose it is. That being said blowing all this config in  via the API is SUPER simple (check out the ACI Power Deployment Tool, a Python library I wrote to help folks get started w/ ACI/ReST). Hopefully this all makes sense and you can see the value in being nice and granular with your naming/deployment strategy – it will help you out a ton later down the road!

Advertisements

13 thoughts on “ACI Fabric Access Policy Best Practices

  1. Out of all the new features, I still think my favorite new v2.0 enhancement is EPG Deployment Through AEP. Being able to define the static path bindings within that association for bare metal hosts makes me very happy Such a simple concept, but makes life easier (and more intuitive).

  2. Great article but wanted to clarify one of the comments in the article. You say that PortChannel and vPC Policy Groups cannot be re-used. In the case of PortChannel type they cannot be re-used on the same switch and in the case of vPC it is per-vPC pair that they cannot be re-used is that accurate?

    • Great question! Had to do some checking to get an answer! My buddy @MattLockCCIE got us the answer though — you can *configure* it, but it throws faults. He tested w/ the same vPC applied to two different sets of interface policies, then tried to deploy it in an EPG as a static path binding and got “nwissues” faults. I kinda thought it *might* work since it would be deployed against two different vPC domains, but guess not. Thanks for the great question, now we know!

      Carl

  3. Excellent article, nice work! It is important to get this right as you say. Thinking about the issue you describe in the ‘interface profile’ section around re-purposing ports that are already part of a range – one suggestion might be to look at ‘interface override’ policies to help with that sort of situation.

    • Thanks Adam! I’ll mess around with the override, I’ve not done anything w/ them so something new to check out!

  4. Any recommendation / best practice for creating VLAN Pools. We are having issues with Overlapping VLANs. For example, we have same VLAN 202 in couple of VLAN pools, which are consumed by 2 ‘Phy Domains’. we are using both domains in same EPG with encap_VLAN: 202, static binding to 2 different leafs. Because of this we have different VXLAN values on each leaf for same EPG for same Encap_VLAN.

    • Ah thats a tough one. Generally I only ever want to/prefer to have a single AEP — which means single set of VLANs obviously. I guess the first question is what is the need for multiple vl202s? You could use PPV to get around having multiple vl202s, but have them mean different things on each switchport.
      carl

  5. And also to let you know, we are planning to consolidate all VLAN-Pools into one VLAN-Pool to resolve on-going issues.
    we learnt the lesson in hard way, recommendation is
    Scenario 1:
    1 VLAN-Pool for Physical domain
    1 VLAN-Pool for VMM domain
    1 VLAN-Pool for L3 out
    = 3 VLAN-Pools in total (Phy, VMM & L3)
    Scenario 2:
    If same VLANs are used in VMM as well as in Physical then consolidate VMM and Phy domains into 1. = 2 VLAN pools in total (Phys+VMM & L3)
    Riki

    • PPV = Per Port VLAN — basically a VLAN can be locally or globally scoped. A locally scoped VLAN is significant only to the single port you’ve configured it on, so you can have 48 different VLAN 10s on a single 48 port switch (for example).

      I 100% agree w/ how you’ve outlined your VLAN pools — sorry you had to learn it the hard way!!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s