Python (relative) Imports and Unittests – Derp!

Been far too long since I wrote something. Spent way longer than I feel good about talking about derping around getting my ass kicked by some obvious shit so figured I should write it up as a reminder for me and to hopefully save some anonymous Internet person some heartache! Anyway, I’ve been meaning to put on my big boy pants and start actually writing test cases for all the Python code I write… I’ve been terrible at this and figured it’s time to get my life together with it. Rather than have a bunch of code that won’t really matter, I’ll just use ridiculously dumb examples here to get the issue and the “fix” (not being dumb!) across.

A pretty standard structure for a python project would look something like this:

mycoolproject/
├── mycoolproject
│   ├── __init__.py
│   ├── my_module_1.py
│   └── my_module_2.py
└── tests
 ├── __init__.py
 └── test_basics.py

You can see that we have our “project” — in this case named “mycoolproject” and within that directory we have a folder for our tests and a folder for the actual project code. In the root directory for our project we would probably have other stuff like a setup.py or a requirements.txt or whatever, but those things don’t matter for us at the moment. One last bit — we need those “__init__.py” files in there to make this a “project” — from the real Python docs:

“The __init__.py files are required to make Python treat the directories as containing packages; this is done to prevent directories with a common name, such as string, from unintentionally hiding valid modules that occur later on the module search path. In the simplest case, __init__.py can just be an empty file, but it can also execute initialization code for the package or set the __all__ variable, described later.”

You can find that doc here if you want to read some more about it.

Ok so now that we have the overall gist of things down, let’s take a look at our ridiculously over simplified “module”. Here is the contents of our file “my_module_1.py”:

my_string = 'whoa, this is so kewl'
print(my_string)

Pretty serious code 🙂

Our other script “my_module_2.py” is also pretty simple, but this one refers back to the first module to use the variable “my_string” — we’ll get to why this tripped me up in a bit:

import my_module_1
my_new_string = f'carl said: {my_module_1.my_string}'
print(f'carl said: {my_module_1.my_string}')

Alrighty, so if we run “my_module_1.py” it will simply print “whoa, this is so kewl” — no surprise there. If we run “my_module_2.py” it will *also* print “whoa, this is so kewl” because it is importing/loading the first script, and then of course it will also print “carl said: whoa, this is so kewl” as expected. Great, so our super fancy project works as desired. Now, because we are trying to be better about testing, let’s write a super simple test to validate this code works as expected.

In our test folder, we’ll create a new script called “test_basics.py” that looks like this:

import unittest
import my_module_1


class TestMe(unittest.TestCase):
   def test_stuff(self):
       assert my_module_1.my_string == 'whoa, this is so kewl'


if __name__ == '__main__':
    unittest.main()

At the top we’ll import the unittest library to use for our testing, and we’ll also import our script “my_module_1” so that we can validate (assert) that our variable “my_string” is actual equal to what we think it should be (“whoa, this is so kewl”).

So let’s go ahead and run our test suite and see what happens:

Carls-MacBook-Pro-2:mycoolproject Carl$ python3 -m unittest tests/test_basics.py
E
======================================================================
ERROR: test_basics (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_basics
Traceback (most recent call last):
 File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/unittest/loader.py", line 153, in loadTestsFromName
 module = __import__(module_name)
 File "/Users/Carl/Desktop/mycoolproject/tests/test_basics.py", line 2, in <module>
 import my_module_1
ModuleNotFoundError: No module named 'my_module_1'


----------------------------------------------------------------------
Ran 1 test in 0.000s

 

Well… not ideal eh? Obviously we have some kind of import error since Python is complaining it can’t find our module “my_module_1”. What gives? Well, we know that Python is mad at the line where we import “my_module_1” so obviously we need to start there. We also know that our modules actually run fine within their directory — they do exactly what we think they should. So with this information we can understand that Python — when ran from the tests directory has no idea how and where to find the module we are trying to run. This makes sense because when you think about it Python will search for modules in the local folder and the system path(s) — we can see where Python is looking by importing sys and printing out the path, let’s see what that looks like in our tests folder:

Carls-MacBook-Pro-2:tests Carl$ python3
 Python 3.6.4 (v3.6.4:d48ecebad5, Dec 18 2017, 21:07:28)
 [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
 Type "help", "copyright", "credits" or "license" for more information.
 >>> import sys
 >>> sys.path
 ['', '/Library/Frameworks/Python.framework/Versions/3.6/lib/python36.zip', '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6', '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/lib-dynload', '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages']
 >>>

Ok, so we know it’s looking in the normal paths, and that very first entry (”) shows us its going to look for stuff locally too, but obviously nowhere to be seen is the module we’ve been building. So somehow we need to tell Python to look there, what happens if we ask Python to import “my_module_1” *from* “mycoolproject” like so:

from mycoolproject import my_module_1

Let’s run it and see what happens:

Carls-MacBook-Pro-2:mycoolproject Carl$ python3 -m unittest tests/test_basics.py
whoa, this is so kewl
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK

Hey that seems a lot better huh? Up to this point everything has been super straight forward and if you’ve done any amount of Python stuff you’ll be more than familiar with import errors as you’ve undoubtedly forgotten to import something and had this happen to you. The next bit is where I got tripped up… let’s add a quick test case to test our other Python file:

import unittest
from mycoolproject import my_module_1
from mycoolproject import my_module_2


class TestMe(unittest.TestCase):
   def test_stuff(self):
       assert my_module_1.my_string == 'whoa, this is so kewl'

  def test_other_stuff(self):
      assert my_module_2.my_new_string == 'carl said: whoa, this is so kewl'


if __name__ == '__main__':
    unittest.main()

Pretty straight forward stuff here too — we simply imported the other module and added a test case to assert that the string is what we think it should be. So what happens if we run our unit tests again?

Carls-MacBook-Pro-2:mycoolproject Carl$ python3 -m unittest tests/test_basics.py
whoa, this is so kewl
E
======================================================================
ERROR: test_basics (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_basics
Traceback (most recent call last):
 File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/unittest/loader.py", line 153, in loadTestsFromName
 module = __import__(module_name)
 File "/Users/Carl/Desktop/mycoolproject/tests/test_basics.py", line 3, in <module>
 from mycoolproject import my_module_2
 File "/Users/Carl/Desktop/mycoolproject/mycoolproject/my_module_2.py", line 1, in <module>
 import my_module_1
ModuleNotFoundError: No module named 'my_module_1'


----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (errors=1)

Annnnnnd we’re back to not working. So what gives? We get kinda the same error as before — complaining about import errors and whatnot. Not cool, especially since we basically did the same thing we did for “my_module_2” as we already did (and got working) for “my_module_1”. This time we can see that Python is upset about not finding a module called “my_module_1” — pretty ridiculous given the fact that importing that module is the second line of our test file right? BUT this is not failing *in* our test file — and here is what tripped me up. The issue is (and I’ll probably butcher the exact technical reasoning for this but you can check out this super handy SO post) that Python is confused about where that module is because the path for the execution (via unittest) is not *in* the same folder as the modules themselves. So we can address this by ensuring that the imports in our modules are not relative, but instead fully qualified if you will. Changing our “my_module_2” file to look like this:

from mycoolproject import my_module_1

my_new_string = f'carl said: {my_module_1.my_string}'
print(my_new_string)

Instead of importing from our local file we are now specifying the project that we are importing from, running our test again we get the following (good) results:

Carls-MacBook-Pro-2:mycoolproject Carl$ python3 -m unittest tests/test_basics.py
whoa, this is so kewl
carl said: whoa, this is so kewl
..
----------------------------------------------------------------------
Ran 2 tests in 0.000s

OK

TL;DR — pay attention to your imports, easy thing to fix but easy to miss it, run it locally and have everything run great and then be dumb like me and get angry at tests for not behaving the way you think they should 🙂

Advertisements

Guest Post! WTF Are all those Checkboxes? (ACI L3 Outs) – Part 2 of ???

My friend and colleague Mr. Jason Banker recently ran into some good times with the mysteries of the ACI L3 Out Checkbox Madness! He Slack’d me and told me he’d found some clowns blog post about it (yours truly) and that some updates and additional information was needed, so he kindly volunteered some time to help out! Without further ado here is Jason’s Checkbox Madness:


 

As we continue to deploy fabrics we always joke about these damn routing checkboxes shooting us in the foot.  We play with different scenarios in the lab to ensure we understand how these pesky boxes work and what other options we have for future deployments.   The scenario here was to use get different OSPF areas connected to the same border leaf using ACI as the transit.  This scenario brings up some certain challenges and hopefully my testing will help others understand it a little better as well.

Design:

We have two external routers coming into a border leaf on ACI, two L3Outs (required because of multiple areas), one is Area 0 (backbone) and one is Area 1.  Here is the breakdown of routes on each router:

External Router 1 (Area 0):

  • Loopback0: 2.2.2.2/32
  • Loopback1: 4.4.4.4/32
  • Transits: 192.168.0.0/29

External Router 2 (Area 1):

  • Loopback0: 3.3.3.3/32
  • Loopback1: 5.5.5.5/32
  • Transits: 172.16.0.0/29

 

Using ACI as a transit we want routes from Area 0 to be imported into Area 1 and vice versa across the two L3Outs.   We will skip the build of the L3Out portion but I want to focus on those pesky checkboxes again.  Whenever I build an L3Out my network EPG usually looks something like this:

By default, “External Subnets for the External EPG” is checked (this checkbox simply enforces policy on this L3out and contracts are applied to the specific subnet) and I am using 0.0.0.0/0 network as a catch-all.  Moving along with the defaults, I show full adjacency:

 

As well as a full routing table within ACI, receiving the networks above in the fabric as expected:

Note:  Anything received from area 0 is shown as backbone and everything from area 1 is 0.0.0.1.

 

Now if look at the routing table for External Router 1 we see no routes across the fabric being received from External Router 2.

 

Let me check my OSPF Neighbors:

 

So, we have no OSPF routes but we have a neighbor relationship.  Let’s go check External Router 2:

 

So, we are showing some OSPF routes but they are only the loopbacks of the ACI Fabric (Area 0), not what we are necessarily looking for.  ACI blocks transit routes between different L3Outs unless permitted by policy via an OSPF area filter-list (to verify ssh to the border leaf and run “show route-map”).  Let’s go look at the Network EPG checkboxes again and see if we can get routing to occur between OSPF Areas across the fabric.

 

As we showed earlier we are using a catch-all 0.0.0.0/0 with “External Subnets for the External EPG”. If we select the “!” on the upper right portion of the screen we will receive the properties of this screen:

 

Based on what this screen states, “Export Route Control Subnet” – controls the export route direction and “Import Route Control Subnet” – controls the import direction.  This sounds like what we need to get routing to traverse the fabric.  Let’s go ahead and select them for Area 0 but before we can select import there is another configuration we need to apply before we can get import to not be grayed out.  If we go back to the top of the L3Out in the navigation pane we need to select the “Route Control Enforcement:” import checkbox:

 

Now if we go back to the Network EPG we should have both options available to us:

 

Now let’s see if we have any changes within our routing table:

 

We still have no change in the table.  Remember that we are using a catch all 0.0.0.0/0? This would require us to also select the aggregate export and import features on the subnet/network epg we have created for Area 0 and Area 1:

 

Time to verify:

 

This looks great.  Now we will verify External Router 2:

 

If we want to take it a step further we can do network specific routes instead of a catch-all:

 

Router 2:

 

Verify Router 2 is receiving 2.2.2.2/32 from Router 1:

 

Now we can send 3.3.3.3/32 from Router 2 into Router 1:

 

Router 1:

 

As I stated earlier these checkboxes are updating route-maps and prefix-lists within ACI.  Prior to us selecting the import/export feature our route-maps had a deny all so no routes would traverse areas.  Upon selecting these checkboxes we can see the change:

 

Instead of giving you route-map blah I will try and breakdown the map for you just focusing on export route-map. Prior to import/export our route-map looked like this:

route-map exp-ctx-2392064-deny-external-tag, deny, sequence 1

  Match clauses:

    tag: 4294967295

  Set clauses:

route-map exp-ctx-2392064-deny-external-tag, deny, sequence 9998

  Match clauses:

    ospf-area: backbone

  Set clauses:

route-map exp-ctx-2392064-deny-external-tag, deny, sequence 9999

  Match clauses:

    ospf-area: 0.0.0.1

  Set clauses:

route-map exp-ctx-2392064-deny-external-tag, permit, sequence 10000

  Match clauses:

  Set clauses:

route-map exp-ctx-2392064-set-external-tag, permit, sequence 2

  Match clauses:

  Set clauses:

    tag 4294967295

route-map imp-ctx-bgp-st-interleak-2392064, deny, sequence 1

  Match clauses:

    tag: 4294967295

  Set clauses:

route-map imp-ctx-bgp-st-interleak-2392064, permit, sequence 10000

  Match clauses:

  Set clauses:

 

You can see that we had “deny” for backbone and area 0.0.0.1 preventing us from using the fabric as a transit.  After we selected import/export features our route-map is updated as such (just focusing on the export route-map):

route-map exp-ctx-2392064-deny-external-tag, deny, sequence 1

  Match clauses:

    tag: 4294967295

  Set clauses:

route-map exp-ctx-2392064-deny-external-tag, permit, sequence 9801

  Match clauses:

    ip address prefix-lists: IPv4-ospf-rt2392064--0-dst-rtpfx

    ipv6 address prefix-lists: IPv6-deny-all

    ospf-area: backbone

  Set clauses:

route-map exp-ctx-2392064-deny-external-tag, permit, sequence 9802

  Match clauses:

    ip address prefix-lists: IPv4-ospf-rt2392064--1-dst-rtpfx

    ipv6 address prefix-lists: IPv6-deny-all

    ospf-area: 0.0.0.1

  Set clauses:

route-map exp-ctx-2392064-deny-external-tag, deny, sequence 9998

  Match clauses:

    ospf-area: backbone

  Set clauses:

route-map exp-ctx-2392064-deny-external-tag, deny, sequence 9999

  Match clauses:

    ospf-area: 0.0.0.1

  Set clauses:

route-map exp-ctx-2392064-deny-external-tag, permit, sequence 10000

  Match clauses:

  Set clauses:

route-map exp-ctx-2392064-set-external-tag, permit, sequence 2

  Match clauses:

  Set clauses:

    tag 4294967295

route-map imp-ctx-bgp-st-interleak-2392064, deny, sequence 1

  Match clauses:

    tag: 4294967295

  Set clauses:

route-map imp-ctx-bgp-st-interleak-2392064, permit, sequence 10000

  Match clauses:

  Set clauses:

Now our route-map has been updated with prefix-lists to allow our traffic across areas, we will look at the prefix-list itself:

 

Leaf-103# show ip prefix-list IPv4-ospf-rt2392064--0-dst-rtpfx

ip prefix-list IPv4-ospf-rt2392064--0-dst-rtpfx: 1 entries

   seq 1 permit 0.0.0.0/0 le 32

 

The 0.0.0.0/0 catch-all has been added and our routes can traverse the fabric.  I suggest you also peak at the import route-maps and see what is happening under the hood there as well.