my cheat sheet on python unittests

Home

1 Python unittests

When developing software, the addition of new features or just fixing of some bugs in the code, can introduce more new bugs. So we should test the code on every change made to it. To speed up and automate this process is very important, hence Python's unittest and pytest modules were created. unittest is part of the standard built-in python modules I think, but pytest needs to be pip installed before it can be imported. Both unittest and pytest are known as a Testing Framework.

I first learned about this in the excellent Socratica youtube video.

Unit tests should be done on functions.

  • Copy the function out of your script, into its own separate .py program
  • Give it a name such as circles.py (to return area)
  • Create a separate but related unit test module with the name test_circles.py

Very often you will see the use of Classes in error and exception handling in unnittest. The example Write the unit test below does exactly that.

1.1 unittest module

For any unit tests, the first step is to import the unittest module. So in my example, start off test_grail-search.py with the line: import unittest

1.2 Write the unit test

The unit test and the function itself can be in the same directory. If so, the import statement in the example below works. Otherwise you have to specify the full path.

Filename: testcirclearea.py 

import unittest
from circles import circle_area
from math import pi

# you cold also just copy the circle_area method into the top
# of this file, right here.  i.e. def circle_area(radius):   etc..

class TestCircleArea(unittest.TestCase):
''' unit tests are always created using the TestCase, as here also'''
  def test_area(self):
    # Test areas when radius >=0
    self.assertAlmostEqual(circle_area(1), pi)
    self.assertAlmostEqual(circle_area(2.1), pi*2.1**2)
    self.assertAlmostEqual(circle_area(0 ), 0)

  def test_values(self):
    # Make sure value errors are raised
    self.assertRaises(ValueError, circle_area, -2)
    self.assertRaises(ValueError, circle_area, -2)

if __name__ == '__main__':

  unittest.main()

Then run this program: python test_circlearea.py

  • (remember chmod 740 testcirclearea.py)

If you wanted to run just one of the tests in test_circlearea.py you can just by using the dot notation.

  • python -m unittest test_circlearea.test.TestCircleArea.test_values
  • python -m unittest test_script.classname.methodname
  • pytyon -m unittest # will run all the tests in the current directory

failed-unittest.png

Figure 1: Failed unittest output

1.3 unittest Help

This is a useful way of seeing the multitudes of tests that are available in the unittest imported module. Use interactive pytyon, and type: >>> import unittest >>> dir(unittest) >>> help(asertRaises) >>> help(asert)

1.4 Run the Unit Test

To run the tests, go to the subdirectory of the testing module and unittest module

python -m unittest test_circles

The -m option tells python to run the unittest module as a script.

If you omit the name the test, i.e. python -m unittest then the unittest will search the directory and run ALL the unittests it finds. this is useful short cut when developing lots of modules.

2 PyTest like Unittest module

Simialr to unittest module that is built-in to python, however PyTest module must be installed from the standar pypi collection, so pip install pytest

Unittests enagles creation of test collections as methods extending a default TestCase class. Read that in this docs.python.org link.

PyTest can run unittest without any changes. PyTests however also let coders build tests as simple functions, rather than as class methods. Also PyTest is used by pyats suite from Cisco.

2.1 Auto execution of test_ or _test.py files

Within these scripts, pytest automatically executes any functions that start with "test_" or "tests_" To test your function myfunc Here is your workflow:

  1. copy myfunc into a file
  2. import pytest
  3. add testing functions (names begin with test_ or tests_)
  4. save the file as test_myfunc.py or myfunc_test.py
  5. run that with pytest module

For example if your function, myfunc is 

def myfunc(x):
  someother_x = x / 1.67
  return someother_x

The your new file should be named test_myfunc.py and conatain: 

import pytest

def myfunc(x):
  someother_x = x / 1.67
  return someother_x

def tests_myfunc():
  r = myfunc(1.67)
  assert r == 1
  r = myfunc(2)
  assert r == 2/1.67
  r = myfunc(16.7)
  assert r == 10

Then run the test using: pytest test_myfunc.py

Sample failed test output: 

FAIL: test_search_found (__main__.json_search_test)
key should be found, return list should not be empty
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_json_search.py", line 11, in test_search_found
    self.assertTrue([]!=json_search(key1,data))
AssertionError: False is not true

----------------------------------------------------------------------
Ran 3 tests in 0.001s

FAILED (failures=1)
developer:unittest > python improved_json_search.py
[{'issueSummary': 'Network Device 10.10.20.82 Is Unreachable From Controller'}]
developer:unittest > python -m unittest test_improved_json_search
[{'issueSummary': 'Network Device 10.10.20.82 Is Unreachable From Controller'}]
..F
======================================================================
FAIL: test_search_not_found (test_improved_json_search.json_search_test)
key should not be found, should return an empty list
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/developer/src/unittest/test_improved_json_search.py", line 15, in test_search_not_found
    self.assertTrue([]==json_search(key2,data))
AssertionError: False is not true

----------------------------------------------------------------------
Ran 3 tests in 0.001s

FAILED (failures=1)
developer:unittest >

2.2 Python pytest example 1.

Module called myscript.py has this function, called total 

def total(xs: List[float]) -> float:
    return -1.0

Then the corresponding test module is called myscript_test.py 

""" tests for myscript.py """
from myscript import total

def test_total_empty() -> None:
    assert total([]) == 0.0

Then run the test like so:

  • ~python -m pytest

2.3 Python pytest example2

2.3.1 Example test for a function called "join.py"

We want an input of =[1, 2, 3] with a delimiter of "-", to output "1-2-3" The function definition could be: 

def join(z_array: List[int], delimiter: str) -> str

2.3.2 Use cases:

  1. a list of strings join(["a", "b", "c"], "_") -> "abc"
  2. a list of integers join([1, 2, 3], ",") -> "1,2,3"
  3. a list of floats join([1.0, 2,71828, 3.14159], ", and ") -> "1.0, and 2.71728, and 3.14159"

2.3.3 Edge cases:

  1. an empty list join([], "-") -> Null
  2. a string (i.e. not a list) join("I a am not a list", "-") - > "I-am-not-a-list"
  3. a single number (i.e. not a list) join(5, ">") -> "5"
  4. an empty string as a delimiter join([1, 2, 3], "") -> "123"

2.3.4 Hacker cases:

  1. a string called "break"
  2. an EOF
  3. a null
  4. ?

First set up our skeleton: 

def join(z_array: List[int], delimiter: str) -> str
""" Produce a string from array so each element is separated by delimiter """
return "SKELETON"

This skeleton is obiously not correct yet, but it is a valid function, and returns a string, as it should.

Now start wriing tests. Naming conventions:

  • "test_<name of function>use_…"
  • "test_<name of function>edge_…"
  • "test_<name of function>hack_…" 
import ~/bin/join 
def test_join_use_case() -> None:
""" assert that join of integers returns a string with those integers."""
    assert join([1, 2, 3], ", ") ->  "1, 2, 3"

def test_join_edge_single_item() -> None:
""" assert that join of a single integers returns a string of that integer."""
    assert join([7], ", ") ->  "7"

def test_join_edge_empty_delimiter() -> None:
""" assert that join of a single integers returns a string of that integer."""
    assert join([1, 2, 3], "") ->  "123 "

You can run pytest in two different ways, from the terminal or from emacs IDE

2.3.5 Run pytest from terminal

Now run these tests, and recursively update the join function until it passes all the tests. (you start with the skeleton function) You run tests with the command python -m pytest join_test.py 

def join(z_array: List[int], delimiter: str) -> str:
   """ Produce a string from array so each element is separated by delimiter """
   generated_string = join([str(x) for x in z_array])
   return generated_string

3 Run pytest from the command line:

= python -m pytest join_test.py

  • pytest join_test.py should also work if all your venv settings are correct and you are in the correct directory (where you project is)

4 running pytest within emacs

You have to install pytest using pip, i.e. pip install pytest and then for emacs you add the package python-pytest Note, there is also a package from melpa called just pytest but I did not install that.

Finally you run the test using M-x python-pytest There are several other python-pytest modes such as python-pytest-mode, python-pytest-files, and python-pytest-repeat Probably a good idea to map a key to the -repeat option. Not yet done though.

Since you have projectile installed as well, you can also try using the command M-x projectile-test-project It will ask you what python test framework you wish to use. By default that is python -m unittest discover which is fine if you have set up unittests in your directory.

If, on the other hand, your directory has pytest tests defined, you can change that to simply be pytest (could also say python -m pytest)

Running pytest will run it from the root of the project folder, and I think it will "discover all tests in that directory", but not sure.

Running again, will default to what you set earlier. i.e. pytest.

You can also run it again using M-x recompile - which works using a quirk in projectile. so your milage may vary.

4.1 Home