Python Testing Cookbook
Over 70 simple but incredibly effective recipes for taking control of automated testing using powerful Python testing tools
The reader can benefit from the previous article on Testing in Python using doctest.
Documenting the basics
Python provides out-of-the-box capability to put comments in code known as docstrings. Docstrings can be read when looking at the source and also when inspecting the code interactively from a Python shell. In this recipe, we will demonstrate how these interactive docstrings can be used as runnable tests.
What does this provide? It offers easy-to-read code samples for the users. Not only are the code samples readable, they are also runnable, meaning we can ensure the documentation stays up to date.
How to do it…
With the following steps, we will create an application combined with runnable docstring comments, and see how to execute these tests:
- Create a new file named recipe16.py to contain all the code we write for this recipe.
- Create a function that converts base-10 numbers to any other base using recursion.
def convert_to_basen(value, base):
import math
def _convert(remaining_value, base, exp):
def stringify(value):
if value > 9:
return chr(value + ord(‘a’)-10)
else:
return str(value)
if remaining_value >= 0 and exp >= 0:
factor = int(math.pow(base, exp))
if factor <= remaining_value:
multiple = remaining_value / factor
return stringify(multiple) +
_convert(remaining_value-multiple*factor,
base, exp-1)
else:
return “0” +
_convert(remaining_value, base, exp-1)
else:
return “”return “%s/%s” % (_convert(value, base,
int(math.log(value, base))), base) - Add a docstring just below the external function, as shown in the highlighted section of the following code. This docstring declaration includes several examples of using the function.
def convert_to_basen(value, base):
“””Convert a base10 number to basen.edur
>>> convert_to_basen(1, 2)
‘1/2′
>>> convert_to_basen(2, 2)
’10/2′
>>> convert_to_basen(3, 2)
’11/2’
>>> convert_to_basen(4, 2)
‘100/2’
>>> convert_to_basen(5, 2)
‘101/2’
>>> convert_to_basen(6, 2)
‘110/2’
>>> convert_to_basen(7, 2)
‘111/2’
>>> convert_to_basen(1, 16)
‘1/16’
>>> convert_to_basen(10, 16)
‘a/16’
>>> convert_to_basen(15, 16)
‘f/16′
>>> convert_to_basen(16, 16)
’10/16’
>>> convert_to_basen(31, 16)
‘1f/16′
>>> convert_to_basen(32, 16)
’20/16’
“””
import math - Add a test runner block that invokes Python’s doctest module.
if __name__ == “__main__”:
import doctest
doctest.testmod() - From an interactive Python shell, import the recipe and view its documentation.
- Run the code from the command line. In the next screenshot, notice how nothing is printed. This is what happens when all the tests pass.
- Run the code from the command line with -v to increase verbosity. In the following screenshot, we see a piece of the output, showing what was run and what was expected. This can be useful when debugging doctest.
How it works…
The doctest module looks for blocks of Python inside docstrings and runs it like real code. >>> is the same prompt we see when we use the interactive Python shell. The following line shows the expected output. doctest runs the statements it sees and then compares the actual with the expected output.
There’s more…
doctest is very picky when matching expected output with actual results.
- An extraneous space or tab can cause things to break.
- Structures like dictionaries are tricky to test, because Python doesn’t guarantee the order of items. On each test run, the items could be stored in a different order. Simply printing out a dictionary is bound to break it.
- It is strongly advised not to include object references in expected outputs. These values also vary every time the test is run.
Catching stack traces
It’s a common fallacy to write tests only for successful code paths. We also need to code against error conditions including the ones that generate stack traces. With this recipe, we will explore how stack traces are pattern-matched in doc testing that allows us to confirm expected errors.
How to do it…
With the following steps, we will see how to use doctest to verify error conditions:
- Create a new file called recipe17.py to write all our code for this recipe.
- Create a function that converts base 10 numbers to any other base using recursion.
def convert_to_basen(value, base):
import math
def _convert(remaining_value, base, exp):
def stringify(value):
if value > 9:
return chr(value + ord(‘a’)-10)
else:
return str(value)
if remaining_value >= 0 and exp >= 0:
factor = int(math.pow(base, exp))
if factor <= remaining_value:
multiple = remaining_value / factor
return stringify(multiple) +
_convert(remaining_value-multiple*factor,
base, exp-1)
else:
return “0” +
_convert(remaining_value, base, exp-1)
else:
return “”
return “%s/%s” % (_convert(value, base,
int(math.log(value, base))), base) - Add a docstring just below the external function declaration that includes two examples that are expected to generate stack traces.
def convert_to_basen(value, base):
“””Convert a base10 number to basen.
>>> convert_to_basen(0, 2)
Traceback (most recent call last):
…
ValueError: math domain error
>>> convert_to_basen(-1, 2)
Traceback (most recent call last):
…
ValueError: math domain error
“””
import math - Add a test runner block that invokes Python’s doctest module.
if __name__ == “__main__”:
import doctest
doctest.testmod() - Run the code from the command line. In the following screenshot, notice how nothing is printed. This is what happens when all the tests pass.
- Run the code from the command line with -v to increase verbosity. In the next screenshot, we can see that 0 and -1 generate math domain errors. This is due to using math.log to find the starting exponent.
How it works…
The doctest module looks for blocks of Python inside docstrings and runs it like real code. >>>; is the same prompt we see when we use the interactive Python shell. The following line shows the expected output. doctest runs the statements it sees and then compares the actual output with the expected output.
With regard to stack traces, there is a lot of detailed information provided in the stack trace. Pattern matching the entire trace is ineffective. By using the ellipsis, we are able to skip the intermediate parts of the stack trace and just match on the distinguishing part: ValueError: math domain error.
This is valuable, because our users can see not only the way it handles good values, but will also observe what errors to expect when bad values are provided.
