## Python Testing Cookbook

Over 70 simple but incredibly effective recipes for taking control of automated testing using powerful Python testing tools

The reader can benefit from the previous article on Testing in Python using doctest.

# Documenting the basics

Python provides out-of-the-box capability to put comments in code known as docstrings. Docstrings can be read when looking at the source and also when inspecting the code interactively from a Python shell. In this recipe, we will demonstrate how these interactive docstrings can be used as runnable tests.

What does this provide? It offers easy-to-read code samples for the users. Not only are the code samples readable, they are also runnable, meaning we can ensure the documentation stays up to date.

## How to do it…

With the following steps, we will create an application combined with runnable docstring comments, and see how to execute these tests:

- Create a new file named
*recipe16.py*to contain all the code we write for this recipe. - Create a function that converts base-10 numbers to any other base using recursion.
def convert_to_basen(value, base):

import math

def _convert(remaining_value, base, exp):

def stringify(value):

if value > 9:

return chr(value + ord(‘a’)-10)

else:

return str(value)

if remaining_value >= 0 and exp >= 0:

factor = int(math.pow(base, exp))

if factor <= remaining_value:

multiple = remaining_value / factor

return stringify(multiple) +

_convert(remaining_value-multiple*factor,

base, exp-1)

else:

return “0” +

_convert(remaining_value, base, exp-1)

else:

return “”return “%s/%s” % (_convert(value, base,

int(math.log(value, base))), base) - Add a
*docstring*just below the external function, as shown in the highlighted section of the following code. This*docstring*declaration includes several examples of using the function.def convert_to_basen(value, base):

“””Convert a base10 number to basen.edur

>>> convert_to_basen(1, 2)

‘1/2′

>>> convert_to_basen(2, 2)

’10/2′

>>> convert_to_basen(3, 2)

’11/2’

>>> convert_to_basen(4, 2)

‘100/2’

>>> convert_to_basen(5, 2)

‘101/2’

>>> convert_to_basen(6, 2)

‘110/2’

>>> convert_to_basen(7, 2)

‘111/2’

>>> convert_to_basen(1, 16)

‘1/16’

>>> convert_to_basen(10, 16)

‘a/16’

>>> convert_to_basen(15, 16)

‘f/16′

>>> convert_to_basen(16, 16)

’10/16’

>>> convert_to_basen(31, 16)

‘1f/16′

>>> convert_to_basen(32, 16)

’20/16’

“””

import math - Add a test runner block that invokes Python’s
*doctest*module.if __name__ == “__main__”:

import doctest

doctest.testmod() - From an interactive Python shell, import the recipe and view its documentation.
- Run the code from the command line. In the next screenshot, notice how nothing is printed. This is what happens when all the tests pass.
- Run the code from the command line with
*-v*to increase verbosity. In the following screenshot, we see a piece of the output, showing what was run and what was expected. This can be useful when debugging*doctest*.

## How it works…

The doctest module looks for blocks of Python inside *docstrings* and runs it like real code. *>>>* is the same prompt we see when we use the interactive Python shell. The following line shows the expected output. *doctest* runs the statements it sees and then compares the actual with the expected output.

## There’s more…

*doctest* is very picky when matching expected output with actual results.

- An extraneous space or tab can cause things to break.
- Structures like dictionaries are tricky to test, because Python doesn’t guarantee the order of items. On each test run, the items could be stored in a different order. Simply printing out a dictionary is bound to break it.
- It is strongly advised not to include object references in expected outputs. These values also vary every time the test is run.

# Catching stack traces

It’s a common fallacy to write tests only for successful code paths. We also need to code against error conditions including the ones that generate stack traces. With this recipe, we will explore how stack traces are pattern-matched in doc testing that allows us to confirm expected errors.

## How to do it…

With the following steps, we will see how to use *doctest* to verify error conditions:

- Create a new file called
*recipe17.py*to write all our code for this recipe. - Create a function that converts base 10 numbers to any other base using recursion.
def convert_to_basen(value, base):

import math

def _convert(remaining_value, base, exp):

def stringify(value):

if value > 9:

return chr(value + ord(‘a’)-10)

else:

return str(value)

if remaining_value >= 0 and exp >= 0:

factor = int(math.pow(base, exp))

if factor <= remaining_value:

multiple = remaining_value / factor

return stringify(multiple) +

_convert(remaining_value-multiple*factor,

base, exp-1)

else:

return “0” +

_convert(remaining_value, base, exp-1)

else:

return “”

return “%s/%s” % (_convert(value, base,

int(math.log(value, base))), base) - Add a
*docstring*just below the external function declaration that includes two examples that are expected to generate stack traces.def convert_to_basen(value, base):

“””Convert a base10 number to basen.

>>> convert_to_basen(0, 2)

Traceback (most recent call last):

…

ValueError: math domain error

>>> convert_to_basen(-1, 2)

Traceback (most recent call last):

…

ValueError: math domain error

“””

import math - Add a test runner block that invokes Python’s
*doctest*module.if __name__ == “__main__”:

import doctest

doctest.testmod() - Run the code from the command line. In the following screenshot, notice how nothing is printed. This is what happens when all the tests pass.
- Run the code from the command line with
*-v*to increase verbosity. In the next screenshot, we can see that*0*and*-1*generate math domain errors. This is due to using*math.log*to find the starting exponent.

## How it works…

The *doctest* module looks for blocks of Python inside *docstrings* and runs it like real code. *>>>*; is the same prompt we see when we use the interactive Python shell. The following line shows the expected output. *doctest* runs the statements it sees and then compares the actual output with the expected output.

With regard to stack traces, there is a lot of detailed information provided in the stack trace. Pattern matching the entire trace is ineffective. By using the ellipsis, we are able to skip the intermediate parts of the stack trace and just match on the distinguishing part: *ValueError: math domain error*.

This is valuable, because our users can see not only the way it handles good values, but will also observe what errors to expect when bad values are provided.