The core idea of a decorator is to transform some original function into another form. A decorator creates a kind of composite function based on the decorator and the original function being decorated. In this tutorial, we’ll understand how decorators can be used as higher-order functions in Python.
This article is an extract from the 2nd edition of the bestseller, Functional Python Programming, authored by Steven Lott.
Working with Decorator function
A decorator function can be used in one of the two following ways:
- As a prefix that creates a new function with the same name as the base function as follows:
@decorator def original_function(): pass
- As an explicit operation that returns a new function, possibly with a new name:
def original_function(): pass original_function = decorator(original_function)
These are two different syntaxes for the same operation. The prefix notation has the advantages of being tidy and succinct. The prefix location is more visible to some readers. The suffix notation is explicit and slightly more flexible.
While the prefix notation is common, there is one reason for using the suffix notation: we may not want the resulting function to replace the original function. We mayt want to execute the following command that allows us to use both the decorated and the undecorated functions:
new_function = decorator(original_function)
This will build a new function, named new_function(), from the original function. Python functions are first-class objects. When using the @decorator syntax, the original function is no longer available for use.
A decorator is a function that accepts a function as an argument and returns a function as the result. This basic description is clearly a built-in feature of the language. The open question then is how do we update or adjust the internal code structure of a function?
The answer is we don’t. Rather than messing about with the inside of the code, it’s much cleaner to define a new function that wraps the original function. It’s easier to process the argument values or the result and leave the original function’s core processing alone.
We have two phases of higher-order functions involved in defining a decorator; they are as follows:
- At definition time, a decorator function applies a wrapper to a base function and returns the new, wrapped function. The decoration process can do some one-time-only evaluation as part of building the decorated function. Complex default values can be computed, for example.
- At evaluation time, the wrapping function can (and usually does) evaluate the base function. The wrapping function can pre-process the argument values or post-process the return value (or both). It’s also possible that the wrapping function may avoid calling the base function. In the case of managing a cache, for example, the primary reason for wrapping is to avoid expensive calls to the base function.
Simple Decorator function
Here’s an example of a simple decorator:
from functools import wraps from typing import Callable, Optional, Any, TypeVar, cast
FuncType = Callable[..., Any] F = TypeVar('F', bound=FuncType) def nullable(function: F) -> F: @wraps(function) def null_wrapper(arg: Optional[Any]) -> Optional[Any]: return None if arg is None else function(arg) return cast(F, null_wrapper)
We almost always want to use the functools.wraps() function to assure that the decorated function retains the attributes of the original function. Copying the __name__, and __doc__ attributes, for example, assures that the resulting decorated function has the name and docstring of the original function.
The resulting composite function, defined as the null_wrapper() function in the definition of the decorator, is also a type of higher-order function that combines the original function, the function() callable object, in an expression that preserves the None values. Within the resulting null_wrapper() function, the original function callable object is not an explicit argument; it is a free variable that will get its value from the context in which the null_wrapper() function is defined.
The decorator function’s return value is the newly minted function. It will be assigned to the original function’s name. It’s important that decorators only return functions and that they don’t attempt to process data. Decorators use meta-programming: a code that creates a code. The resulting null_wrapper() function, however, will be used to process the real data.
Note that the type hints use a feature of a TypeVar to assure that the result of applying the decorator will be a an object that’s a type of Callable. The type variable F is bound to the original function’s type; the decorator’s type hint claims that the resulting function should have the same type as the argument function. A very general decorator will apply to a wide variety of functions, requiring a type variable binding.
Creating composite function
We can apply our @nullable decorator to create a composite function as follows:
@nullable def nlog(x: Optional[float]) -> Optional[float]: return math.log(x)
This will create a function, nlog(), which is a null-aware version of the built-in math.log() function. The decoration process returned a version of the null_wrapper() function that invokes the original nlog(). This result will be named nlog(), and will have the composite behavior of the wrapping and the original wrapped function.
We can use this composite nlog() function as follows:
>>> some_data = [10, 100, None, 50, 60] >>> scaled = map(nlog, some_data) >>> list(scaled) [2.302585092994046, 4.605170185988092, None, 3.912023005428146, 4.0943445622221]
We’ve applied the function to a collection of data values. The None value politely leads to a None result. There was no exception processing involved.
Here’s how we can create a null-aware rounding function using decorator notation:
@nullable def nround4(x: Optional[float]) -> Optional[float]: return round(x, 4)
This function is a partial application of the round() function, wrapped to be null-aware. In some respects, this is a relatively sophisticated bit of functional programming that’s readily available to Python programmers.
The typing module makes it particularly easy to describe the types of null-aware function and null-aware result, using the Optional type definition. The definition Optional[float] means Union[None, float]; either a None object or a float object may be used.
We could also create the null-aware rounding function using the following code:
nround4 = nullable(lambda x: round(x, 4))
Note that we didn’t use the decorator in front of a function definition. Instead, we applied the decorator to a function defined as a lambda form. This has the same effect as a decorator in front of a function definition.
We can use this round4() function to create a better test case for our nlog() function as follows:
>>> some_data = [10, 100, None, 50, 60] >>> scaled = map(nlog, some_data) >>> [nround4(v) for v in scaled] [2.3026, 4.6052, None, 3.912, 4.0943]
This result will be independent of any platform considerations. It’s very handy for doctest testing.
It can be challenging to apply type hints to lambda forms. The following code shows what is required:
nround4l: Callable[[Optional[float]], Optional[float]] = ( nullable(lambda x: round(x, 4)) )
The variable nround4l is given a type hint of Callable with an argument list of [Optional[float]] and a return type of Optional[float]. The use of the Callable hint is appropriate only for positional arguments. In cases where there will be keyword arguments or other complexities, see http://mypy.readthedocs.io/en/latest/kinds_of_types.html#extended-callable-types.
The @nullable decorator makes an assumption that the decorated function is unary. We would need to revisit this design to create a more general-purpose null-aware decorator that works with arbitrary collections of arguments.
If you found this tutorial useful and interested to learn more such techniques, grab the Steven Lott’s bestseller, Functional Python Programming.