In this article written by Cyrille Rossant, author of Learning IPython for Interactive Computing and Data Visualization – Second edition, we look at how the Jupyter Notebook is a highly-customizable platform. You can configure many aspects of the software and can extend the backend (kernels) and the frontend (HTML-based Notebook). This allows you to create highly-personalized user experiences based on the Notebook.
In this article, we will cover the following topics:
- Creating a custom magic command in an IPython extension
- Writing a new Jupyter kernel
- Customizing the Notebook interface with JavaScript
Creating a custom magic command in an IPython extension
IPython comes with a rich set of magic commands. You can get the complete list with the %lsmagic command. IPython also allows you to create your own magic commands. In this section, we will create a new cell magic that compiles and executes C++ code in the Notebook.
We first import the register_cell_magic function:
In [1]: from IPython.core.magic import register_cell_magicTo create a new cell magic, we create a function that takes a line (containing possible options) and a cell’s contents as its arguments, and we decorate it with @register_cell_magic, as shown here:
In [2]: @register_cell_magic
def cpp(line, cell):
"""Compile, execute C++ code, and return the
standard output."""
# We first retrieve the current IPython interpreter
# instance.
ip = get_ipython()
# We define the source and executable filenames.
source_filename = '_temp.cpp'
program_filename = '_temp'
# We write the code to the C++ file.
with open(source_filename, 'w') as f:
f.write(cell)
# We compile the C++ code into an executable.
compile = ip.getoutput("g++ {0:s} -o {1:s}".format(
source_filename, program_filename))
# We execute the executable and return the output.
output = ip.getoutput('./{0:s}'.format(program_filename))
print('n'.join(output))C++ compiler
This recipe requires the gcc C++ compiler. On Ubuntu, type sudo apt-get install build-essential in a terminal. On OS X, install Xcode. On Windows, install MinGW (http://www.mingw.org) and make sure that g++ is in your system path.
This magic command uses the getoutput() method of the IPython InteractiveShell instance. This object represents the current interactive session. It defines many methods for interacting with the session. You will find the comprehensive list at http://ipython.org/ipython-doc/dev/api/generated/IPython.core.interactiveshell.html#IPython.core.interactiveshell.InteractiveShell.
Let’s now try this new cell magic.
In [3]: %%cpp
#include<iostream>
int main()
{
std::cout << "Hello world!";
}
Out[3]: Hello world!
This cell magic is currently only available in your interactive session. To distribute it, you need to create an IPython extension. This is a regular Python module or package that extends IPython.
To create an IPython extension, copy the definition of the cpp() function (without the decorator) to a Python module, named cpp_ext.py for example. Then, add the following at the end of the file:
def load_ipython_extension(ipython):
"""This function is called when the extension is loaded.
It accepts an IPython InteractiveShell instance.
We can register the magic with the `register_magic_function`
method of the shell instance."""
ipython.register_magic_function(cpp, 'cell')
Then, you can load the extension with %load_ext cpp_ext. The cpp_ext.py file needs to be in the PYTHONPATH, for example in the current directory.
Writing a new Jupyter kernel
Jupyter supports a wide variety of kernels written in many languages, including the most-frequently used IPython. The Notebook interface lets you choose the kernel for every notebook. This information is stored within each notebook file.
The jupyter kernelspec command allows you to get information about the kernels. For example, jupyter kernelspec list lists the installed kernels. Type jupyter kernelspec –help for more information.
At the end of this section, you will find references with instructions to install various kernels including IR, IJulia, and IHaskell. Here, we will detail how to create a custom kernel.
There are two methods to create a new kernel:
- Writing a kernel from scratch for a new language by re-implementing the whole Jupyter messaging protocol.
- Writing a wrapper kernel for a language that can be accessed from Python.
We will use the second, easier method in this section. Specifically, we will reuse the example from the last section to write a C++ wrapper kernel.
We need to slightly refactor the last section’s code because we won’t have access to the InteractiveShell instance. Since we’re creating a kernel, we need to put the code in a Python script in a new folder named cpp:
In [1]: %mkdir cppThe %%writefile cell magic lets us create a cpp_kernel.py Python script from the Notebook:
In [2]: %%writefile cpp/cpp_kernel.py
import os
import os.path as op
import tempfile
# We import the `getoutput()` function provided by IPython.
# It allows us to do system calls from Python.
from IPython.utils.process import getoutput
def exec_cpp(code):
"""Compile, execute C++ code, and return the standard
output."""
# We create a temporary directory. This directory will
# be deleted at the end of the 'with' context.
# All created files will be in this directory.
with tempfile.TemporaryDirectory() as tmpdir:
# We define the source and executable filenames.
source_path = op.join(tmpdir, 'temp.cpp')
program_path = op.join(tmpdir, 'temp')
# We write the code to the C++ file.
with open(source_path, 'w') as f:
f.write(code)
# We compile the C++ code into an executable.
os.system("g++ {0:s} -o {1:s}".format(
source_path, program_path))
# We execute the program and return the output.
return getoutput(program_path)
Out[2]: Writing cpp/cpp_kernel.py
Now we create our wrapper kernel by appending some code to the cpp_kernel.py file created above (that’s what the -a option in the %%writefile cell magic is for):
In [3]: %%writefile -a cpp/cpp_kernel.py
"""C++ wrapper kernel."""
from ipykernel.kernelbase import Kernel
class CppKernel(Kernel):
# Kernel information.
implementation = 'C++'
implementation_version = '1.0'
language = 'c++'
language_version = '1.0'
language_info = {'name': 'c++',
'mimetype': 'text/plain'}
banner = "C++ kernel"
def do_execute(self, code, silent,
store_history=True,
user_expressions=None,
allow_stdin=False):
"""This function is called when a code cell is
executed."""
if not silent:
# We run the C++ code and get the output.
output = exec_cpp(code)
# We send back the result to the frontend.
stream_content = {'name': 'stdout',
'text': output}
self.send_response(self.iopub_socket,
'stream', stream_content)
return {'status': 'ok',
# The base class increments the execution
# count
'execution_count': self.execution_count,
'payload': [],
'user_expressions': {},
}
if __name__ == '__main__':
from ipykernel.kernelapp import IPKernelApp
IPKernelApp.launch_instance(kernel_class=CppKernel)
Out[3]: Appending to cpp/cpp_kernel.py
In production code, it would be best to test the compilation and execution, and to fail gracefully by showing an error. See the references at the end of this section for more information.
Our wrapper kernel is now implemented in cpp/cpp_kernel.py. The next step is to create a cpp/kernel.json file describing our kernel:
In [4]: %%writefile cpp/kernel.json
{
"argv": ["python",
"cpp/cpp_kernel.py",
"-f",
"{connection_file}"
],
"display_name": "C++"
}
Out[4]: Writing cpp/kernel.json
The argv field describes the command that is used to launch a C++ kernel. More information can be found in the references below.
Finally, let’s install this kernel with the following command:
In [5]: !jupyter kernelspec install --replace --user cpp
Out[5]: [InstallKernelSpec] Installed kernelspec cpp in /Users/cyrille/Library/Jupyter/kernels/cpp
The –replace option forces the installation even if the kernel already exists. The –user option serves to install the kernel in the user directory. We can test the installation of the kernel with the following command:
In [6]: !jupyter kernelspec list
Out[6]: Available kernels:
cpp
python3
Now, C++ notebooks can be created in the Notebook, as shown in the following screenshot:
C++ kernel in the Notebook
Finally, wrapper kernels can also be used in the IPython terminal or the Qt console, using the –kernel option, for example ipython console –kernel cpp.
Here are a few references:
- Kernel documentation at http://jupyter-client.readthedocs.org/en/latest/kernels.html
- Wrapper kernels at http://jupyter-client.readthedocs.org/en/latest/wrapperkernels.html
- List of kernels at https://github.com/ipython/ipython/wiki/IPython%20kernels%20for%20other%20languages
- bash kernel at https://github.com/takluyver/bash_kernel
- R kernel at https://github.com/takluyver/IRkernel
- Julia kernel at https://github.com/JuliaLang/IJulia.jl
- Haskell kernel at https://github.com/gibiansky/IHaskell
Customizing the Notebook interface with JavaScript
The Notebook application exposes a JavaScript API that allows for a high level of customization. In this section, we will create a new button in the Notebook toolbar to renumber the cells.
The JavaScript API is not stable and not well-documented. Although the example in this section has been tested with IPython 4.0, nothing guarantees that it will work in future versions without changes.
The commented JavaScript code below adds a new Renumber button.
In [1]: %%javascript
// This function allows us to add buttons
// to the Notebook toolbar.
IPython.toolbar.add_buttons_group([
{
// The button's label.
'label': 'Renumber all code cells',
// The button's icon.
// See a list of Font-Awesome icons here:
// http://fortawesome.github.io/Font-Awesome/icons/
'icon': 'fa-list-ol',
// The callback function called when the button is
// pressed.
'callback': function () {
// We retrieve the lists of all cells.
var cells = IPython.notebook.get_cells();
// We only keep the code cells.
cells = cells.filter(function(c)
{
return c instanceof IPython.CodeCell;
})
// We set the input prompt of all code cells.
for (var i = 0; i < cells.length; i++) {
cells[i].set_input_prompt(i + 1);
}
}
}]);
Executing this cell displays a new button in the Notebook toolbar, as shown in the following screenshot:
Adding a new button in the Notebook toolbar
You can use the jupyter nbextension command to install notebook extensions (use the –help option to see the list of possible commands).
Here are a few repositories with custom JavaScript extensions contributed by the community:
- https://github.com/minrk/ipython_extensions
- https://github.com/ipython-contrib/IPython-notebook-extensions
So, we have covered several customization options of IPython and the Jupyter Notebook, but there’s so much more that can be done.
Take a look at the IPython Interactive Computing and Visualization Cookbook to learn how to create your own custom widgets in the Notebook.