In this article by Denis Perevalov and Igor (Sodazot) Tatarnikov, authors of the book openFrameworks Essentials, we will investigate how to create a distributed project consisting of several programs working together and communicating with each other via networking.
(For more resources related to this topic, see here.)
Distributed computing with networking
Networking is a way of sending and receiving data between programs, which work on a single or different computers and mobile devices. Using networking, it is possible to split a complex project into several programs working together.
There are at least three reasons to create distributed projects:
- The first reason is splitting to obtain better performance. For example, when creating a big interactive wall with cameras and projectors, it is possible to use two computers. The first computer (tracker) will process data from cameras and send the result to the second computer (render), which will render the picture and output it to projectors.
- The second reason is creating a heterogeneous project using different development languages. For example, consider a project that generates a real-time visualization of data captured from the Web. It is easy to capture and analyze the data from the Web using a programming language like Python, but it is hard to create a rich, real-time visualization with it.
On the opposite side, openFrameworks is good for real-time visualization but is not very elegant when dealing with data from the Web. So, it is a good idea to build a project consisting of two programs. The first Python program will capture data from the Web, and the second openFrameworks program will perform rendering. - The third reason is synchronization with, and external control of, one program with other programs/devices. For example, a video synthesizer can be controlled from other computers and mobiles via networking.
Networking in openFrameworks
openFrameworks’ networking capabilities are implemented in two core addons: ofxNetwork and ofxOsc.
To use an addon in your project, you need to include it in the new project when creating a project using Project Generator, or by including the addon’s headers and libraries into the existing project manually. If you need to use only one particular addon, you can use an existing addon’s example as a sketch for your project.
The ofxNetwork addon
The ofxNetwork addon contains classes for sending and receiving data using the Transmission Control Protocol (TCP) and the User Datagram Protocol (UDP). The difference between these protocols is that TCP guarantees receiving data without losses and errors but requires the establishment of a preliminary connection (known as handshake) between a sender and a receiver. UDP doesn’t require the establishment of any preliminary connection but also doesn’t guarantee delivery and correctness of the received data.
Typically, TCP is used in tasks where data needs to be received without errors, such as downloading a JPEG file from a web server. UDP is used in tasks where data should be received in real time at a fast rate, such as receiving a game state 60 times per second in a networking game.
The ofxNetwork addon’s classes are quite generic and allow the implementation of a wide range of low-level networking tasks. In this article, we don’t explore it in detail.
The ofxOsc addon
The ofxOsc addon is intended for sending and receiving messages using the Open Sound Control (OSC) protocol. Messages of this protocol (OSC messages) are intended to store control commands and parameter values.
This protocol is very popular today and is implemented in many VJ and multimedia programs and software for live electronic sound performance. All the popular programming tools support OSC too.
An OSC protocol can use the UDP or TCP protocols for data transmission. Most often, as in openFrameworks implementation, a UDP protocol is used. See details of the OSC protocol at opensoundcontrol.org/spec-1_0.
The main classes of ofxOsc are the following:
- ofxOscSender: This sends OSC messages
- ofxOscReceiver: This receives OSC messages
- ofxOscMessage: This class is for storing a single OSC message
- ofxOscBundle: This class is for storing several OSC messages, which can be sent and received as a bundle
Let’s add the OSC receiver to our VideoSynth project and then create a simple OSC sender, which will send messages to the VideoSynth project.
Implementing the OSC messages receiver
To implement the receiving of OSC messages in the VideoSynth project, perform the following steps:
- Include the ofxOsc addon’s header to the ofApp.h file by inserting the following line after the #include “ofxGui.h” line:
#include "ofxOsc.h"
- Add a declaration of the OSC receiver object to the ofApp class:
ofxOscReceiver oscReceiver;
- Set up the OSC receiver in setup():
oscReceiver.setup( 12345 );
The argument of the setup() method is the networking port number. After executing this command, oscReceiver begins listening on this port for incoming OSC messages. Each received message is added to a special message queue for further processing.
A networking port is a number from 0 to 65535. Ports from 10000 to 65535 normally are not used by existing operating systems, so you can use them as port numbers for OSC messages. Note that two programs receiving networking data and working on the same computer must have different port numbers.
- Add the processing of incoming OSC messages to update():
while ( oscReceiver.hasWaitingMessages() ) {
ofxOscMessage m;
oscReceiver.getNextMessage( &m );
if ( m.getAddress() == "/pinchY" ) {
pinchY = m.getArgAsFloat( 0 );
}
}
The first line is a while loop, which checks whether there are unprocessed messages in the message queue of oscReceiver. The second line declares an empty OSC message m. The third line pops the latest message from the message queue and copies it to m. Now, we can process this message.
Any OSC message consists of two parts: an address and (optionally) one or several arguments. An address is a string beginning with the / character. An address denotes the name of a control command or the name of a parameter that should be adjusted. Arguments can be float, integer, or string values, which specify some parameters of the command.
In our example, we want to adjust the pinchY slider with OSC commands, so we expect to have an OSC message with the address /pinchY and the first argument with its float value. Hence, in the fourth line, we check whether the address of the m message is equal to /pinchY. If this is true, in the fifth line, we get the first message’s argument (an argument with the index value 0) and set the pinchY slider to this value.
Of course, we could use any other address instead of /pinchY (for example, /val), but normally, it is convenient to have the address similar to the parameter’s name.
It is easy to control other sliders with OSC. For example, to add control of the extrude slider, just add the following code:
if ( m.getAddress() == "/extrude" ) {
extrude = m.getArgAsFloat( 0 );
}
After running the project, nothing new happens; it works as always. But now, the project is listening for incoming OSC messages on port 12345. To check this, let’s create a tiny openFrameworks project that sends OSC messages.
Creating an OSC sender with openFrameworks
Let’s create a new project OscOF, one that contains a GUI panel with one slider, and send the slider’s value via OSC to the VideoSynth project.
Here, we assume that the OSC sender and receiver run on the same computer. See the details on running the sender on a separate computer in the upcoming Sending OSC messages between two separate computers section.
Now perform the following steps:
- Create a new project using Project Generator. Namely, start Project Generator, set the project’s name to OscOF (that means OSC with openFrameworks), and include the ofxGui and ofxOsc addons to the newly created project. The ofxGui addon is needed to create the GUI slider, and the ofxOsc addon is needed to send OSC messages.
- Open this project in your IDE.
- Include both addons’ headers to the ofApp.h file by inserting the following lines (after the #include “ofMain.h” line):
#include "ofxGui.h"
#include "ofxOsc.h" - Add the declarations of the OSC sender object, the GUI panel, and the GUI slider to the ofApp class declaration:
ofxOscSender oscSender;
ofxPanel gui;
ofxFloatSlider slider;
void sliderChanged( float &value );The last line declares a new function, which will be called by openFrameworks when the slider’s value is changed. This function will send the corresponding OSC message. The symbol & before value means that the value argument is passed to the function as a reference.
Using reference here is not important for us, but is required by ofxGui; please see the information on the notion of a reference in the C++ documentation.
- Set up the OSC sender, the GUI panel with the slider, and the project’s window title and size by adding the following code to setup():
oscSender.setup( "localhost", 12345 );
slider.addListener( this, &ofApp::sliderChanged );
gui.setup( "Parameters" );
gui.add( slider.setup("pinchY", 0, 0, 1) );
ofSetWindowTitle( "OscOF" );
ofSetWindowShape( 300, 150 );The first line starts the OSC sender. Here, the first argument specifies the IP address to which the OSC sender will send its messages. In our case, it is “localhost”. This means the sender will send data to the same computer on which the sender runs. The second argument specifies the networking port, 12345. The difference between setting up the OSC sender and receiver is that we need to specify the address and port for the sender, and not only the port. Also, after starting, the sender does nothing until we give it the explicit command to send an OSC message.
The second line starts listening to the slider’s value changes. The first and second arguments of the addListener() command specify the object (this) and its member function (sliderChanged), which should be called when the slider is changed.
The remaining lines set up the GUI panel, the GUI slider, and the project’s window title and shape.
- Now, add the sliderChanged() function definition to ofApp.cpp:
void ofApp::sliderChanged( float &value ) {
ofxOscMessage m;
m.setAddress( "/pinchY" );
m.addFloatArg( value );
oscSender.sendMessage( m );
}This function is called when the slider value is changed, and the value parameter is its new value. The first three lines of the function create an OSC message m, set its address to /pinchY, and add a float argument equal to value. The last line sends this OSC message.
As you may see, the m message’s address (/pinchY) coincides with the address implemented in the previous section, which is expected by the receiver. Also, the receiver expects that this message has a float argument—and it is true too! So, the receiver will properly interpret our messages and set its pinchY slider to the desired value.
- Finally, add the command to draw GUI to draw():
gui.draw();
On running the project, you will see its window, consisting of a GUI panel with a slider, as shown in the following screenshot:
This is the OSC sender made with openFrameworks
Don’t stop this project for a while. Run the VideoSynth project and change the pinchY slider’s value in the OscOF window using the mouse. The pinchY slider in VideoSynth should change accordingly. This means that the OSC transmission between the two openFrameworks programs works.
If you are not interested in sending data between two separate computers, feel free to skip the following section.
Sending OSC messages between two separate computers
We have checked passing OSC messages between two programs that run on the same computer. Now let’s consider a situation when an OSC sender and an OSC receiver run on two separate computers connected to the same Local Area Network (LAN) using Ethernet or Wi-Fi.
If you have two laptops, most probably they are already connected to the same networking router and hence are in the same LAN.
To make an OSC connection work in this case, we need to change the “localhost” value in the sender’s setup command by the local IP address of the receiver’s computer.
Typically, this address has a form like “192.168.0.2”, or it could be a name, for example, “LAPTOP3”.
You can get the receiver’s computer IP address by opening the properties of your network adapter or by executing the ifconfig command in the terminal window (for OS X or Linux) or ipconfig in the command prompt window (for Windows).
Connection troubleshooting
If you set the IP address in the sender’s setup, but OSC messages from the OSC sender don’t come to the OSC receiver, then it could be caused by the network firewall or antivirus software, which blocks transmitting data over our 12345 port. So please check the firewall and antivirus settings.
To make sure that the connection between the two computers exists, use the ping command in the terminal (or the command prompt) window.
Creating OSC senders with TouchOSC and Python
At this point, we create the OSC sender using openFrameworks and send its data out to the VideoSynth project. But, it’s easy to create the OSC sender using other programming tools. Such an opportunity can be useful for you in creating complex projects.
So, let’s show how to create an OSC sender on a mobile device using the TouchOSC app and also create simple senders using the Python and Max/MSP languages.
If you are not interested in sending OSC from mobile devices or in Python or Max/MSP, feel free to skip the corresponding sections.
Creating an OSC sender for a mobile device using the TouchOSC app
It is very handy to control your openFrameworks project by a mobile device (or devices) using the OSC protocol.
You can create a custom OSC sender by yourself, or you can use special apps made for this purpose.
One such application is TouchOSC. It’s a paid application available for iOS (see hexler.net/software/touchosc) and Android (see hexler.net/software/touchosc-android).
Working with TouchOSC consists of four steps: creating the GUI panel (called layout) on the laptop, uploading it to a mobile device, setting up the OSC receiver’s address and port, and working with the layout. Let’s consider them in detail:
- To create the layout, download, unzip, and run a special program, TouchOSC Editor, on a laptop (it’s available for OS X, Windows, and Linux). Add the desired GUI elements on the layout by right-clicking on the layout.
- When the layout is ready, upload it to a mobile device by running the TouchOSC app on the mobile and pressing the Sync button in TouchOSC Editor.
- In the TouchOSC app, go to the settings and set up the OSC receiver’s IP address and port number. Next, open the created layout by choosing it from the list of all the existing layouts.
- Now, you can use the layout’s GUI elements to send the OSC messages to your openFrameworks project (and, of course, to any other OSC-supporting software).
Creating an OSC sender with Python
In this section, we will create a project that sends OSC messages using the Python language.
Here, we assume that the OSC sender and receiver run on the same computer. See the details on running the sender on a separate computer in the previous Sending OSC messages between two separate computers section.
Python is a free, interpreted language available for all operating systems. It is extremely popular nowadays in various fields, including teaching programming, developing web projects, and performing computations in natural sciences.
Using Python, you can easily capture information from the Web and social networks (using their API) and send it to openFrameworks for further processing, such as visualization or sonification, that is, converting data to a picture or sound.
Using Python, it is quite easy to create GUI applications, but here we consider creating a project without a GUI.
Perform the following steps to install Python, create an OSC sender, and run it:
- Install Python from www.python.org/downloads (the current version is 3.4).
- Download the python-osc library from pypi.python.org/pypi/python-osc and unzip it. This library implements the OSC protocol support in Python.
- Install this library, open the terminal (or command prompt) window, go to the folder where you unzipped python-osc and type the following:
python setup.py install
If this doesn’t work, type the following:
python3 setup.py install
Python is ready to send OSC messages. Now let’s create the sender program.
- Using your preferred code or text editor, create the OscPython.py file and fill it with the following code:
from pythonosc import udp_client
from pythonosc import osc_message_builder
import time
if __name__ == "__main__":
oscSender = udp_client.UDPClient("localhost", 12345)
for i in range(10):
m = osc_message_builder.OscMessageBuilder(address =
"/pinchY")
m.add_arg(i*0.1)
oscSender.send(m.build())
print(i)
time.sleep(1)The first three lines import the udp_client, osc_message_builder, and time modules for sending the UDP data (we will send OSC messages using UDP), creating OSC messages, and working with time respectively.
The if __name__ == “__main__”: line is generic for Python programs and denotes the part of the code that will be executed when the program runs from the command line.
The first line of the executed code creates the oscSender object, which will send the UDP data to the localhost IP address and the 12345 port. The second line starts a for cycle, where i runs the values 0, 1, 2, …, 9.
The body of the cycle consists of commands for creating an OSC message m with address /pinchY and argument i*0.1, and sending it by OSC. The last two lines print the value i to the console and delay the execution for one second.
- Open the terminal (or command prompt) window, go to the folder with the OscPython.py file, and execute it by the python OscPython.py command. If this doesn’t work, use the python3 OscPython.py command.
The program starts and will send 10 OSC messages with the /pinchY address and the 0.0, 0.1, 0.2, …, 0.9 argument values, with 1 second of pause between the sent messages. Additionally, the program prints values from 0 to 9, as shown in the following screenshot:
This is the output of an OSC sender made with Python
Run the VideoSynth project and start our Python sender again. You will see how its pinchY slider gradually changes from 0.0 to 0.9. This means that OSC transmission from a Python program to an openFrameworks program works.
Summary
In this article, we learned how to create distributed projects using the OSC networking protocol. At first, we implemented receiving OSC in our openFrameworks project. Next, we created a simple OSC sender project with openFrameworks. Then, we considered how to create an OSC sender on mobile devices using TouchOSC and also how to build senders using the Python language. Now, we can control the video synthesizer from other computers or mobile devices via networking.
Resources for Article:
Further resources on this subject:
- Kinect in Motion – An Overview [Article]
- Learn Cinder Basics – Now [Article]
- Getting Started with Kinect [Article]