(For more resources related to this topic, see here.)
Mission Briefing
To create the Processing sketches for this project, we will need to install the Processing library ttslib. This library is a wrapper around the FreeTTS Java library that helps us to write a sketch that reads out text. We will learn how to change the voice parameters of the kevin16 voice of the FreeTTS package to make our robot’s voices distinguishable. We will also create a parser that is able to read the Shakespeare script and which generates text-line objects that allow our script to know which line is read by which robot.
A Drama thread will be used to control the text-to-speech objects, and the draw() method of our sketch will print the script on the screen while our robots perform it, just in case one of them forgets a line. Finally, we will use some cardboard boxes and a pair of cheap speakers to create the robots and their stage. The following figure shows how the robots work:
Why Is It Awesome?
Since the 18th century, inventors have tried to build talking machines (with varying success). Talking toys swamped the market in the 1980s and 90s. In every decent Sci-Fi novel, computers and robots are capable of speaking. So how could building talking robots not be awesome? And what could be more appropriate to put these speaking capabilities to test than performing a Shakespeare play? So as you see, building actor robots is officially awesome, just in case your non-geek family members should ask.
Your Hotshot Objectives
We will split this project into four tasks that will guide you through the general on of the robots from beginning to end. Here is a short overview of what we are going to do:
- Making Processing talk
- Reading Shakespeare
- Adding more actors
- Building robots
Making Processing talk
Since Processing has no speaking capabilities out of the box, our first task is adding an external library using the new Processing Library Manager. We will use the ttslib package, which is a wrapper library around the FreeTTS library.
We will also create a short, speaking Processing sketch to check the installation.
Engage Thrusters
- Processing can be extended by contributed libraries. Most of these additional libraries can be installed by navigating to Sketch | Import Library… | Add Library…, as shown in the following screenshot:
- In the Library Manager dialog, enter ttslib in the search field to filter the list of libraries.
- Click on the ttslib entry and then on the Install button, as shown in the following screenshot, to download and install the library:
- To use the new library, we need to import it to our sketch. We do this by clicking on the Sketch menu and choosing Import Library… and then ttslib.
- We will now add the setup() and draw() methods to our sketch. We will leave the draw() method empty for now and instantiate a TTS object in the setup() method. Your sketch should look like the following code snippet:
import guru.ttslib.*;
TTS tts;
void setup() {
tts = new TTS();
}
void draw() {
} - Now we will add a mousePressed() method to our sketch, which will get called if someone clicks on our sketch window. In this method, we are calling the speak() method of the TTS object we created in the setup() method.
void mousePressed() {
tts.speak("Hello, I am a Computer");
} - Click on the Run button to start the Processing sketch. A little gray window should appear.
- Turn on your speakers or put on your headphones, and click on the gray window. If nothing went wrong, a friendly male computer voice named kevin16 should greet you now.
Objective Complete – Mini Debriefing
In steps 1 to 3, we installed an additional library to Processing. The ttslib is a wrapper library around the FreeTTS text-to-speech engine.
Then we created a simple Processing sketch that imports the installed library and creates an instance of the TTS class. The TTS objects match the speakers we need in our sketches. In this case, we created only one speaker and added a mousePressed() method that calls the speak() method of our tts object.
Reading Shakespeare
In this part of the project, we are going to create a Drama thread and teach Processing how to read a Shakespeare script. This thread runs in the background and is controlling the performance. We focus on reading and executing the play in this task, and add the speakers in the next one.
Prepare for Lift Off
Our sketch needs to know which line of the script is read by which robot. So we need to convert the Shakespeare script into a more machine-readable format. For every line of text, we need to know which speaker should read the line. So we take the script and add the letter J and a separation character that is used nowhere else in the script, in front of every line our Juliet-Robot should speak, and we add R and the separation letter for every line our Romeo-Robot should speak. After all these steps, our text file looks something like the following:
R# Lady, by yonder blessed moon I vow,
R# That tips with silver all these fruit-tree tops --
J# O, swear not by the moon, the inconstant moon,
J# That monthly changes in her circled orb,
J# Lest that thy love prove likewise variable.
R# What shall I swear by?
J# Do not swear at all.
J# Or if thou wilt, swear by thy gracious self,
J# Which is the god of my idolatry,
J# And I'll believe thee.
Engage Thrusters
Let’s write our parser:
- Let’s start a new sketch by navigating to File | New.
- Add a setup() and a draw() method.
- Now add the prepared script to the Processing sketch by navigating to Sketch | Add File and selecting the file you just downloaded.
- Add the following line to your setup() method:
void setup() {
String[] rawLines = loadStrings
( "romeo_and_juliet.txt" );
} - If you renamed your text file, change the filename accordingly.
- Create a new tab by clicking on the little arrow icon on the right and choosing New Tab.
- Name the class Line. This class will hold our text lines and the speaker.
- Add the following code to the tab we just created:
public class Line {
String speaker;
String text;
public Line( String speaker, String text ) {
this.speaker = speaker;
this.text = text;
}
} - Switch back to our main tab and add the following highlighted lines of code to the setup() method:
void setup() {
String[] rawLines = loadStrings
( "romeo_and_juliet.txt" );
ArrayList lines = new ArrayList();
for ( int i=0; i<rawLines.length; i++) {
if (!"".equals(rawLines[i])) {
String[] tmp = rawLines[i].split("#");
lines.add( new Line( tmp[0], tmp[1].trim() ));
}
}
} - We have read our text lines and parsed them into the lines array list, but we still need a class that does something with our text lines. So create another tab by clicking on the arrow icon and choosing New Tab from the menu; name it Drama.
- Our Drama class will be a thread that runs in the background and tells each of the speaker objects to read one line of text. Add the following lines of code to your Drama class:
public class Drama extends Thread {
int current;
ArrayList lines;
boolean running;
public Drama( ArrayList lines ) {
this.lines = lines;
current = 0;
running = false;
}
public int getCurrent() {
return current;
}
public Line getLine( int num ) {
if ( num >=0 && num < lines.size()) {
return (Line)lines.get( num );
} else {
return null;
}
}
public boolean isRunning() {
return running;
}
} - Now we add a run() method that gets executed in the background if we start our thread. Since we have no speaker objects yet, we will print the lines on the console and include a little pause after each line.
public void run() {
running = true;
for ( int i =0; i < lines.size(); i++) {
current = i;
Line l = (Line)lines.get(i);
System.out.println( l.text );
delay( 1 );
}
running = false;
} - Switch back to the main sketch tab and add the highlighted code to the setup() method to create a drama thread object, and then feed it the parsed text-lines.
Drama drama;
void setup() {
String[] rawLines = loadStrings
( "romeo_and_juliet.txt" );
ArrayList lines = new ArrayList();
for ( int i=0; i<rawLines.length; i++) {
if (!"".equals(rawLines[i])) {
String[] tmp = rawLines[i].split("#");
lines.add( new Line( tmp[0], tmp[1].trim() ));
}
}
drama = new Drama( lines );
} - So far our sketch parses the text lines and creates a Drama thread object. What we need next is a method to start it. So add a mousePressed() method to start the drama thread.
void mousePressed() {
if ( !drama.isRunning()) {
drama.start();
}
} - Now add a little bit of text to the draw() method to tell the user what to do. Add the following code to the draw() method:
void draw() {
background(255);
textAlign(CENTER);
fill(0);
text( "Click here for Drama", width/2, height/2 );
} - Currently, our sketch window is way too small to contain the text, and we also want to use a bigger font. To change the window size, we simply add the following line to the setup() method:
void setup() {
size( 800, 400 );
String[] rawLines = loadStrings
( "romeo_and_juliet.txt" );
ArrayList lines = new ArrayList();
for ( int i=0; i<rawLines.length; i++) {
if (!"".equals(rawLines[i])) {
String[] tmp = rawLines[i].split("#");
lines.add( new Line( tmp[0], tmp[1].trim() ));
}
}
drama = new Drama( lines );
} - To change the used font, we need to tell Processing which font to use. The easiest way to find out the names of the fonts that are currently installed on the computer is to create a new sketch, type the following line, and run the sketch:
println(PFont.list());
- Copy one of the font names you like and add the following line to the Romeo and Juliet sketch:
void setup() {
size( 800, 400 );
textFont( createFont( "Georgia", 24 ));
... - Replace the font name in the code lines with one of the fonts on your computer.
Objective Complete – Mini Debriefing
In this section, we wrote the code that parses a text file and generates a list of Line objects. These objects are then used by a Drama thread that runs in the background as soon as anyone clicks on the sketch window. Currently, the Drama thread prints out the text line on the console.
In steps 6 to 8, we created the Line class. This class is a very simple, so-called Plain Old Java Object (POJO) that holds our text lines, but it doesn’t add any functionality.
The code that is controlling the performance of our play was created in steps 10 to 12. We created a thread that is able to run in the background, since in the next step we want to be able to use the draw() method and some TTS objects simultaneously.
The code block in step 12 defines a Boolean variable named running, which we used in the mousePressed() method to check if the sketch is already running or should be started.
Classified Intel
In step 17, we used the list() method of the PFont class to get a list of installed fonts. This is a very common pattern in Processing. You would use the same approach to get a list of installed midi-interfaces, web-cams, serial-ports, and so on.
