In this article by Mark Vasilkov, author of the book, Kivy Blueprints, we will emulate the Modern UI by using the grid structure and scalable vector icons and develop a sound recorder for the Android platform using Android Java classes.
(For more resources related to this topic, see here.)
Kivy apps usually end up being cross-platform, mainly because the Kivy framework itself supports a wide range of target platforms. In this write-up, however, we’re building an app that will be single-platform. This gives us an opportunity to rely on platform-specific bindings that provide extended functionality.
The need for such bindings arises from the fact that the input/output capabilities of a pure Kivy program are limited to those that are present on all platforms. This amounts to a tiny fraction of what a common computer system, such as a smartphone or a laptop, can actually do.
Comparison of features
Let’s take a look at the API surface of a modern mobile device (let’s assume it’s running Android). We’ll split everything in two parts: things that are supported directly by Python and/or Kivy and things that aren’t.
The following are features that are directly available in Python or Kivy:
- Hardware-accelerated graphics
- Touchscreen input with optional multitouch
- Sound playback (at the time of writing, this feature is available only from the file on the disk)
- Networking, given the Internet connectivity is present
The following are the features that aren’t supported or require an external library:
- Modem, support for voice calls, and SMS
- Use of built-in cameras for filming videos and taking pictures
- Use of a built-in microphone to record sound
- Cloud storage for application data associated with a user account
- Bluetooth and other near-field networking features
- Location services and GPS
- Fingerprinting and other biometric security
- Motion sensors, that is, accelerometer and gyroscope
- Screen brightness control
- Vibration and other forms of haptic feedback
- Battery charge level
For most entries in the “not supported” list, different Python libraries are already present to fill the gap, such as audiostream for a low-level sound recording, and Plyer that handles many platform-specific tasks.
So, it’s not like these features are completely unavailable to your application; realistically, the challenge is that these bits of functionality are insanely fragmented across different platforms (or even consecutive versions of the same platform, for example, Android); thus, you end up writing platform-specific, not portable code anyway.
As you can see from the preceding comparison, a lot of functionality is available on Android and only partially covered by an existing Python or Kivy API. There is a huge untamed potential in using platform-specific features in your applications. This is not a limitation, but an opportunity. Shortly, you will learn how to utilize any Android API from Python code, allowing your Kivy application to do practically anything.
Another advantage of narrowing the scope of your app to only a small selection of systems is that there are whole new classes of programs that can function (or even make sense) only on a mobile device with fitting hardware specifications. These include augmented reality apps, gyroscope-controlled games, panoramic cameras, and so on.
Introducing Pyjnius
To harness the full power of our chosen platform, we’re going to use a platform-specific API, which happens to be in Java and is thus primarily Java oriented. We are going to build a sound recorder app, similar to the apps commonly found in Android and iOS, albeit more simplistic. Unlike pure Kivy apps, the underlying Android API certainly provides us with ways of recording sound programmatically.
The rest of the article will cover this little recorder program throughout its development to illustrate the Python-Java interoperability using the excellent Pyjnius library, another great project made by Kivy developers. The concept we chose—sound recording and playback—is deliberately simple so as to outline the features of such interoperation without too much distraction caused by the sheer complexity of a subject and abundant implementation details.
The source code of Pyjnius, together with the reference manual and some examples, can be found in the official repository at https://github.com/kivy/pyjnius.
Modern UI
While we’re at it, let’s build a user interface that resembles the Windows Phone home screen. This concept, basically a grid of colored rectangles (tiles) of various sizes, was known as Metro UI at some point in time but was later renamed to Modern UI due to trademark issues. Irrespective of the name, this is how it looks. This will give you an idea of what we’ll be aiming at during the course of this app’s development:
Design inspiration – a Windows Phone home screen with tiles
Obviously, we aren’t going to replicate it as is; we will make something that resembles the depicted user interface. The following list pretty much summarizes the distinctive features we’re after:
- Everything is aligned to a rectangular grid
- UI elements are styled using the streamlined, flat design—tiles use bright, solid colors and there are no shadows or rounded corners
- Tiles that are considered more useful (for an arbitrary definition of “useful”) are larger and thus easier to hit
If this sounds easy to you, then you’re absolutely right. As you will see shortly, the Kivy implementation of such a UI is rather straightforward.
The buttons
To start off, we are going to tweak the Button class in Kivy language (let’s name the file recorder.kv):
#:import C kivy.utils.get_color_from_hex <Button>:
background_normal: 'button_normal.png'
background_down: 'button_down.png'
background_color: C('#95A5A6')
font_size: 40
The texture we set as the background is solid white, exploiting the same trick that was used while creating the color palette. The background_color property acts as tint color, and assigning a plain white texture equals to painting the button in background_color. We don’t want borders this time.
The second (pressed background_down) texture is 25 percent transparent white. Combined with the pitch-black background color of the app, we’re getting a slightly darker shade of the same background color the button was assigned:
Normal (left) and pressed (right) states of a button – the background color is set to #0080FF
The grid structure
The layout is a bit more complex to build. In the absence of readily available Modern UI-like tiled layout, we are going to emulate it with the built-in GridLayout widget.
One such widget could have fulfilled all our needs, if not for the last requirement: we want to have bigger and smaller buttons. Presently, GridLayout doesn’t allow the merging of cells to create bigger ones (a functionality similar to the rowspan and colspan attributes in HTML would be nice to have). So, we will go in the opposite direction: start with the root GridLayout with big cells and add another GridLayout inside a cell to subdivide it.
Thanks to nested layouts working great in Kivy, we arrive at the following Kivy language structure (in recorder.kv):
#:import C kivy.utils.get_color_from_hex GridLayout:
padding: 15
Button:
background_color: C('#3498DB')
text: 'aaa'
GridLayout:
Button:
background_color: C('#2ECC71')
text: 'bbb1 '
Button:
background_color: C('#1ABC9C')
text: 'bbb2'
Button:
background_color: C('#27AE60')
text: 'bbb3'
Button:
background_color: C('#16A085')
text: 'bbb4'
Button:
background_color: C('#E74C3C')
text: 'ccc'
Button:
background_color: C('#95A5A6')
text: 'ddd'
Note how the nested GridLayout sits on the same level as that of outer, large buttons. This should make perfect sense if you look at the previous screenshot of the Windows Phone home screen: a pack of four smaller buttons takes up the same space (one outer grid cell) as a large button. The nested GridLayout is a container for those smaller buttons.
Visual attributes
On the outer grid, padding is provided to create some distance from the edges of the screen. Other visual attributes are shared between GridLayout instances and moved to a class. The following code is present inside recorder.kv:
<GridLayout>:
cols: 2
spacing: 10
row_default_height:
(0.5 * (self.width - self.spacing[0]) -
self.padding[0])
row_force_default: True
It’s worth mentioning that both padding and spacing are effectively lists, not scalars. spacing[0] refers to a horizontal spacing, followed by a vertical one. However, we can initialize spacing with a single value, as shown in the preceding code; this value will then be used for everything.
Each grid consists of two columns with some spacing in between. The row_default_height property is trickier: we can’t just say, “Let the row height be equal to the row width.” Instead, we compute the desired height manually, where the value 0.5 is used because we have two columns:
If we don’t apply this tweak, the buttons inside the grid will fill all the available vertical space, which is undesirable, especially when there aren’t that many buttons (every one of them ends up being too large). Instead, we want all the buttons nice and square, with empty space at the bottom left, well, empty.
The following is the screenshot of our app’s “Modern UI” tiles, which we obtained as result from the preceding code:
The UI so far – clickable tiles of variable size not too dissimilar from our design inspiration
Scalable vector icons
One of the nice finishing touches we can apply to the application UI is the use of icons, and not just text, on buttons. We could, of course, just throw in a bunch of images, but let’s borrow another useful technique from modern web development and use an icon font instead—as you will see shortly, these provide great flexibility at no cost.
Icon fonts
Icon fonts are essentially just like regular ones, except their glyphs are unrelated to the letters of a language. For example, you type P and the Python logo is rendered instead of the letter; every font invents its own mnemonic on how to assign letters to icons.
There are also fonts that don’t use English letters, instead they map icons to Unicode’s “private use area” character code. This is a technically correct way to build such a font, but application support for this Unicode feature varies—not every platform behaves the same in this regard, especially the mobile platform. The font that we will use for our app does not assign private use characters and uses ASCII (plain English letters) instead.
Rationale to use icon fonts
On the Web, icon fonts solve a number of problems that are commonly associated with (raster) images:
- First and foremost, raster images don’t scale well and may become blurry when resized—there are certain algorithms that produce better results than others, but as of today, the “state of the art” is still not perfect. In contrast, a vector picture is infinitely scalable by definition.
- Raster image files containing schematic graphics (such as icons and UI elements) tend to be larger than vector formats. This does not apply to photos encoded as JPEG obviously.
- With an icon font, color changes literally take seconds—you can do just that by adding color: red (for example) to your CSS file. The same is true for size, rotation, and other properties that don’t involve changing the geometry of an image. Effectively, this means that making trivial adjustments to an icon does not require an image editor, like it normally would when dealing with bitmaps.
Some of these points do not apply to Kivy apps that much, but overall, the use of icon fonts is considered a good practice in contemporary web development, especially since there are many free high-quality fonts to choose from—that’s hundreds of icons readily available for inclusion in your project.
Using the icon font in Kivy
In our application, we are going to use the Modern Pictograms (Version 1) free font, designed by John Caserta. To load the font into our Kivy program, we’ll use the following code (in main.py):
from kivy.app import App
from kivy.core.text import LabelBase
class RecorderApp(App):
pass
if __name__ == '__main__':
LabelBase.register(name='Modern Pictograms',
fn_regular='modernpics.ttf')
RecorderApp().run()
The actual use of the font happens inside recorder.kv. First, we want to update the Button class once again to allow us to change the font in the middle of a text using markup tags. This is shown in the following snippet:
<Button>:
background_normal: 'button_normal.png'
background_down: 'button_down.png'
font_size: 24
halign: 'center'
markup: True
The halign: ‘center’ attribute means that we want every line of text centered inside the button. The markup: True attribute is self-evident and required because the next step in customization of buttons will rely heavily on markup.
Now we can update button definitions. Here’s an example of this:
Button:
background_color: C('#3498DB')
text:
('[font=Modern Pictograms][size=120]'
'e[/size][/font]nNew recording')
Notice the character ‘e’ inside the [font][size] tags. That’s the icon code. Every button in our app will use a different icon, and changing an icon amounts to replacing a single letter in the recorder.kv file. Complete mapping of these code for the Modern Pictograms font can be found on its official website at http://modernpictograms.com/.
Long story short, this is how the UI of our application looks after the addition of icons to buttons:
The sound recorder app interface – a modern UI with vector icons from the Modern Pictograms font
This is already pretty close to the original Modern UI look.
Using the native API
Having completed the user interface part of the app, we will now turn to a native API and implement the sound recording and playback logic using the suitable Android Java classes, MediaRecorder and MediaPlayer.
Thankfully, the task at hand is relatively simple. To record a sound using the Android API, we only need the following five Java classes:
- The class android.os.Environment provides access to many useful environment variables. We are going to use it to determine the path where the SD card is mounted so we can save the recorded audio file. It’s tempting to just hardcode ‘/sdcard/‘ or a similar constant, but in practice, every other Android device has a different filesystem layout. So let’s not do this even for the purposes of the tutorial.
- The class android.media.MediaRecorder is our main workhorse. It facilitates capturing audio and video and saving it to the filesystem.
- The classes android.media.MediaRecorder$AudioSource, android.media.MediaRecorder$AudioEncoder, and android.media.MediaRecorder$OutputFormat are enumerations that hold the values we need to pass as arguments to the various methods of MediaRecorder.
Loading Java classes
The code to load the aforementioned Java classes into your Python application is as follows:
from jnius import autoclass
Environment = autoclass('android.os.Environment')
MediaRecorder = autoclass('android.media.MediaRecorder')
AudioSource = autoclass('android.media.MediaRecorder$AudioSource')
OutputFormat = autoclass('android.media.MediaRecorder$OutputFormat')
AudioEncoder = autoclass('android.media.MediaRecorder$AudioEncoder')
If you try to run the program at this point, you’ll receive an error, something along the lines of:
- ImportError: No module named jnius: You’ll encounter this error if you don’t have Pyjnius installed on your machine
- jnius.JavaException: Class not found ‘android/os/Environment’: You’ll encounter this error if Pyjnius is installed, but the Android classes we’re trying to load are missing (for example, when running on a desktop)
This is one of the rare cases when receiving an error means we did everything right. From now on, we should do all of the testing on Android device or inside an emulator because the code isn’t cross-platform anymore. It relies unequivocally on Android-specific Java features.
Now we can use Java classes seamlessly in our Python code.
Looking up the storage path
Let’s illustrate the practical cross-language API use with a simple example. In Java, we will do something like this in order to find out where an SD card is mounted:
import android.os.Environment;
String path = Environment.getExternalStorageDirectory()
.getAbsolutePath();
When translated to Python, the code is as follows:
Environment = autoclass('android.os.Environment')
path = Environment.getExternalStorageDirectory().getAbsolutePath()
This is the exact same thing as shown in the previous code, only written in Python instead of Java.
While we’re at it, let’s also log this value so that we can see which exact path in the Kivy log the getAbsolutePath method returned to our code:
from kivy.logger import Logger
Logger.info('App: storage path == "%s"' % path)
On my testing device, this produces the following line in the Kivy log:
[INFO] App: storage path == "/storage/sdcard0"
Recording sound
Now, let’s dive deeper into the rabbit hole of the Android API and actually record a sound from the microphone. The following code is again basically a translation of Android API documents into Python. If you’re interested in the original Java version of this code, you may find it at http://developer.android.com/guide/topics/media/audio-capture.html —it’s way too lengthy to include here.
The following preparation code initializes a MediaRecorder object:
storage_path = (Environment.getExternalStorageDirectory()
.getAbsolutePath() + '/kivy_recording.3gp')
recorder = MediaRecorder()
def init_recorder():
recorder.setAudioSource(AudioSource.MIC)
recorder.setOutputFormat(OutputFormat.THREE_GPP)
recorder.setAudioEncoder(AudioEncoder.AMR_NB)
recorder.setOutputFile(storage_path)
recorder.prepare()
This is the typical, straightforward, verbose, Java way of initializing things, which is rewritten in Python word for word.
Now for the fun part, the Begin recording/End recording button:
class RecorderApp(App):
is_recording = False
def begin_end_recording(self):
if (self.is_recording):
recorder.stop()
recorder.reset()
self.is_recording = False
self.root.ids.begin_end_recording.text =
('[font=Modern Pictograms][size=120]'
'e[/size][/font]nBegin recording')
return
init_recorder()
recorder.start()
self.is_recording = True
self.root.ids.begin_end_recording.text =
('[font=Modern Pictograms][size=120]'
'%[/size][/font]nEnd recording')
As you can see, no rocket science was applied here either. We just stored the current state, is_recording, and then took the action depending on it, namely:
- Start or stop the MediaRecorder object (the highlighted part).
- Flip the is_recording flag.
- Update the button text so that it reflects the current state (see the next screenshot).
The last part of the application that needs updating is the recorder.kv file. We need to tweak the Begin recording/End recording button so that it calls our begin_end_recording() function:
Button:
id: begin_end_recording
background_color: C('#3498DB')
text:
('[font=Modern Pictograms][size=120]'
'e[/size][/font]nBegin recording')
on_press: app.begin_end_recording()
That’s it! If you run the application now, chances are that you’ll be able to actually record a sound file that is going to be stored on the SD card. However, please see the next section before you do this. The button that you created will look something like this:
Begin recording and End recording – this one button summarizes our app’s functionality so far.
Major caveat – permissions
The default Kivy Launcher app at the time of writing this doesn’t have the necessary permission to record sound, android.permission.RECORD_AUDIO. This results in a crash as soon as the MediaRecorder instance is initialized.
There are many ways to mitigate this problem. For the sake of this tutorial, we provide a modified Kivy Launcher that has the necessary permission enabled. The latest version of the package is also available for download at https://github.com/mvasilkov/kivy_launcher_hack.
Before you install the provided .apk file, please delete the existing version of the app, if any, from your device.
Alternatively, if you’re willing to fiddle with the gory details of bundling Kivy apps for Google Play, you can build Kivy Launcher yourself from the source code. Everything you need to do this can be found in the official Kivy GitHub account, https://github.com/kivy.
Playing sound
Getting sound playback to work is easier; there is no permission for this and the API is somewhat more concise too. We need to load just one more class, MediaPlayer:
MediaPlayer = autoclass('android.media.MediaPlayer')
player = MediaPlayer()
The following code will run when the user presses the Play button. We’ll also use the reset_player() function in the Deleting files section discussed later in this article; otherwise, there could have been one slightly longer function:
def reset_player():
if (player.isPlaying()):
player.stop()
player.reset()
def restart_player():
reset_player()
try:
player.setDataSource(storage_path)
player.prepare()
player.start()
except:
player.reset()
The intricate details of each API call can be found in the official documents, but overall, this listing is pretty self-evident: reset the player to its initial state, load the sound file, and press the Play button. The file format is determined automatically, making our task at hand a wee bit easier.
Deleting files
This last feature will use the java.io.File class, which is not strictly related to Android. One great thing about the official Android documentation is that it contains reference to these core Java classes too, despite the fact they predate the Android operating system by more than a decade. The actual code needed to implement file removal is exactly one line; it’s highlighted in the following listing:
File = autoclass('java.io.File')
class RecorderApp(App):
def delete_file(self):
reset_player()
File(storage_path).delete()
First, we stop the playback (if any) by calling the reset_player() function and then remove the file—short and sweet.
Interestingly, the File.delete() method in Java won’t throw an exception in the event of a catastrophic failure, so there is no need to perform try … catch in this case. Consistency, consistency everywhere.
An attentive reader will notice that we could also delete the file using Python’s own os.remove() function. Doing this using Java achieves nothing special compared to a pure Python implementation; it’s also slower. On the other hand, as a demonstration of Pyjnius, java.io.File works as good as any other Java class.
At this point, with the UI and all three major functions done, our application is complete for the purposes of this tutorial.
Summary
Writing nonportable code has its strengths and weaknesses, just like any other global architectural decision. This particular choice, however, is especially hard because the switch to native API typically happens early in the project and may be completely impractical to undo at a later stage.
The major advantage of the approach was discussed at the beginning of this article: with platform-specific code, you can do virtually anything that your platform is capable of. There are no artificial limits; your Python code has unrestricted access to the same underlying API as the native code.
On the downside, depending on a single-platform is risky for a number of reasons:
- The market of Android alone is provably smaller than that of Android plus iOS (this holds true for about every combination of operating systems).
- Porting the program over to a new system becomes harder with every platform-specific feature you use.
- If the project runs on just one platform, exactly one political decision may be sufficient to kill it. The chances of getting banned by Google is higher than that of getting the boot from both App Store and Google Play simultaneously. (Again, this holds true for practically every set of application marketplaces.)
Now that you’re well aware of the options, it’s up to you to make an educated choice regarding every app you develop.
Resources for Article:
Further resources on this subject:
- Reversing Android Applications [Article]
- Creating a Direct2D game window class [Article]
- Images, colors, and backgrounds [Article]