Android Application Testing: Getting Started

0
110
9 min read

We will avoid introductions to Android and the Open Handset Alliance (http://www.openhandsetalliance.com) as they are covered in many books already and I am inclined to believe that if you are reading this article covering this more advanced topic you have started with Android development before.

However, we will be reviewing the main concepts behind testing and the techniques, frameworks, and tools available to deploy your testing strategy on Android.

Brief history

Initially, when Android was introduced by the end of 2007, there was very little support for testing in the platform, and for some of us very accustomed to using testing as a component intimately coupled with the development process, it was the time to start developing some frameworks and tools to permit this approach.

By that time Android had some rudimentary support for unit testing using JUnit (https://junit.org/junit5/), but it was not fully supported and even less documented.

In the process of writing my own library and tools, I discovered Phil Smith’s Positron , an Open Source library and a very suitable alternative to support testing on Android, so I decided to extend his excellent work and bring some new and missing pieces to the table.

Some aspects of test automation were not included and I started a complementary project to fill that gap, it was consequently named Electron. And although positron is the anti-particle of the electron, and they annihilate if collide, take for granted that that was not the idea, but more the conservation of energy and the generation of some visible light and waves.

Later on, Electron entered the first Android Development Challenge (ADC1) in early 2008 and though it obtained a rather good score in some categories, frameworks had no place in that competition. Should you be interested in the origin of testing on Android, please find some articles and videos that were published in my personal blog (http://dtmilano.blogspot.co.uk/search/label/electron).

By that time Unit Tests could be run on Eclipse. However, testing was not done on the real target but on a JVM on the local development computer.

Google also provided application instrumentation code through the Instrumentation class. When running an application with instrumentation turned on, this class is instantiated for you before any of the application code, allowing you to monitor all of the interaction the system has with the application. An Instrumentation implementation is described to the system through an AndroidManifest.xml file.

Software bugs

It doesn’t matter how hard you try and how much time you invest in design and even how careful you are when programming, mistakes are inevitable and bugs will appear.

Bugs and software development are intimately related. However, the term bugs to describe flaws, mistakes, or errors has been used in hardware engineering many decades before even computers were invented. Notwithstanding the story about the term bug coined by Mark II operators at Harvard University, Thomas Edison wrote this in 1878 in a letter to Puskás Tivadar showing the early adoption of the term:

“It has been just so in all of my inventions. The first step is an intuition, and comes with a burst, then difficulties arise — this thing gives out and [it is] then that ‘Bugs’ — as such little faults and difficulties are called — show themselves and months of intense watching, study and labor are requisite before commercial success or failure is certainly reached.”

How bugs severely affect your projects

Bugs affect many aspects of your software development project and it is clearly understood that the sooner in the process you find and squash them, the better. It doesn’t matter if you are developing a simple application to publish on the Android Market, you are re-branding the Android experience for an operator, or creating a customized version of Android for a device manufacturer, bugs will delay your shipment and will cost you money.

From all of the software development methodologies and techniques, Test Driven Development, an agile component of the software development process, is likely the one that forces you to face your bugs earlier in the development process and thus it is also likely that you will solve more problems up front.

Furthermore, the increase in productivity can be clearly appreciated in a project where a software development team uses this technique versus one that is, in the best of the cases, writing tests at the end of the development cycle. If you have been involved in software development for the mobile industry, you will have reasons to believe that with all the rush this stage never occurs. It’s funny because usually, this rush is to solve problems that could have been avoided.

In a study conducted by the National Institute of Standards and Technology (USA) in 2002, it was reported that software bugs cost the country economy $59.5 billion annually. More than a third of this cost can be avoided if better software testing is performed.

But please, don’t misunderstand this message. There are no silver bullets in software development and what will lead you to an increase in productivity and manageability of your project is the discipline applying these methodologies and techniques to stay in control.

Why, what, how, and when to test

You should understand that early bug detection saves huge amount of project resources and reduces software maintenance costs. This is the best known reason to write software tests for your development project. Increased productivity will soon be evident.

Additionally, writing the tests will give you a deeper understanding of the requirements and the problem to be solved. You will not be able to write tests for a piece of software you don’t understand.

This is also the reason behind the approach of writing tests to clearly understand legacy or third party code and having the infrastructure to confidently change or update it.

The more the code is covered by your tests, the higher could be your expectations of discovering the hidden bugs.

If during this coverage analysis you find that some areas of your code are not exercised, additional tests should be added to cover this code as well.

This technique requires a special instrumented Android build to collect probe data and must be disabled for any release code because the impact on performance could severely affect application behavior.

To fill in this gap, enter EMMA (http://emma.sourceforge.net/), an open-source toolkit for measuring and reporting Java code coverage, that can offline instrument classes for coverage. It supports various coverage types:

  • class
  • method
  • line
  • basic block

Coverage reports can also be obtained in different output formats. EMMA is supported up to some degree by the Android framework and it is possible to build an EMMA instrumented version of Android.

This screenshot shows how an EMMA code coverage report is displayed in the Eclipse editor, showing green lines when the code has been tested, provided the corresponding plugin is installed.

(Move the mouse over the image to enlarge it.)

Unfortunately, the plugin doesn’t support Android tests yet, so right now you can use it for your JUnit tests only. Android coverage analysis report is only available through HTML.

Tests should be automated and you should run some or all tests every time you introduce a change or addition to your code in order to ensure that all the conditions that were met before are still met and that the new code satisfies the tests as expected.

This leads us to the introduction of Continuous Integration. It relies on the automation of tests and building processes.

If you don’t use automated testing, it is practically impossible to adopt Continuous Integration as part of the development process and it is very difficult to ensure that changes would not break existing code.

What to test

Strictly speaking you should test every statement in your code but this also depends on different criteria and can be reduced to test the path of execution or just some methods. Usually there’s no need to test something that can’t be broken, for example it usually makes no sense to test getters and setters as you probably won’t be testing the Java compiler on your own code and the compiler would have already performed its tests.

In addition to the functional areas you should test, there are some specific areas of Android applications that you should consider. We will be looking at these in the following sections.

Activity lifecycle events

You should test that your activities handle lifecycle events correctly.

If your activity should save its state during onPause() or onDestroy() events and later be able to restore it in onCreate(Bundle savedInstanceState), you should be able to reproduce and test all these conditions and verify that the state was correctly saved and restored.

Configuration-changed events should also be tested as some of these events cause the current Activity to be recreated, and you should test correct handling of the event and the newly created Activity preserves the previous state. Configuration changes are triggered even by rotation events, so you should test you application’s ability to handle these situations.

Database and filesystem operations

Database and filesystem operations should be tested to ensure that they are handled correctly. These operations should be tested in isolation at the lower system level, at a higher level through ContentProviders, or from the application itself.

To test these components in isolation, Android provides some mock objects in the android.test.mock package.

Physical characteristics of the device

Much before delivering your application you should be sure that all of the different devices it can be run on are supported or at less you should detect the situation and take pertinent measures.

Among other characteristics of the devices, you may find that you should test:

  • Network capabilities
  • Screen densities
  • Screen resolutions
  • Screen sizes
  • Availability of sensors
  • Keyboard and other input devices
  • GPS
  • External storage

In this respect Android Virtual Devices play an important role because it is practically impossible to have access to all of the devices with all of the possible combinations of features but you can configure AVD for almost every situation. However, as it was mentioned before, leave your final tests for actual devices where the real users will run the application to understand its behavior.

LEAVE A REPLY

Please enter your comment!
Please enter your name here