The Aliens Have Landed!

27 min read

(For more resources related to this topic, see here.)

The progression of testing

Way back when testing used to be primarily manual, test cases were created and executed by developers or quality assurance team members. These test cases would comprise of anything from simple unit tests (testing single methods or classes of code) or integration tests (testing multiple components within code) or even functional tests (tests that ensure the system performs as required). As we began to develop differently, whether it was from the agile project methodology or extreme programming methodologies, we needed more robust tools to support our needs.

This led to the advent of automated testing. Instead of a tester working with your application and running tests against it, they could simply press a few buttons, hit a few key strokes, and execute a 100 or 200 test case suite against your application to see the functionality. In some realms, something called a test harness was used. Test harnesses usually included running the compiled application in some kind of a sandbox environment that was probably something like production (this is after all the final binary that would be rolled out) that may or may not have pointed to a database (if it did, and it was smart, it probably pointed to a completely non-discreet database instance) to perform some level of testing. User input would be simulated and a report (possibly in some cryptic format that only few understood) would be generated indicating whether the application did what was expected or not.

Since then, new tools such as JUnit, Selenium, SoapUI to name a few, have been introduced to add more functionality to your test cases. These are meant to drive both unit and functional testing of your application. They are meant to be standard tools, easy to use and reuse, and overall a platform that many developers can work with, and can be a desirable skill set for employers. Standardizing of tools also allows for more integrations to occur; it may be difficult to get leverage to build an integration with your own built tools, with many developers wanting an integration with widely used frameworks A and B.

What is Arquillian

If you haven’t heard of Arquillian before (or are very new to it), this may be the section for you. Arquillian is a testing framework for Java that leverages JUnit and TestNG to execute test cases against a Java container. The Arquillian framework is broken up into three major sections: test runners (JUnit or TestNG), containers (Weld, OpenWebBeans, Tomcat, Glassfish, and so on), and test enrichers (integration of your test case into the container that your code is running in). ShrinkWrap is an external dependency for you to use with Arquillian; they are almost sibling projects. ShrinkWrap helps you define your deployments, and your descriptors to be loaded to the Java container you are testing against.

The JUnit test container is used throughout. If you’d like to use TestNG with your code, you simply need to replace the JUnit container with the TestNG container, and have your test classes extend the Arquillian class found there. The JUnit test cases use a JUnit Runner to start Arquillian. The containers used will vary with each case.

The Arquillian difference

Arquillian can be considered a standardized test harness for JVM-based applications. It abstracts the container or application start-up logic away from your unit tests and instead drives a deployment runtime paradigm with your application, allowing you to deploy your program, both via command line and to a Java EE application server.

Arquillian allows you to deploy your application to your targeted runtime to execute test cases. Your targeted runtime can be an embedded application server (or series of libraries), a managed application server (where Arquillian performs the calls necessary to start and stop the JVM), or even a remote application server (which can be local to your machine, remote in your corporate infrastructure, or even the cloud).

Arquillian fits in to certain areas of testing, which can vary based on testing strategies for your application. If you are using Java EE 6, you may want to use an embedded CDI container (such as Weld) to unit test parts of your application. These tests could happen hourly or every time someone commits a code change. You could also use Arquillian to automate your integration test cases, where you use a managed or embedded application server to run your application, or even just parts of your application. You can even use Arquillian to perform automated acceptance testing of your application, using other tools such as Selenium to drive requests through the user interface. This can also be used to smoke test deployments of applications.

Arquillian can be used with a wide variety of containers that support everything from JBoss 4.2 (slightly pre-Java EE 5) through Java EE 6 and can control these containers in what Arquillian considers types – embedded, managed, and remote. Embedded application servers run within the same JVM as your test cases always do. Managed run within a separate JVM and are started and stopped by Arquillian. A managed container will start on the first test that requires a deployment and stop once all tests have been executed; there is no restart. Remote containers are as the name implies, remote JVMs. This could be on the same physical hardware that your test runs on or a remote piece of hardware that the application server runs on. The application server must be running in order to deploy. Note that if there is a problem deploying, such as the managed application server will not start or the remote application server will not start, Arquillian will fail once and assume that deployments will fail afterwards for the remaining test cases:

  • Do not mix your unit test application servers that are used for automated testing and those that you use for manual testing. Whether it’s a separate instance required, a distinct domain, profile, whichever your application server vendor supports, avoid mixing them. One of your biggest blockers may be from your manually deployed application interfering with your automated testing application.
  • Even though remote application servers can be physically separated from your testing, they typically require the binaries to be locally available. Plan to have a copy of your application server available on your CI server(s) for this purpose.
  • Prepare your application for this kind of testing. Whether it’s the automatic deployment or undeployment of resources (JMS queues, JDBC connections, users, and so on) or ensuring that the server is up and running (for example, prebuild, kick off a kill, and restart process) make sure this can all happen from your build, either in your CI server or using scripts within your source repository.
  • Do not try to reuse application servers across applications. If two test cases are running in parallel, you can run into inconsistent results.

The fundamentals of a test case

As our software has evolved, our strategy for testing it must evolve as well. We have become more dependent on techniques such as dependency injection (or inversion of control – IoC). When we take this in to consideration, we realize that our testing has to change. Take a look at the following example:

public void testCalculationOfBusinessData() {
CalculatorData cd = new CalculatorData(1, 3, 5);
CalculatorService ms = new CalculatorServiceImpl();
assertEquals(1 + 3 + 5, cd.getResult());

We can assume that the method calculateSum takes the int values passed in to MyDataObject and sums up the values. As a result, when I construct it using 1, 3, 5 the total should come out to 9. This is a valid test case for our service layer, since our service layer knows what implementations exist out there and how they should be tested. If there was another implementation of CalculatorService that multiplied all results by 2, a separate test case or test class would exist which tested that object.

Let’s say we look at the business layer that invokes this service object:

public class CalculatorController {
private CalculatorService service;
private CalculatorForm form;
/**For the injected form, calculates the total of the input**/
public void sum() {
CalculatorData data = new CalculatorData(form.getX(),form.

This example uses JSR-330 annotations to inject references to CalculatorService, a service layer object that can perform basic calculator functions and CalculatorForm, some sort of UI component that has form input and output that can be read or returned. If we want to test this class, we will immediately run into a problem. Any invocation of the sum method outside of a JSR-330 (dependency injection for Java) container will result in a NullPointerException.

So what does Arquillian do to make our test case more legitimate? Let’s take a look at the test case and review the anatomy to understand that better:

public class CalculatorTest {
public static JavaArchive createArchive() {
return ShrinkWrap.create(JavaArchive.class,"foo.jar")
@Inject CalculatorForm form;
@Inject CalculatorController controller;
public void testInjectedCalculator() {

There are a few pieces that make up this test case, each of which we’ll need to review. These are given as follows:

  • The @RunWith annotation: It tells JUnit to use the Arquillian runner for running this class.
  • The @Deployment annotation: It tells Arquillian to use the specified archive for deployment purposes and testing purposes.
  • The injection points: In this case, CDI injection points represent the objects under test.
  • The Test: This is where we process the actual test case. Using the injected objects, we simulate form input by inserting values in X, Y, and Z in the form, then invoke the controller’s sum method, which would be called from your user interface. We then validate that the resulting sum matches our expectations.

What we gained was leveraging Arquillian to perform the same IoC injection that we would expect to see in our application. In addition, we have the following dependencies within our Maven pom file:

<dependencyManagement> <dependencies> <dependency> <groupId>org.jboss.shrinkwrap.resolver</groupId> <artifactId>shrinkwrap-resolver-bom</artifactId> <version>2.0.0-alpha-1</version> <scope>import</scope> <type>pom</type> </dependency> <dependency> <groupId>org.jboss.arquillian</groupId> <artifactId>arquillian-bom</artifactId> <version>${}</version> <scope>import</scope> <type>pom</type> </dependency> </dependencies> </dependencyManagement> <dependencies> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.10</version> <scope>test</scope> </dependency> <dependency> <groupId>org.jboss.spec</groupId> <artifactId>jboss-javaee-6.0</artifactId> <version>1.0.0.Final</version> <type>pom</type> <scope>provided</scope> </dependency> <dependency> <groupId>org.jboss.shrinkwrap.resolver</groupId> <artifactId>shrinkwrap-resolver-impl-maven</artifactId> <scope>test</scope> </dependency> <dependency> <groupId>org.jboss.arquillian.junit</groupId> <artifactId>arquillian-junit-container</artifactId> <scope>test</scope> </dependency> <dependency> <groupId>org.jboss.arquillian.container</groupId> <artifactId>arquillian-weld-ee-embedded-1.1</artifactId> <version>1.0.0.CR3</version> <scope>test</scope> </dependency> <dependency> <groupId>org.jboss.weld</groupId> <artifactId>weld-core</artifactId> <version>1.1.8.Final</version> <scope>test</scope> </dependency> <dependency> <groupId>org.slf4j</groupId> <artifactId>slf4j-simple</artifactId> <version>1.6.4</version> <scope>test</scope> </dependency> </dependencies>

Our dependencies are as follows:

  • Arquillian BOM: It’s an overall list of all artifacts for running Arquillian
  • ShrinkWrap: It’s a separate API for creating deployments
  • JUnit and Arquillian JUnit Container: Arquillian support for working with JUnit
  • Arquillian Weld EE: Arquillian containers control what your deployment runtime should be

Testing profiles

One of the benefits of using Arquillian with Maven is that you can control how your tests execute using Maven profiles. With this approach, you can define a profile per container to test against. In some cases, you may want to test your application or library against multiple containers. This allows you to reuse deployments against different libraries. We still remain with the core limitation: you can only have one container on your classpath at a time. Let’s suppose we take this same application, but want to run it against Apache OpenWebBeans as well as Weld.

Each Arquillian container has a default Maven profile configuration that can be used for testing. These containers are covered within the Arquillian documentation, found at

Steps to try out containers are given as follows:

  1. Import the configuration defined by the container definition.
  2. Run a mvn clean install Pconfiguration_name.
  3. You can choose to set a default profile as well if you like. If you don’t choose a default profile, you will need to specify one every time you want to run tests.

Running the tests for this project, for both the weld-ee-embedded-1.1 container and the openwebbeans-embedded-1 profile should result in the same thing – a working test suite that is valid in both implementations.

At the time of writing, I used Weld 1.1.8.Final and OpenWebBeans 1.1.3.

It is important to point out at this time that these profile names are only useful if your application is designed purely for cross-platform testing and you want to run all test cases against each individual platform. If your application only targets a single platform, you may want to derive test cases that run on that platform as well as any subcomponents of that platform (for example, if you are a WebSphere v8 user, you may want your unit tests against OpenWebBeans and integration against WebSphere; however, if you are a WebLogic user, you would want to use Weld and WebLogic for your testing).

Typically, when it comes to testing, you will use a distinct Maven profile to cover your stages of testing. You should set up a default Maven profile that runs only your basic tests (your unit tests); this will be set as activeByDefault. This should include any testing dependencies needed to run only these unit tests. You may optionally choose to only run certain parts of your test suite, which could be distinct packages under src/test/java or even standalone projects that are children to your parent that are only run under certain circumstances. I prefer the former approach, since the usage of conditional child projects can become confusing for developers.

Profiles are useful for conditional dependencies, since they do include a dependency and dependencyManagement section in their pom files. You can also avoid dependency leaking. For example, most applications require the use of the full Java EE 6 APIs, but including these APIs with your Glassfish build will cause your tests to fail. Likewise, deploying to a managed application server may require different APIs than deploying to a remote application server.

Categorizing your test cases

One thing that Arquillian ultimately derives is that names mean everything. There are two naming schemes that you should follow always, they relate to the questions “what” and “how”. What components are you testing? What phase of testing are you under? How does this class relate to your test cases? How is this code being deployed?

These questions really pop up due to the nature of Arquillian, and really show off its robustness. Considering some of the main testing phases, unit test, integration test, system test, compatibility test, smoke test, acceptance test, performance test, usability test should all relate to specific packages in your test code. Here’s a proposed naming scheme. I will assume that your code starts with com.mycompany.fooapp where com.mycompany is your company’s domain, fooapp is the name of your application. You may have packages below this such as com.mycompany.fooapp.model or com.mycompany.fooapp.controller or even com.mycompany.fooapp.modulea.controller all of which represent varying testing scenarios that you may consider.

  • com.mycompany.fooapp.test: This is the parent package for all test classes. There may or may not be any classes in this package.
  • com.mycompany.fooapp.test.utils: This is where all of your test utility code goes. This would be where any deployments are actually build, but invoked from your test classes.
  • com.mycompany.fooapp.test.unit: This is where all unit tests should exist. The packages/classes under test should fall relative under here. For example, com.mycompany.fooapp.test.unit.controller should test your controller logic.
  • com.mycompany.fooapp.test.integration: Likewise, all integration test cases should fall under here.

Following this pattern, you can derive your functional, acceptance, usability, and so on test case names.

Following this pattern, you can easily define Maven profiles within your projects to test out your various layers. Let’s suppose you want to define a unittest profile where the build includes running all unit tests (usually light weight tests, that maybe use an embedded container with Arquillian), you could do something like this:


This will tell Maven to only run the tests in the unit test packages by default. This allows your builds to go by quickly and test key components of your code when needed. Since it’s active by default, you would get these tests run anytime you kick off mvn install or mvn test from the command line, or your continuous integration server.

Likewise, you may want to run more tests during your integration testing or system testing phases. These may or may not overlap with one another, but would likely include your unit tests as well. You could use the following, very similar Maven profile to achieve that:


With the way this profile is configured, any time you invoke mvn test – integrationtest you will run all unit, integration, system, and regression test cases. These tests will probably take a while. If you’re lucky and your application server supports an embedded mode it probably has a quick start up/shutdown. These tests may be running against a remote container though, so the deployment and execution time may take longer.

Enriching your tests

One of the core principles noted in the demo test case is that we injected a reference to the business object being tested, which itself had injection points as well that were satisfied. There are three core injection points supported by default with Arquillian, though others are supported by extensions as well as some custom injection points defined by Arquillian itself. They are as follows:

  • @Resource: It defines references to any JNDI entry as per standard naming and injection schemes.
  • @EJB: As long as you are including a deployed EJB in your archive, you can refer to it as long as your target container includes EJB support. EJB JNDI entry conventions remain intact. It is also worthwhile to note that Local and Remote interfaces matter. When you deploy to a same JVM application server, or a remote application server you must ensure that your injected test cases can see the EJB reference appropriately.
  • @Inject: This is available as long as you have deployed a valid bean archive (requires beans.xml in META-INF or WEB-INF).

Running out of the container

Sometimes you will need to mix your Arquillian tests with non Arquillian tests, or sometimes you have special test cases that require a deployment but should not be run with that deployment (simulating a remote client to that deployment, for example). This may include a set of unit tests that have deployments to the container and some that do not require this. You can run the test outside of the container JVM by using the @RunAsClient annotation. This can be done at the class level or at the method level, allowing you to have multiple run-types within a single test class.

This approach would be useful if you want to test a web service client, either SOAP or a REST API, ensuring that you are executing outside of the container JVM for your client. One of the custom injection points that Arquillian supports is a @ArquillianResource URL baseUrl; this represents the base URL of a remote container deployment that you can use in your client-level tests. The ArquillianResource annotation supports an additional value attribute that can be the class of a Servlet deployed in case you have multiple deployments occurring as a part of your test. Once you have the URL for your application deployment, you can build the location of your web services to be able to use your client application for testing purposes. This would also be used if you wanted to functionally test your application via HTTP, perhaps using Selenium as the driver for the application.

The other types of objects available for injection using this annotation are the Deployer used and the InitialContext of the remote application server. The deployer gives you access to the underlying deployment while InitialContext would allow you to look up remote objects – for example, EJBs, JMS Queues/Topics/ConnectionFactories, or any other remotely accessible resource.

Efficient test authoring

Now that we have a test that can run against multiple containers, we need to start adding more test cases to our suite. When we do this, we have to keep two key elements in mind. Don’t repeat yourself, and don’t bloat your software.

In many of our applications, we have a number of components that make up various layers. Some of them are dependent on one another, others are more generic. One of the key things to think about when planning your Arquillian testing is how should your JAR files look. Let’s suppose we have an entire data object layer that has dependencies all throughout your application. We must have those classes in every test case. However, we can usually skip to only certain controllers and business beans within their specific test cases. Remember to create utilities to define your object structure, this gives you a single entry point for creating your deployment archive and allows for better extensibility. Here is a prototype class that can be used to start, it supports creating both a JAR file as well as a full web application:

public class TestUtils {
public static JavaArchive createBasicJar() {
return ShrinkWrap.create(JavaArchive.class,"test.jar")
public static WebArchive createWebApp() {
return ShrinkWrap.create(WebArchive.class,"test.war")
public static Package[] getCorePackages() {
return new Package[]{CalculatorData.class.getPackage()};

What these methods add is a significant reduction in code that is impacted by a package change, or code refactoring in your primary code base. Let’s suppose that we want to extend getCorePackages to also take in a number of classes, which when added, add the entire package to the deployment:

public static Package[] getCorePackages(Class<?>...classes) {
List<Package> packages = new ArrayList<Package>();
if(classes != null) {
for(Class<?> c : classes) {
return packages.toArray(new Package[]{});

One benefit that we have using this approach is that anyone who was using getCorePackages() does not need to change their code, since the argument is not required.

Note that the ShrinkWrap API has several addPackage/addPackages methods. This one used has a first argument, Boolean, whether to re-curse through child packages to find your code. Going back a few pages to some naming conventions I proposed, what would happen if you add the package com.mycompany.fooapp to your bundle? All of your application classes, including test classes, would be added to the deployment you are creating. This is probably not what you would have expected, as a result, the recommendation is to not re-curse into child packages and instead just list out each package you want added explicitly.

Another option to consider is to have your test classes delegate their work. Commonly thought of as the façade programming paradigm, you can actually apply this to your test code as well. If you have code that should be tested distinctly but use different deployments you may want to use distinct deployment objects but reuse your test case. This may involve using a controller type test case that simply delegates its test calls to another object that is meant to purely handle the testing logic. Your controller would have methods annotated @Test and include your @Deployment(s) but would delegate logic to another class, potentially an injected test executor class.

ShrinkWrap – building your own app

One of the more curious things about Arquillian is that your test case is responsible for constructing the application to be deployed. When you’re working with Maven it is especially odd, since all of the information to build the application is there, either implicitly based on the project structure or explicitly listed in your pom file. There are two realizations around this that are important:

  • Your test cases are meant to test anything from a subset of your code to your entire application. Arquillian is flexible enough to handle both extremes.
  • Arquillian works great with Maven, but also works with other build tools such as Ant and Gradle.

In order to support the dynamic generation of your application, the ShrinkWrap project exists to help dynamically build Java archive files. There are four primary archive types supported in ShrinkWrap: Java Archive (plain JAR files), Web Archive (WAR files), Enterprise Archive (EAR files), and Resource Adapters (RARs). Your Arquillian test case can declare any number of these archive files to be created for testing purposes.

Another place that ShrinkWrap helps with is the creation of deployment descriptors. These could be application.xml files, or persistence.xml files, or any of the standard deployment descriptors that you would use in your application. Likewise, it has extensibility built in to allow the creation of new descriptors in a programmatic fashion.

This article assumes that you are using the ShrinkWrap 2.0 APIs; one of the key features added is support for resolving dependency files from reading a Maven pom file. Another key thing you need to do is modify your test classpath to include some files from your core application. Here is an example of what to do, from another project I was working on recently:


This will make it easier to reference your main application resources without requiring full paths or file manipulation.

ShrinkWrap also provides ways to build deployment descriptors. This is a programmatic approach to adding the descriptor to your deployment, including programmatically creating the descriptor. Because of the need to test using the same descriptors being built with the production application, I have found it easier to reference to an existing descriptor. However, in some cases it may make more sense to use one customized to your test applications. In this scenario, I would strongly recommend creating a utility method to build the descriptor.

To do this, we will add the following to our Maven pom.xml in the dependencyManagement section:


Declare the following dependency:


Then add the following method to our code:

public static StringAsset createWebXml() {
return new StringAsset(
public static WebArchive createWebApp() {
return ShrinkWrap.create(WebArchive.class,"test.war")
.addAsWebInfResource(createWebXml(), "web.xml")

This will generate the web.xml file expected for your test case. One of the benefits of using the descriptors is that we can actually import the one from our application and make changes as needed.

public static StringAsset createWebXml() {
WebAppDescriptor descriptor = Descriptors.
.fromFile(new File("web/web.xml"));
return new StringAsset(descriptor.exportAsString());

As a result, this will use your base web.xml file but change the welcome file to something else. You can do this with a number of other descriptors, including persistence.xml, beans.xml, web-fragment.xml, and so on.

Getting the most from Arquillian

Arquillian does take some understanding to work with. In order to get the most from it, you have to work with it and invest time.

Your standard rules still apply. Arquillian does not use any special rules when it comes to processing deployments. The rules about deployment descriptors, archive contents, and so on and so forth still apply. You need to keep the rule of thumb – if it deploys to your application server, then you can deploy the same archive via Arquillian; just make sure that you are deploying the same archive.

Note that you can use the Archive’s toString method to print out the contents when in doubt. This supports a formatter as well, to make it easier to read the contents.

Alternatively, you can export an Archive using if you want to manually review the file.

Run as many tests as you can with Arquillian. Due to Arquillian’s nature, you’re going to start to find inconsistencies in your code if you don’t test it with Arquillian. This could include unexpected dependency injection expectations, which Arquillian will process for you. Arquillian, since it executes your code the way the application server would as requests come in, makes your tests more consistent with the real world of how the code works. Testing more in Arquillian, even if it is just using a basic CDI container or OpenEJB container, will allow you to test more effectively. You make the best use of Arquillian when you use it throughout 100 percent of your test cases.

Finally, my last key advice to getting the most from Arquillian is to remember to not over-complicate your build to make use of Arquillian. Arquillian works fine as a part of your standard build. It has no requirement to create distinct projects for each build type or make overly complicated build structures on top of your application. If you are attempting to run Arquillian with your continuous integration server, then you must ensure that different test cases run either as different steps of the build or as separate build jobs.

Arquillian extensions

There are a number of extensions available for Arquillian; they are designed to extend Arquillian functionality to do some domain-specific testing:

  • Persistence, using DBUnit and validating that results of interacting with the database.
  • REST, invoke REST APIs from Arquillian the were deployed as a part of the test case.
  • Spring, use a Spring Context and additional Spring libraries with your test cases.
  • Drone/Selenium, functionally test your web applications using Arquillian.


My main goal in this article was to give you an overview of Arquillian and introduce you to some of its core concepts.

Resources for Article:

Further resources on this subject:


Please enter your comment!
Please enter your name here