(For more resources related to this topic, see here.)

Getting ready

You need Ubuntu 10.04 LTS or later (Mac OS X is also supported by the build system, but we will be using Ubuntu for this article). This is the supported build operating system, and the one for which you will get the most help from the online community. In my examples, I use Ubuntu 11.04, which is also reasonably well supported. You need approximately 6 GB of free space for the Android code files. For a complete build, you need 25 GB of free space. If you are using Linux in a virtual machine, make sure the RAM or the swap size is at least 16 GB, and you have 30 GB of disk space to complete the build.

As of Android Versions 2.3 (Gingerbread) and later, building the system is only possible on 64-bit computers. Using 32-bit machines is still possible if you work with Froyo (Android 2.2). However, you can still build later versions on a 32-bit computer using a few “hacks” on the build scripts that I will describe later.

The following steps outline the process needed to set up a build environment and compile the Android framework and kernel:

• Setting up a build environment
• Downloading the Android framework sources
• Building the Android framework
• Building a custom kernel

In general, your (Ubuntu Linux) build computer needs the following:

• Git 1.7 or newer (GIT is a source code management tool), JDK 6 to build Gingerbread and later versions, or JDK 5 to build Froyo and older versions
• Python 2.5 – 2.7
• GNU Make 3.81 – 3.82

How to do it…

We will first set up the build environment with the help of the following steps:

All of the following steps are targeted towards 64-bit Ubuntu.

1. Install the required JDK by executing the following command:

   JDK6
   sudo add-apt-repository "deb http: //archive.canonical.com/ lucid 
   partner" 
   sudo apt-get update 
   sudo apt-get install sun-java6-jdk
   JDK5
   sudo add-apt-repository "deb http: //archive.ubuntu.com/ubuntu 
   hardy main multiverse" 
   sudo add-apt-repository "deb http: //archive.ubuntu.com/ubuntu 
   hardy-updates main multiverse" 
   sudo apt-get update 
   sudo apt-get install sun-java5-jdk

2. Install the required library dependencies:

   sudo apt-get install git-core gnupg flex bison gperf build-
   essential  
     zip curl zlib1g-dev libc6-dev lib32ncurses5-dev ia32-libs  
     x11proto-core-dev libx11-dev lib32readline5-dev lib32z-dev  
     libgl1-mesa-dev g++-multilib mingw32 tofrodos python-markdown  
     libxml2-utils xsltproc

3. [OPTIONAL]. On Ubuntu 10.10, a symlink is not created between libGL.so.1 and libGL.so, which sometimes causes the build process to fail:

   sudo ln -s /usr/lib32/mesa/libGL.so.1 /usr/lib32/mesa/libGL.so

4. [OPTIONAL] On Ubuntu 11.10, an extra dependency is

   sudo apt-get install libx11-dev:i386

5. Now, we will download the Android sources from Google’s repository.
6. Install repo. Make sure you have a /bin directory and that it exists in your PATH variable:

   mkdir ~/bin 
   PATH=~/bin:$PATH 
   curl https: //dl-ssl.google.com/dl/googlesource/git-repo/repo > ~/
   bin/repo 
   chmod a+x ~/bin/repo

   Repo is a python script used to download the Android sources, among other tasks. It is designed to work on top of GIT.

7. Initialize repo.
8. In this step, you need to decide the branch of the Android source you wish to download. If you wish to make use of Gerrit, which is the source code reviewing tool used, make sure you have a live Google mail address. You will be prompted to use this e-mail address when repo initializes.
9. Create a working directory on your local machine. We will call this

   mkdir android_src
   cd android_src

10. The following command will initialize repo to download the “master” branch:

    repo init -u https://android.googlesource.com/platform/manifest

11. The following command will initialize repo to download the Gingerbread 2.3.4 branch:

    repo init -u https: //android.googlesource.com/platform/manifest 
    -b android-2.3.4_r1 

    The -b switch is used to specify the branch you wish to download.

12. Once repo is configured, we are ready to obtain the source files. The format of the command is as follows:

    repo sync -jX

    -jX is optional, and is used for parallel fetch.

13. The following command will sync all the necessary source files for the Android framework. Note that these steps are only to download the Android framework files.Kernel download is a separate process.

    repo sync -j16 

    The source code access is anonymous, that is, you do not need to be registered with Google to be able to download the source code. The servers allocate a fixed quota to each IP address that accesses the source code. This is to protect the servers against excessive download traffic. If you happen to be behind a NAT and share an IP address with others, who also wish to download the code, you may encounter error messages from the source code servers warning about excessive usage. In this case, you can solve the problem with authenticated access. In this method, you get a separate quota based on your user ID, generated by the password generator system. The password generator and associated instructions are available at https://android.googlesource.com/new-password.

14. Once you have obtained a user ID/password and set up your system appropriately, you can force authentication by using the following command:

    repo init -u https://android.googlesource.com/a/platform/manifest

    Notice the /a/ in the URI. This indicates authenticated access.

    Proxy issues

    If you are downloading from behind a proxy, set the following environment variables:

    export HTTP_PROXY=http://<proxy_user_id>:<proxy_
    password>@<proxy_server>:<proxy_port>
    export HTTPS_PROXY=http://<proxy_user_
    id>:<proxy_password>@<proxy_server>:<proxy_port>

Next, we describe the steps needed to build the Android framework sources:

1. Initialize the terminal environment.
2. Certain build-time tools need to be included in your current terminal environment. So, navigate to your source directory:

   cd android_src/
   source build/envsetup.sh

   The sources can be built for various targets. Each target descriptor has the BUILD-BUILDTYPE format:

   • BUILD: Refers to a specific combination of the source code for a certain device. For example, full_maguro targets Galaxy Nexus or generic targets the emulator.
   • BUILDTYPE: This can be one of the following three values:
     • user: Suitable for production builds
     • userdebug: Similar to user, with with root access in ADB for easier debugging
     • eng: Development build only
3. We will be building for the emulator in our current example. Issue the following command to do so:

   lunch full-eng

   To actually build the code, we will use make. The format is as follows:

   make -jX

   Where X indicates the number of parallel builds. The usual rule is: X is the number of CPU cores + 2.

   This is an experimental formula, and the reader should feel free to test it with different values.

4. To build the code:

   make -j6

   Now, we must wait till the build is complete. Depending on your system’s specifications, this can take anywhere between 20 minutes and 1 hour. At the end of a successful build, the output looks similar to the following (note that this may vary depending on your target):

   ...
   target Dex: SystemUI 
   Copying: out/target/common/obj/APPS/SystemUI_intermediates/
   noproguard.classes.dex 
   target Package: SystemUI (out/target/product/generic/obj/APPS/
   SystemUI_intermediates/package.apk) 
    'out/target/common/obj/APPS/SystemUI_intermediates//classes.dex' 
   as 'classes.dex'... 
   Install: out/target/product/generic/system/app/SystemUI.apk 
   Finding NOTICE files: out/target/product/generic/obj/NOTICE_FILES/
   hash-timestamp 
   Combining NOTICE files: out/target/product/generic/obj/NOTICE.html 
   Target system fs image: out/target/product/generic/obj/PACKAGING/
   systemimage_intermediates/system.img 
   Install system fs image: out/target/product/generic/system.img 
   Installed file list: out/target/product/generic/installed-files.
   txt 
   DroidDoc took 440 sec. to write docs to out/target/common/docs/
   doc-comment-check 

   A better check for a successful build is to examine the newly created files inside the following directory.

   The build produces a few main files inside android_src/out/target/product/<DEVICE>/, which are as follows:

   • system.img: The system image file
   • boot.img: Contains the kernel
   • recovery.img: Contains code for recovery partition of the device

   In the case of an emulator build, the preceding files will appear at android_src/out/target/product/generic/.

5. Now, we can test our build simply by issuing the emulator command:

   emulator

   This launches an Android emulator, as shown in the following screenshot, running the code we’ve just built:

   

   The code we’ve downloaded contains prebuilt Linux kernels for each supported target. If you only wish to change the framework files, you can use the prebuilt kernels, which are automatically included in the build images. If you are making specific changes to the kernel, you will have to obtain a specific kernel and build it separately (shown here), which is explained later:

   Faster Builds – CCACHE

   The framework code contains C language and Java code. The majority of the C language code exists as shared objects that are built during the build process. If you issue the make clean command, which deletes all the built code (simply deleting the build output directory has the same effect as well) and then rebuild, it will take a significant amount of time. If no changes were made to these shared libraries, the build time can be sped up with CCACHE, which is a compiler cache.

6. In the root of the source directory android_src/, use the following command:

   export USE_CCACHE=1
   export CCACHE_DIR=<PATH_TO_YOUR_CCACHE_DIR>

   To set a cache size:

   prebuilt/linux-x86/ccache/ccache -M 50G

   This reserves a cache size of 50 GB.

   To watch how the cache is used during the build process, use the following command (navigate to your source directory in another terminal):

   watch -n1 -d prebuilt/linux-x86/ccache/ccache -s

   In this part, we will obtain the sources and build the goldfish emulator kernel. Building kernels for devices is done in a similar way.

   goldfish is the name of the kernel modified for the Android QEMU-based emulator.

7. Get the kernel sources:

   Create a subdirectory of android_src:

   mkdir kernel_code
   cd kernel_code
   git clone https: //android.googlesource.com/kernel/goldfish.git
   git branch -r

   This will clone goldfish.git into a folder named goldfish (created automatically) and then list the remote branches available. The output should look like the following (this is seen after the execution of the git branch):

   origin/HEAD -> origin/master 
     origin/android-goldfish-2.6.29 
     origin/linux-goldfish-3.0-wip 
     origin/master

   Here, in the following command, we notice origin/android-goldfish-2.6.29, which is the kernel we wish to obtain:

   cd goldfish
   git checkout --track -b android-goldfish-2.6.29 origin/android-
   goldfish-2.6.29

   This will obtain the kernel code:

   1. Set up the build environment.
   2. We need to initialize the terminal environment by updating the system PATH variable to point to a cross compiler which will be used to compile the Linux kernel. This cross compiler is already available as a prebuilt binary distributed with the Android framework sources:

      export PATH=<PATH_TO_YOUR_ANDROID_SRC_DIR>/prebuilt/
      linux-x86/toolchain/arm-eabi-4.4.3/bin:$PATH

   3. Run an emulator (you may choose to run the emulator with the system image that we just built earlier. We need this to obtain the kernel configuration file. Instead of manually configuring it, we choose to pull the config file of a running kernel.

      Make sure ADB is still in your path. It will be in your PATH variable if you haven’t closed the terminal window since building the framework code, otherwise execute the following steps sequentially.

      (Note that you have to change directory to ANDROID_SRC to execute the following command).

      source build/envsetup.sh
      lunch full_eng
      adb pull /proc/config.gz
      gunzip config.gz 
      cp config .config
      

      The preceding command will copy the config file of the running kernel into our kernel build tree.

   4. Start the compilation process:

      export ARCH=arm
      export SUBARCH=arm 
      make

      If the following comes up:

      Misc devices (MISC_DEVICES) [Y/n/?] y 
        Android pmem allocator (ANDROID_PMEM) [Y/n] y 
        Enclosure Services (ENCLOSURE_SERVICES) [N/y/?] n 
        Kernel Debugger Core (KERNEL_DEBUGGER_CORE) [N/y/?] n 
        UID based statistics tracking exported to /proc/uid_stat 
      (UID_STAT) [N/y] n 
      Virtual Device for QEMU tracing (QEMU_TRACE) [Y/n/?] y 
      Virtual Device for QEMU pipes (QEMU_PIPE) [N/y/?] (NEW)

      Enter y as the answer. This is some additional Android-specific configuration needed for the build.

   5. Now we have to wait till the build is complete. The final lines of the build output should look like the following (note that this can change depending on your target):

      ...
          LD      vmlinux 
        SYSMAP  System.map 
        SYSMAP  .tmp_System.map 
        OBJCOPY arch/arm/boot/Image 
        Kernel: arch/arm/boot/Image is ready 
        AS      arch/arm/boot/compressed/head.o 
        GZIP    arch/arm/boot/compressed/piggy.gz 
        AS      arch/arm/boot/compressed/piggy.o 
        CC      arch/arm/boot/compressed/misc.o 
        LD      arch/arm/boot/compressed/vmlinux 
        OBJCOPY arch/arm/boot/zImage 
        Kernel: arch/arm/boot/zImage is ready

      As the last line states, the new zImage is available inside arch/arm/ boot/.

   6. To test it, we boot the emulator with this newly built image.
   7. Copy zImage to an appropriate directory. I just copied it to android_src/:

      emulator -kernel zImage

   8. To verify that the emulator is indeed running our kernel, use the following command:

      adb shell 
      # cat /proc/version 

      The output will look like:

      Linux version 2.6.29-gef9c64a (earlence@earlence-
      Satellite-L650) (gcc version 4.4.3 (GCC) ) #1 Mon Jun 4 
      16:35:00 CEST 2012

      This is our custom kernel, since we observe the custom build string (earlence@earlence-Satellite-L650) present as well as the time of the compilation. The build string will be the name of your computer.

   9. Once the emulator has booted up, you will see a window similar to the following: