Distcc Cross-Compiling

Introduction
This is the official cross-compiling method used at Arch Linux ARM. If you plan on building a lot of packages and want to speed up the process, the following guide will turn an Arch Linux or Ubuntu/Debian Linux box into an ARM cross-compiler. It's also much easier than most cross-compile setups.

This guide makes use of distcc in order to not have to build a full development environment. As the distcc project website states, "distcc does not require all machines to share a filesystem, have synchronized clocks, or to have the same libraries or header files installed." This is particularly advantageous to us since all that is needed is a working cross-compiler for ARM on a faster machine, while controlling the build from a plug computer that has all of the current libraries and headers.

Distcc
Before beginning, a distcc environment needs to be set up. Follow the Distributed Compiling with distcc guide to establish a master plug. The x86/x86_64 machines will be known as the clients.

Ubuntu-specific client notes
If you're planning to use Ubuntu or Debian as a client, note the following. The guide below uses Arch Linux directories and file names.

  • "/etc/conf.d/distccd" is "/etc/default/distcc"
  • /etc/hosts.allow entry is not required
  • Start distcc with "/etc/init.d/distcc start"
  • Run the following to install everything you need:
    sudo apt-get install texinfo libncurses5-dev automake gawk build-essential bison flex libtool cvs curl

Install crosstool-ng
This process is very automated, courtesy of crosstool-ng. As a normal user (not root!), download the latest version to a directory called "cross" in your home directory and extract it. Enter the source directory and configure with a prefix for the "cross" directory, make, and make install:
mkdir -p cross/src cd cross wget http://crosstool-ng.org/download/crosstool-ng/crosstool-ng-1.13.0.tar.bz2 tar -xjf crosstool-ng-1.13.0.tar.bz2 cd crosstool-ng-1.13.0/ ./configure --prefix=/home/your_user/cross make make install

At this point crosstool-ng is ready to be configured. "ct-ng" in the "bin" directory is where the magic happens. It also has a menu configuration like the Linux kernel.

Downloading a CrossTool Configuration
Download the default .config file to place in "~/cross/bin" as shown below. Once you do this, do not run "menuconfig" or values will be overwritten. Choose either "v5" or "v7" depending on the target platform.

cd /home/your_user/cross/bin/ wget http://archlinuxarm.org/mirror/development/ct-ng/xtools-dotconfig-[v5|v7] mv xtools-dotconfig-[v5|v7] .config ./ct-ng build

Make nice with distcc
The toolchain will install under "~/x-tools" for armv5 and "~/x-tools7h" for armv7 (hard-float). You can move this somewhere else if you like, or leave it where it is. Before we can use the compiler binaries that were created, links need to be created to make their names more appropriate. When compile jobs are sent to distcc, the program specified in the CC environment variable on the build master is what gets executed on the build clients. All the binaries that have been produced by crosstool-ng are in the (correct) format specifying the target platform as a prefix. This script will fix our problem. Make "~/x-tools/arm-unknown-linux-gnueabi/bin/" writable and in there create a file called "link" and paste this into it. Make it executable ("chmod +x") and run it afterward.

#!/bin/bash for file in `ls`; do if [[ "$file" == "link" ]]; then continue fi ln -s $file ${file#arm-unknown-linux-gnueabi-} done

Now the "bin" directory contains links with names that distcc will play nice with. To get distcc to use these binaries instead of the default system ones, we need to place this directory into the path for the distcc daemon:

  • Debian/Ubuntu: Edit "/etc/init.d/distcc"
  • Arch Linux: Edit "/etc/rc.d/distccd"

After the initial header block, add this line or modify PATH if it already exists in the file. Note that we are placing our binary directory at the very front.
PATH=/home/your_user/x-tools/arm-unknown-linux-gnueabi/bin:$PATH

At this point distcc can be started, or restarted if it's already running:

  • Debian/Ubuntu: /etc/init.d/distcc start|restart
  • Arch Linux: /etc/rc.d/distccd start|restart

Compile!
Back on ARM "master" device, make sure that distcc has been enabled in makepkg.conf per the distcc guide, and specify the cross-compiler computer's IP address in the DISTCC_HOSTS variable. Build packages like you normally would, in a fraction of the time!