Marcelo Schmitt

Raspberry Pi kernel compilation

In this post, we shall go through most steps needed to build up Linux images for Raspberry Pi 3B. It is recommended to have an 8GB or larger micro SD card, Raspberry Pi 3B or 3B+, and other peripherals to test the generated kernel image.

Warning: This is an old post with potentially outdated information. Read with caution.

Get a Linux Tree

There are several repositories that contain the source code of the Linux kernel. These repositories are known as trees. Therefore, a Linux tree is a repo where some development for the kernel happens. A Linux tree may be widely known and downloaded (upstream), like Linus’ tree, or maybe one that sends its contributions (patches) to upstream trees.

Some examples of Linux trees are:

To get a Linux tree to start working with, you might use git to download (clone) it.

For instance, to get the IIO tree one might type:

git clone git://git.kernel.org/pub/scm/linux/kernel/git/jic23/iio.git

For the Raspberry Pi tree:

git clone git@github.com:raspberrypi/linux.git

For the Analog Devices tree:

git clone https://github.com/analogdevicesinc/linux.git

For this post, I’ll be using the Analog Devices tree. However, the steps taken here will be much similar to steps needed for working on other Linux trees.

More about Linux trees may be seen in some good articles like The linux-next and -stable trees from the LWN.net journal, and in great talks like Greg’s Linux Kernel Development at Git Merge.

A list of upstream Linux trees can be seen at https://git.kernel.org/.

Configure Compilation Symbols

The Kernel Build System (kbuild) uses the configurations symbols and respective values stored inside a file called .config. It is not recommended to edit this file manually unless you know exactly what you’re doing. The configuration options are sensible to ordering. Every entry may have its own dependencies. Disabling an option may limit the visibility of dependent entries. Nevertheless, if you are wiling to play with .config files, you may specify a different config file setting with the KCONFIG_CONFIG variable, like:

export KCONFIG_CONFIG=.my_config

It may also be set up just when asking kbuild to compile the kernel:

make KCONFIG_CONFIG=.my_config

The kernel configuration symbols are defined in Kconfig files. Most directories inside the kernel source tree have a Kconfig file that includes (sources) Kconfig files from its subdirectories. For instance, to add a configuration symbol for the AD7292 driver, the following entry would be added to the Kconfig file at drivers/iio/adc/Kconfig.

config AD7292
	tristate "Analog Devices AD7292 ADC driver"
	depends on SPI
	help
	  Say yes here to build support for Analog Devices AD7292
	  8 Channel ADC with temperature sensor.

	  To compile this driver as a module, choose M here: the
	  module will be called ad7292.

The config keyword defines a new symbol which in turn is presented as a configuration entry in the .config file, within tools like menuconfig and nconfig, or at compilation time. In particular, the AD7292 config symbol has the following attributes:

There are a few ways to prepare a kernel compilation file. It may be done manually, with the aid of tools like menuconfig, or with a defconfig file. Defconfig files are intended to store only specific non-default values (i.e options we changed for our board or project) for compilation symbols. Some defconfig files for the arm architecture may be found at arch/arm/configs/. Commonly configurations used for Rasbperry Pi boards are stored in the files bcm2709_defconfig, bcm2835_defconfig, and bcmrpi_defconfig. Also, Analog Devices Inc. has a custom defconfig file at their repository (adi_bcm2709_defconfig). To add a custom compilation value for the AD7292 symbol, we would add the following line to the adi_bcm2709_defconfig file:

CONFIG_AD7292=y

Comment: bcm comes from Broadcom, manufacturer of the processors used largely on Raspberry Pi boards. See the Raspberry Pi hardware documentation.

We may apply the configurations defined at a defconfig file passing its name as a make rule. Like this:

make adi_bcm2709_defconfig

This will set the values stored in adi_bcm2709_defconfig into the .config file to be used during kernel compilation. Also, kbuild will set default values for the symbols listed as dependencies for those in the defconfig file, so we won’t be asked for any other configuration to compile the kernel.

There is one more thing to do before compiling the kernel, though. Even though we have included a new configuration symbol for kernel compilation, we did not bound it to the compilation of the AD7292 driver, but we need to do that. The kernel image is build by the kbuild Makefiles. Like the Kconfig files, kbuild Makefiles are present in most kernel directories. From Javier Canillas’ article at the Linux Journal:

The whole build is done recursively — a top Makefile descends into its subdirectories and executes each subdirectory’s Makefile to generate the binary objects for the files in that directory. Then, these objects are used to generate the modules and the Linux kernel image.

Thus, to have a C file compiled, a build goal has to be added to a Makefile. Moreover, the convention kernel developers follow is to add build goals to the Makefile laying in the same directory as the .c file. For instance, if we wanted our AD7292 driver to be compiled, we would add the following to drivers/iio/adc/Makefile:

obj-$(CONFIG_AD7292) += ad7292.o

Canillas also made a summary of the steps needed to add new kernel functionality:

To add a feature to a Linux kernel (such as the coin driver), you need to do three things:

More about attributes of kconfig symbols, kernel Makefiles, the kbuild system as a whole, can be found at the official documentation for the Kernel Build System. Also available inside the kernel source tree at: Documentation/kbuild/.

A nice thread regarding defconfig usage and its motivation can be found at Stack Overflow.

Instructions on how to configure and build the official Raspberry Pi tree can be found at the Raspberry Pi documentation: Configuring the kernel and Kernel building.

Lastly, a comprehensive explanation of the Linux build process can be found in Javier Canillas’ article.

Kernel Cross Compilation

Different processor architectures usually have different instruction sets and register names. Due to this, using a compiler that produces code for computers which processor belongs to the x86 architecture family (assuming you are programming on such a machine), will generate binaries that won’t work on arm processors. Because of this, we need to use a cross compiler. The cross compiler will use a different compilations process to generate binaries (one that can produce binaries compatible with the arm architecture).

Surely, there are several cross compilers to arm architecture on the Internet however, I’ll be using Linaro’s one (which is a customized GCC). By the time I am writing this tutorial, one of the latest Linaro cross compiling tools is Linaro GCC 7.

After downloaded, you may extract it with tar:

tar -xf gcc-linaro-7.4.1-2019.02-x86_64_arm-linux-gnueabi.tar.xz

To indicate that we want to use a cross compiler, we set the CROSS_COMPILE variable to point to the Linaro’s GCC. For instance, if you have extracted it one directory above the kernel source tree, you might set:

export CROSS_COMPILE=../gcc-linaro-7.4.1-2019.02-x86_64_arm-linux-gnueabi/bin/arm-linux-gnueabi-

Also, as stated in the documentation:

By default, the top Makefile sets $(ARCH) to be the same as the host system architecture. For a cross build, a user may override the value of $(ARCH) on the command line: make ARCH=m68k …

So, we need to specify the target architecture we want binaries be generated for. This is done by setting the ARCH variable. In our case:

export ARCH=arm

Note: All these variables can be set just before compilation, on the command line:

make ARCH=arm export CROSS_COMPILE=../gcc-linaro-7.4.1-2019.02-x86_64_arm-linux-gnueabi/bin/arm-linux-gnueabi-

That’s a pretty long command in my opinion. This is the reason why I prefer to export the values for the variables in separate commands, instead of preparing a giant one that may be not as much readable. Though, each one decides what works best for themselves, from now on, I will assume the ARCH and CROSS_COMPILE variables are exported.

Now, when kbuild start compiling the Linux kernel files, it will use the cross compiler set to do so.

Finally, we are ready to compile the Linux kernel to be use in our Raspberry Pi. =D

Type:

make zImage modules dtbs -j4

Cross your fingers, press Enter and go fetch a mug of coffee or any drink you like. The compilation process might take some time (around half an hour in my quad core machine). If you’ve got more than 4 cores in your machine, you may put them to work increasing the number of threads used by the kbuild. This is done by setting the j flag. For instance, if you want the compilation process to be carried on with 8 threads, you may use -j8. The dtbs rule makes kbuild to compile the device tree files at arch/arm/boot/dts into device tree blobs to be used during system boot. The modules rule indicates to build code that is marked as modules (m) (duhh). The zImage refers to architecture specific kind of image we want to be generated. Different form the default vmlinux image, the zImage is a compressed kernel image.

If everything goes well, the last output lines should look like this:

  LD      arch/arm/boot/compressed/vmlinux
  OBJCOPY arch/arm/boot/zImage
  Kernel: arch/arm/boot/zImage is ready

The compilation will create (or overwrite) the following files:

That’s it. Congratulations, you’ve compiled the Linux kernel for Raspberry Pi! =D

If you got any trouble through this process, you might find additional references and tips at:

Now I’ll go through the steps for testing the compiled kernel.

Install an Operating System: Burn a Raspbian SD card

To properly test our newly compiled kernel image, I recommend to prepare a micro SD card with Raspbian. Also, once Graphical User Interface (GUI) will not be needed, I would recommend to use the Raspbian Lite version.

The official installation guide is very intuitive and comprehensive. Therefore, I will give no further explanation about it.

Prepare The SD Card

Once you have flashed a Raspbian image to a SD card there should be two partitions within it: boot and rootfs. Some OS automatically mount the partitions present on SD cards when they are inserted. You may use those mount points to copy files in and out of the card. If you got no mount points, the device and partitions to be mounted may be found with fdisk:

sudo fdisk -l
List of filesystems on the SD card used for raspbian.
Figure 1. List of filesystems on the SD card used for raspbian.

The boot filesystem is around 50 megabytes size and is FAT32 formatted (at /dev/mmcblk0p1 in my setup). It contains device tree blob files, overlays, the Linux kernel image, and configuration files needed to boot the system. The rootfs filesystem uses the rest of the SD card storage (at /dev/mmcblk0p2) and contains the root filesystem for the Raspbian operating system.

You may mount these partition to be able to copy the files. For instance:

sudo mkdir /mnt/rpi-boot
sudo mkdir /mnt/rpi-rootfs
sudo mount /dev/mmcblk0p1 /mnt/rpi-boot
sudo mount /dev/mmcblk0p2 /mnt/rpi-rootfs

Before we proceed, I recommend to backup the current kernel image in case of any trouble.

sudo cp /mnt/rpi-boot/kernel7.img ~/

Then, to test our generated Linux image, we need to copy it to the boot partition and name the image in such a way that it gets selected for the next boot. From the Linux root directory where the kernel was compiled you might type:

sudo cp arch/arm/boot/zImage /mnt/rpi-boot/kernel7.img

Note: kernel7.img is the default kernel filename on Pi 2 and Pi 3. On Pi 1 and Pi Zero it is kernel.img.

Note 2: The Linux image name may be set on the config.txt file in the boot filesystem. To boot with a custom image name you may add (or change) kernel=<kernel file name> in config.txt. For instance: echo kernel=my_kernel.img >> /mnt/rpi-boot/config.txt. See the Raspberry Pi boot options for more details.

The same idea applies to the device tree files. We may backup them:

mkdir ~/rpi-dtb-files
sudo cp /mnt/rpi-boot/*.dtb ~/rpi-dtb-files

Then copy the compiled dtb files in:

sudo cp arch/arm/boot/dts/*.dtb /mnt/rpi-boot/
sudo cp arch/arm/boot/dts/overlays/*.dtbo /mnt/rpi-boot/overlays/

To install the kernel modules, do:

sudo make INSTALL_MOD_PATH=/mnt/rpi-rootfs modules_install

Synchronize and umount the partitions to ensure data will persist on them:

rsync /mnt/rpi-boot
rsync /mnt/rpi-rootfs
sudo umount /mnt/rpi-boot
sudo umount /mnt/rpi-rootfs

For additional references on Raspberry Pi boot configurations, see the config-txt documentation.

We’re now ready to take the SD card out from the computer, attach it to the Raspberry Pi, and power the Pi on.

Test if the kernel image works

That’s the hour of truth. Attach the SD card to the Pi along with any peripherals you want (monitor, keyboard, etc.) so to be able to see what’s happening. In this tutorial, I’ve explored alternative ways to see what’s happening on a Raspberry Pi.

Power on and wait it to (hopefully) boot. If boot process completes normally, you may get a terminal to login. Once logged in, you may check the kernel version being used:

uname -r

The number displayed should be the than the one at the Makefile in the base directory of the kernel you’ve compiled. It should follow the format VERSION.PATCHLEVEL.SUBLEVEL.

You may also see the installed modules at */lib/modules/*. For instance, to see the installed IIO modules for kernel version 4.14.50-v7+ you may look for them:

ls /lib/modules/4.14.50-v7+/kernel/drivers/iio/

That’s it! You have successfully downloaded, configured, compiled, and tested a Linux kernel for Raspberry Pi. Congratulations!!!

Revision History: