🥇 Creating an Ubuntu image for ARM "from scratch"

When development just begins, it is often unclear which packages will go into the target rootfs.

In other words, it's too early to grab LFS, buildroot, or yocto (or something else), but we need to start. For those with resources (I have 4GB eMMC in my pilot samples), there's an option to provide a distribution to developers that will allow quickly delivering what is currently missing, and then we can always compile package lists and create a list for the target rootfs.

This article doesn't break new ground and is a simple copy-paste instruction.

The goal of the article is to build an Ubuntu rootfs for an ARM board (in my case based on Colibri imx7d).

Image Build

We are building the target rootfs for replication.

Unpacking Ubuntu Base

We choose the release based on necessity and personal preferences. Here I provided 20.

$ mkdir ubuntu20
$ cd ubuntu20
$ mkdir rootfs
$ wget http://cdimage.ubuntu.com/ubuntu-base/releases/20.04/release/ubuntu-base-20.04-base-armhf.tar.gz
$ tar xf ubuntu-base-20.04-base-armhf.tar.gz -C rootfs

Checking BINFMT Support in the Kernel

If you have a popular distribution, then BINFMT_MISC support is present and everything is configured. If not, I’m sure you know how to enable BINFMT support in the kernel.

Ensure that BINFMT_MISC is enabled in the kernel:

$ zcat /proc/config.gz | grep BINFMT
CONFIG_BINFMT_ELF=y
CONFIG_COMPAT_BINFMT_ELF=y
CONFIG_BINFMT_SCRIPT=y
CONFIG_BINFMT_MISC=y

Now we need to check the settings:

$ ls /proc/sys/fs/binfmt_misc
qemu-arm  register  status
$ cat /proc/sys/fs/binfmt_misc/qemu-arm
enabled
interpreter /usr/bin/qemu-arm
flags: OC
offset 0
magic 7f454c4601010100000000000000000002002800
mask ffffffffffffff00fffffffffffffffffeffffff

You can register manually using, for example, this instruction.

Configuring static arm qemu

Now we will need a statically built qemu instance.

!!! ATTENTION!!!
If you plan to use a container for building anything, please read:
https://sourceware.org/bugzilla/show_bug.cgi?id=23960
https://bugs.launchpad.net/qemu/+bug/1805913
Then, for an x86_64 host and an arm guest, you need to use the i386 version of qemu:
http://ftp.ru.debian.org/debian/pool/main/q/qemu/qemu-user-static_5.0-13_i386.deb

$ wget http://ftp.debian.org/debian/pool/main/q/qemu/qemu-user-static_5.0-13_amd64.deb
$ alient -t qemu-user-static_5.0-13_amd64.deb
# the path in rootfs and the executable name must match /proc/sys/fs/binfmt_misc/qemu-arm
$ mkdir qemu
$ tar xf qemu-user-static-5.0.tgz -C qemu
$ file qemu/usr/bin/qemu-arm-static
qemu/usr/bin/qemu-arm-static: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, BuildID[sha1]=be45f9a321cccc5c139cc1991a4042907f9673b6, for GNU/Linux 3.2.0, stripped
$ cp qemu/usr/bin/qemu-arm-static rootfs/usr/bin/qemu-arm
$ file rootfs/usr/bin/qemu-arm
rootfs/usr/bin/qemu-arm: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, BuildID[sha1]=be45f9a321cccc5c139cc1991a4042907f9673b6, for GNU/Linux 3.2.0, stripped

Chroot

Simple script:

ch-mount.sh

#!/bin/bash

function mnt() {
    echo "MOUNTING"
    sudo mount -t proc /proc ${2}proc
    sudo mount --rbind /sys ${2}sys
    sudo mount --make-rslave ${2}sys
    sudo mount --rbind /dev ${2}dev
    sudo mount --make-rslave ${2}dev
    sudo mount -o bind /dev/pts ${2}dev/pts
    sudo chroot ${2}
}

function umnt() {
    echo "UNMOUNTING"
    sudo umount ${2}proc
    sudo umount ${2}sys
    sudo umount ${2}dev/pts
    sudo umount ${2}dev

}

if [ "$1" == "-m" ] && [ -n "$2" ] ;
then
    mnt $1 $2
elif [ "$1" == "-u" ] && [ -n "$2" ];
then
    umnt $1 $2
else
    echo ""
    echo "Either 1'st, 2'nd or both parameters were missing"
    echo ""
    echo "1'st parameter can be one of these: -m(mount) OR -u(umount)"
    echo "2'nd parameter is the full path of rootfs directory(with trailing '/')"
    echo ""
    echo "For example: ch-mount -m /media/sdcard/"
    echo ""
    echo 1st parameter : ${1}
    echo 2nd parameter : ${2}
fi

Let's admire the result:

$ .\/ch-mount.sh -m rootfs\/\n# cat \/etc\/os-release\nNAME="Ubuntu"\nVERSION="20.04 LTS (Focal Fossa)"\nID=ubuntu\nID_LIKE=debian\nPRETTY_NAME="Ubuntu 20.04 LTS"\nVERSION_ID="20.04"\nHOME_URL="https:\/\/www.ubuntu.com\/"\nSUPPORT_URL="https:\/\/help.ubuntu.com\/"\nBUG_REPORT_URL="https:\/\/bugs.launchpad.net\/ubuntu\/"\nPRIVACY_POLICY_URL="https:\/\/www.ubuntu.com\/legal\/terms-and-policies\/privacy-policy"\nVERSION_CODENAME=focal\nUBUNTU_CODENAME=focal\n# uname -a\nLinux NShubin 5.5.9-gentoo-x86_64 #1 SMP PREEMPT Mon Mar 16 14:34:52 MSK 2020 armv7l armv7l armv7l GNU\/Linux

Out of curiosity, let's measure the size before and after installing the minimal (for me) set of packages:

# du -d 0 -h / 2>/dev/null
63M     /

Let's update:

# apt update
# apt upgrade --yes

Let's install the packages we are interested in:

# SYSTEMD_IGNORE_CHROOT=yes apt install --yes autoconf kmod socat ifupdown ethtool iputils-ping net-tools ssh g++ iproute2 dhcpcd5 incron ser2net udev systemd gcc minicom vim cmake make mtd-utils util-linux git strace gdb libiio-dev iiod

Kernel header files, modules, are a separate topic. We certainly won't be installing the bootloader, kernel, modules, or device tree via Ubuntu. They will come to us from external sources, or we will assemble them ourselves, or the manufacturer will provide them, but in any case, that is beyond the scope of this instruction.

To some extent, version discrepancies are acceptable, but it's better to take them from the kernel build.

# apt install --yes linux-headers-generic

Let's see what we got, and we got quite a lot:

# apt clean
# du -d 0 -h / 2>/dev/null
770M    /

Don't forget to set the password.

Packing the image

$ sudo tar -C rootfs --transform "s|^.\/||" --numeric-owner --owner=0 --group=0 -c .\/ | tar --delete .\/ | gzip > rootfs.tar.gz

We can additionally install etckeeper with autopush setup

Let's say we distributed our build, the work has started, how best to assemble various versions of our system later.

Etckeeper can help us.

Security is a personal matter for everyone:

you can protect specific branches

generate a unique key for each device

forbid force push

etc....

# ssh-keygen
# apt install etckeeper
# etckeeper init
# cd /etc
# git remote add origin ...

Let's configure autopush

Of course, we can create branches on the device in advance (let's say create a script or service that will run on the first boot).

# cat /etc/etckeeper/etckeeper.conf
PUSH_REMOTE="origin"

Or we can take a clever approach...

The lazy way

Let’s have some unique identifier, say the CPU serial number (or MAC — serious companies buy a range):

cat \/proc\/cpuinfo

# cat /proc/cpuinfo
processor       : 0
model name      : ARMv7 Processor rev 5 (v7l)
BogoMIPS        : 60.36
Features        : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm 
CPU implementer : 0x41
CPU architecture: 7
CPU variant     : 0x0
CPU part        : 0xc07
CPU revision    : 5

processor       : 1
model name      : ARMv7 Processor rev 5 (v7l)
BogoMIPS        : 60.36
Features        : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm 
CPU implementer : 0x41
CPU architecture: 7
CPU variant     : 0x0
CPU part        : 0xc07
CPU revision    : 5

Hardware        : Freescale i.MX7 Dual (Device Tree)
Revision        : 0000
Serial          : 06372509

Then we can use it as the branch name we will push to:

# cat /proc/cpuinfo | grep Serial | cut -d':' -f 2 | tr -d [:blank:]
06372509

Let's create a simple script:

# cat /etc/etckeeper/commit.d/40myown-push
#!/bin/sh
set -e

if [ "$VCS" = git ] && [ -d .git ]; then
  branch=$(cat /proc/cpuinfo | grep Serial | cut -d':' -f 2 | tr -d [:blank:])
  cd /etc/
  git push origin master:${branch}
fi

And that's it — after some time we can look at the changes and form a list of packages for the target firmware.