ARM์šฉ Ubuntu ์ด๋ฏธ์ง€๋ฅผ "์ฒ˜์Œ๋ถ€ํ„ฐ" ์ƒ์„ฑํ•˜๊ธฐ

๊ฐœ๋ฐœ์ด ๋ง‰ ์‹œ์ž‘๋˜๋ฉด ์–ด๋–ค ํŒจํ‚ค์ง€๊ฐ€ ๋Œ€์ƒ rootfs๋กœ ์ด๋™๋ ์ง€ ๋ช…ํ™•ํ•˜์ง€ ์•Š์€ ๊ฒฝ์šฐ๊ฐ€ ๋งŽ์Šต๋‹ˆ๋‹ค.

์ฆ‰, LFS, buildroot ๋˜๋Š” yocto(๋˜๋Š” ๋‹ค๋ฅธ ๊ฒƒ)๋ฅผ ์žก๊ธฐ์—๋Š” ๋„ˆ๋ฌด ์ด๋ฅด์ง€๋งŒ ์ด๋ฏธ ์‹œ์ž‘ํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค. ๋ถ€์ž(ํŒŒ์ผ๋Ÿฟ ์ƒ˜ํ”Œ์— 4GB eMMC๊ฐ€ ์žˆ์Œ)์˜ ๊ฒฝ์šฐ ํ˜„์žฌ ๋ˆ„๋ฝ๋œ ํ•ญ๋ชฉ์„ ์‹ ์†ํ•˜๊ฒŒ ์ œ๊ณตํ•  ์ˆ˜ ์žˆ๋Š” ๋ฐฐํฌํŒ์„ ๊ฐœ๋ฐœ์ž์—๊ฒŒ ๋ฐฐํฌํ•  ์ˆ˜ ์žˆ๋Š” ๋ฐฉ๋ฒ•์ด ์žˆ์Šต๋‹ˆ๋‹ค. ๊ทธ๋Ÿฐ ๋‹ค์Œ ์–ธ์ œ๋“ ์ง€ ํŒจํ‚ค์ง€ ๋ชฉ๋ก์„ ์ˆ˜์ง‘ํ•˜๊ณ  ํŒจํ‚ค์ง€ ๋ชฉ๋ก์„ ์ƒ์„ฑํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ๋Œ€์ƒ rootfs.

์ด ๊ธฐ์‚ฌ๋Š” ์ƒˆ๋กœ์šด ๊ธฐ์‚ฌ๊ฐ€ โ€‹โ€‹์•„๋‹ˆ๋ฉฐ ๊ฐ„๋‹จํ•œ ๋ณต์‚ฌ-๋ถ™์—ฌ๋„ฃ๊ธฐ ์ง€์นจ์ž…๋‹ˆ๋‹ค.

์ด ๊ธฐ์‚ฌ์˜ ๋ชฉ์ ์€ ARM ๋ณด๋“œ์šฉ Ubuntu rootfs(์ œ ๊ฒฝ์šฐ์—๋Š” Colibri imx7d ๊ธฐ๋ฐ˜)๋ฅผ ๊ตฌ์ถ•ํ•˜๋Š” ๊ฒƒ์ž…๋‹ˆ๋‹ค.

์ด๋ฏธ์ง€ ๊ตฌ์ถ•

๋ณต์ œ๋ฅผ ์œ„ํ•ด ๋Œ€์ƒ rootfs๋ฅผ ์–ด์…ˆ๋ธ”ํ•ฉ๋‹ˆ๋‹ค.

์šฐ๋ถ„ํˆฌ ๋ฒ ์ด์Šค ํฌ์žฅ ํ’€๊ธฐ

์šฐ๋ฆฌ๋Š” ํ•„์š”์™€ ์„ ํ˜ธ๋„์— ๋”ฐ๋ผ ๋ฆด๋ฆฌ์Šค๋ฅผ ์ง์ ‘ ์„ ํƒํ•ฉ๋‹ˆ๋‹ค. ์—ฌ๊ธฐ์„œ๋Š” 20์„ ์ฃผ์—ˆ์–ด์š”.

$ mkdir ubuntu20
$ cd ubuntu20
$ mkdir rootfs
$ wget http://cdimage.ubuntu.com/ubuntu-base/releases/20.04/release/ubuntu-base-20.04-base-armhf.tar.gz
$ tar xf ubuntu-base-20.04-base-armhf.tar.gz -C rootfs

์ปค๋„์—์„œ BINFMT ์ง€์› ํ™•์ธ

๊ณตํ†ต ๋ฐฐํฌํŒ์ด ์žˆ๋Š” ๊ฒฝ์šฐ BINFMT_MISC์— ๋Œ€ํ•œ ์ง€์›์ด ์žˆ๊ณ  ๋ชจ๋“  ๊ฒƒ์ด ๊ตฌ์„ฑ๋˜์–ด ์žˆ์Šต๋‹ˆ๋‹ค. ๊ทธ๋ ‡์ง€ ์•Š์€ ๊ฒฝ์šฐ ์ปค๋„์—์„œ BINFMT ์ง€์›์„ ํ™œ์„ฑํ™”ํ•˜๋Š” ๋ฐฉ๋ฒ•์„ ์•Œ๊ณ  ์žˆ๋‹ค๊ณ  ํ™•์‹ ํ•ฉ๋‹ˆ๋‹ค.

์ปค๋„์—์„œ BINFMT_MISC๊ฐ€ ํ™œ์„ฑํ™”๋˜์–ด ์žˆ๋Š”์ง€ ํ™•์ธํ•˜์„ธ์š”.

$ zcat /proc/config.gz | grep BINFMT
CONFIG_BINFMT_ELF=y
CONFIG_COMPAT_BINFMT_ELF=y
CONFIG_BINFMT_SCRIPT=y
CONFIG_BINFMT_MISC=y

์ด์ œ ์„ค์ •์„ ํ™•์ธํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.

$ ls /proc/sys/fs/binfmt_misc
qemu-arm  register  status
$ cat /proc/sys/fs/binfmt_misc/qemu-arm
enabled
interpreter /usr/bin/qemu-arm
flags: OC
offset 0
magic 7f454c4601010100000000000000000002002800
mask ffffffffffffff00fffffffffffffffffeffffff

์˜ˆ๋ฅผ ๋“ค์–ด ๋‹ค์Œ์„ ์‚ฌ์šฉํ•˜์—ฌ ์ˆ˜๋™์œผ๋กœ ๋“ฑ๋กํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ์—ฌ๊ธฐ ์ง€์นจ์ด ์žˆ์Šต๋‹ˆ๋‹ค.

qemu ์ •์  ์•” ์„ค์ •

์ด์ œ ์ •์ ์œผ๋กœ ์กฐ๋ฆฝ๋œ qemu ์ธ์Šคํ„ด์Šค๊ฐ€ ํ•„์š”ํ•ฉ๋‹ˆ๋‹ค.

!!! ์ฃผ๋ชฉ!!!
์ปจํ…Œ์ด๋„ˆ๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ๋ฌด์–ธ๊ฐ€๋ฅผ ๋งŒ๋“ค ๊ณ„ํš์ด๋ผ๋ฉด ๋‹ค์Œ์„ ํ™•์ธํ•˜์„ธ์š”.
https://sourceware.org/bugzilla/show_bug.cgi?id=23960
https://bugs.launchpad.net/qemu/+bug/1805913
๊ทธ๋Ÿฐ ๋‹ค์Œ x86_64 ํ˜ธ์ŠคํŠธ ๋ฐ arm ๊ฒŒ์ŠคํŠธ์˜ ๊ฒฝ์šฐ i386 ๋ฒ„์ „์˜ qemu๋ฅผ ์‚ฌ์šฉํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.
http://ftp.ru.debian.org/debian/pool/main/q/qemu/qemu-user-static_5.0-13_i386.deb

$ wget http://ftp.debian.org/debian/pool/main/q/qemu/qemu-user-static_5.0-13_amd64.deb
$ alient -t qemu-user-static_5.0-13_amd64.deb
# ะฟัƒั‚ัŒ ะฒ rootfs ะธ ะธะผั ะธัะฟะพะปะฝัะตะผะพะณะพ ั„ะฐะนะปะฐ ะดะพะปะถะฝะพ ัะพะฒะฟะฐะดะฐั‚ัŒ ั /proc/sys/fs/binfmt_misc/qemu-arm
$ mkdir qemu
$ tar xf qemu-user-static-5.0.tgz -C qemu
$ file qemu/usr/bin/qemu-arm-static
qemu/usr/bin/qemu-arm-static: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, BuildID[sha1]=be45f9a321cccc5c139cc1991a4042907f9673b6, for GNU/Linux 3.2.0, stripped
$ cp qemu/usr/bin/qemu-arm-static rootfs/usr/bin/qemu-arm
$ file rootfs/usr/bin/qemu-arm
rootfs/usr/bin/qemu-arm: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, BuildID[sha1]=be45f9a321cccc5c139cc1991a4042907f9673b6, for GNU/Linux 3.2.0, stripped

chroot

๊ฐ„๋‹จํ•œ ์Šคํฌ๋ฆฝํŠธ:

ch-mount.sh

#!/bin/bash

function mnt() {
    echo "MOUNTING"
    sudo mount -t proc /proc proc
    sudo mount --rbind /sys sys
    sudo mount --make-rslave sys
    sudo mount --rbind /dev dev
    sudo mount --make-rslave dev
    sudo mount -o bind /dev/pts dev/pts
    sudo chroot 
}

function umnt() {
    echo "UNMOUNTING"
    sudo umount proc
    sudo umount sys
    sudo umount dev/pts
    sudo umount dev

}

if [ "$1" == "-m" ] && [ -n "$2" ] ;
then
    mnt $1 $2
elif [ "$1" == "-u" ] && [ -n "$2" ];
then
    umnt $1 $2
else
    echo ""
    echo "Either 1'st, 2'nd or both parameters were missing"
    echo ""
    echo "1'st parameter can be one of these: -m(mount) OR -u(umount)"
    echo "2'nd parameter is the full path of rootfs directory(with trailing '/')"
    echo ""
    echo "For example: ch-mount -m /media/sdcard/"
    echo ""
    echo 1st parameter : 
    echo 2nd parameter : 
fi

์šฐ๋ฆฌ๋Š” ๊ทธ ๊ฒฐ๊ณผ์— ๊ฐํƒ„ํ–ˆ์Šต๋‹ˆ๋‹ค.

$ ./ch-mount.sh -m rootfs/
# cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
# uname -a
Linux NShubin 5.5.9-gentoo-x86_64 #1 SMP PREEMPT Mon Mar 16 14:34:52 MSK 2020 armv7l armv7l armv7l GNU/Linux

์žฌ๋ฏธ์‚ผ์•„ ์ตœ์†Œ ํŒจํ‚ค์ง€ ์„ธํŠธ๋ฅผ ์„ค์น˜ํ•˜๊ธฐ ์ „๊ณผ ํ›„์˜ ํฌ๊ธฐ๋ฅผ ์ธก์ •ํ•ด ๋ณด๊ฒ ์Šต๋‹ˆ๋‹ค.

# du -d 0 -h / 2>/dev/null
63M     /

์—…๋ฐ์ดํŠธํ•˜์ž:

# apt update
# apt upgrade --yes

๊ด€์‹ฌ ์žˆ๋Š” ํŒจํ‚ค์ง€๋ฅผ ์„ค์น˜ํ•ด ๋ณด๊ฒ ์Šต๋‹ˆ๋‹ค.

# SYSTEMD_IGNORE_CHROOT=yes apt install --yes autoconf kmod socat ifupdown ethtool iputils-ping net-tools ssh g++ iproute2 dhcpcd5 incron ser2net udev systemd gcc minicom vim cmake make mtd-utils util-linux git strace gdb libiio-dev iiod

์ปค๋„ ํ—ค๋” ํŒŒ์ผ๊ณผ ๋ชจ๋“ˆ์€ ๋ณ„๊ฐœ์˜ ๋ฌธ์ œ์ž…๋‹ˆ๋‹ค. ๋ฌผ๋ก  Ubuntu๋ฅผ ํ†ตํ•ด ๋ถ€ํŠธ๋กœ๋”, ์ปค๋„, ๋ชจ๋“ˆ, ์žฅ์น˜ ํŠธ๋ฆฌ๋ฅผ ์„ค์น˜ํ•˜์ง€๋Š” ์•Š์Šต๋‹ˆ๋‹ค. ์™ธ๋ถ€์—์„œ ์šฐ๋ฆฌ์—๊ฒŒ ์˜ค๊ฑฐ๋‚˜ ์šฐ๋ฆฌ๊ฐ€ ์ง์ ‘ ์กฐ๋ฆฝํ•˜๊ฑฐ๋‚˜ ๋ณด๋“œ ์ œ์กฐ์—…์ฒด์—์„œ ์ œ๊ณตํ•  ๊ฒƒ์ž…๋‹ˆ๋‹ค. ์–ด๋–ค ๊ฒฝ์šฐ์—๋„ ์ด๋Š” ์ด ์ง€์นจ์˜ ๋ฒ”์œ„๋ฅผ ๋ฒ—์–ด๋‚ฉ๋‹ˆ๋‹ค.

๋ฒ„์ „ ์ฐจ์ด๋Š” ์–ด๋Š ์ •๋„ ํ—ˆ์šฉ๋˜์ง€๋งŒ ์ปค๋„ ๋นŒ๋“œ์—์„œ ๊ฐ€์ ธ์˜ค๋Š” ๊ฒƒ์ด ๋” ์ข‹์Šต๋‹ˆ๋‹ค.

# apt install --yes linux-headers-generic

๋ฌด์Šจ ์ผ์ด ์ผ์–ด๋‚ฌ๋Š”์ง€ ์‚ดํŽด๋ณด๊ณ  ๋งŽ์€ ๊ฒฐ๊ณผ๊ฐ€ ๋‚˜์™”์Šต๋‹ˆ๋‹ค.

# apt clean
# du -d 0 -h / 2>/dev/null
770M    /

๋น„๋ฐ€๋ฒˆํ˜ธ๋ฅผ ์„ค์ •ํ•˜๋Š” ๊ฒƒ์„ ์žŠ์ง€ ๋งˆ์„ธ์š”.

์ด๋ฏธ์ง€ ํŒจํ‚น

$ sudo tar -C rootfs --transform "s|^./||" --numeric-owner --owner=0 --group=0 -c ./ | tar --delete ./ | gzip > rootfs.tar.gz

๋˜ํ•œ autopush ์„ค์ •์„ ์‚ฌ์šฉํ•˜์—ฌ etckeeper๋ฅผ ์„ค์น˜ํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

๊ธ€์Ž„, ์šฐ๋ฆฌ๊ฐ€ ์–ด์…ˆ๋ธ”๋ฆฌ๋ฅผ ๋ฐฐํฌํ–ˆ๋‹ค๊ณ  ๊ฐ€์ •ํ•˜๊ณ  ์‹œ์Šคํ…œ์˜ ์ตœ์‹  ๋ฒ„์ „์„ ๊ฐ€์žฅ ์ž˜ ์กฐ๋ฆฝํ•˜๋Š” ๋ฐฉ๋ฒ•์— ๋Œ€ํ•œ ์ž‘์—…์ด ์‹œ์ž‘๋˜์—ˆ์Šต๋‹ˆ๋‹ค.

etckeeper๊ฐ€ ์šฐ๋ฆฌ๋ฅผ ๋„์™€์ค„ ์ˆ˜ ์žˆ์–ด์š”.

์•ˆ์ „์€ ๊ฐœ์ธ์ ์ธ ๋ฌธ์ œ์ž…๋‹ˆ๋‹ค.

  • ํŠน์ • ์ง€์ ์„ ๋ณดํ˜ธํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค
  • ๊ฐ ์žฅ์น˜์— ๋Œ€ํ•œ ๊ณ ์œ  ํ‚ค ์ƒ์„ฑ
  • ๊ฐ•์ œ ํ‘ธ์‹œ ๋น„ํ™œ์„ฑํ™”
  • ๋“ฑ. ...
# ssh-keygen
# apt install etckeeper
# etckeeper init
# cd /etc
# git remote add origin ...

์ž๋™ ํ‘ธ์‹œ๋ฅผ ์„ค์ •ํ•ด ๋ณด๊ฒ ์Šต๋‹ˆ๋‹ค.

๋ฌผ๋ก , ์‚ฌ์ „์— ์žฅ์น˜์— ๋ถ„๊ธฐ๋ฅผ ์ƒ์„ฑํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค(์ฒ˜์Œ ์‹คํ–‰๋  ๋•Œ ์‹คํ–‰๋  ์Šคํฌ๋ฆฝํŠธ๋‚˜ ์„œ๋น„์Šค๋ฅผ ๋งŒ๋“ ๋‹ค๊ณ  ๊ฐ€์ •ํ•ด ๋ณด๊ฒ ์Šต๋‹ˆ๋‹ค).

# cat /etc/etckeeper/etckeeper.conf
PUSH_REMOTE="origin"

์•„๋‹ˆ๋ฉด ์ข€ ๋” ๋˜‘๋˜‘ํ•œ ์ผ์„ ํ•  ์ˆ˜๋„ ์žˆ๊ฒ ๋„ค์š”...

๊ฒŒ์œผ๋ฅธ ๋ฐฉ๋ฒ•

์ผ์ข…์˜ ๊ณ ์œ  ์‹๋ณ„์ž๊ฐ€ ์žˆ๋‹ค๊ณ  ๊ฐ€์ •ํ•ด ๋ณด๊ฒ ์Šต๋‹ˆ๋‹ค. ํ”„๋กœ์„ธ์„œ์˜ ์ผ๋ จ ๋ฒˆํ˜ธ(๋˜๋Š” MAC - ์‹ฌ๊ฐํ•œ ํšŒ์‚ฌ๊ฐ€ ํ•ด๋‹น ์ œํ’ˆ๊ตฐ์„ ๊ตฌ๋งคํ•จ)๋ฅผ ๊ฐ€์ •ํ•ด ๋ณด๊ฒ ์Šต๋‹ˆ๋‹ค.

cat / proc / cpuinfo

# cat /proc/cpuinfo
processor       : 0
model name      : ARMv7 Processor rev 5 (v7l)
BogoMIPS        : 60.36
Features        : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm 
CPU implementer : 0x41
CPU architecture: 7
CPU variant     : 0x0
CPU part        : 0xc07
CPU revision    : 5

processor       : 1
model name      : ARMv7 Processor rev 5 (v7l)
BogoMIPS        : 60.36
Features        : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm 
CPU implementer : 0x41
CPU architecture: 7
CPU variant     : 0x0
CPU part        : 0xc07
CPU revision    : 5

Hardware        : Freescale i.MX7 Dual (Device Tree)
Revision        : 0000
Serial          : 06372509

๊ทธ๋Ÿฐ ๋‹ค์Œ ์ด๋ฅผ ํ‘ธ์‹œํ•  ๋ธŒ๋žœ์น˜์˜ ์ด๋ฆ„์œผ๋กœ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

# cat /proc/cpuinfo | grep Serial | cut -d':' -f 2 | tr -d [:blank:]
06372509

๊ฐ„๋‹จํ•œ ์Šคํฌ๋ฆฝํŠธ๋ฅผ ๋งŒ๋“ค์–ด ๋ณด๊ฒ ์Šต๋‹ˆ๋‹ค.

# cat /etc/etckeeper/commit.d/40myown-push
#!/bin/sh
set -e

if [ "$VCS" = git ] && [ -d .git ]; then
  branch=$(cat /proc/cpuinfo | grep Serial | cut -d':' -f 2 | tr -d [:blank:])
  cd /etc/
  git push origin master:${branch}
fi

๊ทธ๊ฒŒ ์ „๋ถ€์ž…๋‹ˆ๋‹ค. ์ž ์‹œ ํ›„ ๋ณ€๊ฒฝ ์‚ฌํ•ญ์„ ํ™•์ธํ•˜๊ณ  ๋Œ€์ƒ ํŽŒ์›จ์–ด์— ๋Œ€ํ•œ ํŒจํ‚ค์ง€ ๋ชฉ๋ก์„ ์ƒ์„ฑํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

์ถ”์ฒœ์žฌ๋ฃŒ

BINFMT_MISC
๊ธฐํƒ€ ๋ฐ”์ด๋„ˆ๋ฆฌ ํ˜•์‹์— ๋Œ€ํ•œ ์ปค๋„ ์ง€์›(binfmt_misc)
qemu ์‚ฌ์šฉ์ž chroot๋กœ ์ปดํŒŒ์ผ
ARM์šฉ Ubuntu rootfs ๊ตฌ์ถ•
์‚ฌ์šฉ์ž ์ง€์ • Ubuntu ๋ผ์ด๋ธŒ๋ฅผ ์ฒ˜์Œ๋ถ€ํ„ฐ ๋งŒ๋“œ๋Š” ๋ฐฉ๋ฒ•
Crossdev qemu-์ •์ -์‚ฌ์šฉ์ž-chroot
etckeeper

getdents64 ๋ฌธ์ œ

readdir()์€ 32๋น„ํŠธ ํ˜ธ์ŠคํŠธ์˜ 64๋น„ํŠธ ์‚ฌ์šฉ์ž ์ •์  qemu์— ๋Œ€ํ•ด NULL(errno=EOVERFLOW)์„ ๋ฐ˜ํ™˜ํ•ฉ๋‹ˆ๋‹ค.
Ext4 64๋น„ํŠธ ํ•ด์‹œ๊ฐ€ 32๋น„ํŠธ glibc 2.28+๋ฅผ ์ค‘๋‹จํ•ฉ๋‹ˆ๋‹ค.
QEMU ์‚ฌ์šฉ์ž ๋ชจ๋“œ ์—๋ฎฌ๋ ˆ์ด์…˜์„ ์‚ฌ์šฉํ•  ๋•Œ armhf์— ๋Œ€ํ•œcompile_id_Detection์ด ์‹คํŒจํ•ฉ๋‹ˆ๋‹ค.
CMake๊ฐ€ qemu-arm์—์„œ ์ œ๋Œ€๋กœ ์ž‘๋™ํ•˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค.

์ถœ์ฒ˜ : habr.com