Content from 2019-09
It's been quite a long while since the 64-bit CPUs took over the world. It seems that most of the ecosystem moved on, and people stopped paying close attention to the 32-bit compatibility issues. I know I did. It's all fine - compatibility can be quite a pain for very negligible gain. However, some notable software projects are still stuck in the 32-bit-only rut. Furthermore, it seems to be the case even when their primary target platforms have been 64-bit capable for quite a long time now.
One of the projects in the first category is Bitcoin. The Bitcoin developers made their stable release without noticing that the crypto unit tests fail on 32-bit platforms. On the other side of the spectrum, there is the RaspberryPi software stack. It could not run well in 64-bit mode up until a couple of days ago. The good people at Balena Linux distro did the necessary porting work and submitted it to the upstream RPi kernel. This work is so fresh and has been done mainly by a third-party, even though the boards from RaspberryPi 2B 1.2 onwards have an ARMv8 CPU. That's early 2016. I guess that this situation is the result of not pushing the changes to the upstream kernel that seems to be very common in the embedded world and the lack of documentation.
The reason I am writing about this is that I have just hit both of these issues when trying to run a full Bitcoin node on my fresh and new RaspberryPi 4B. I ended up having to install stock Debian on it. It was not exactly hard, but there seemed to have been no instruction on the Internet, so I write down what I did here.
Except for a RaspberryPi board, you will need a microSD card and a Linux system that can write to this card. Following what Raspbian does, I partitioned my card in the following way:
]==> fdisk /dev/sdb Welcome to fdisk (util-linux 2.34). Changes will remain in memory only, until you decide to write them. Be careful before using the write command. Command (m for help): p Disk /dev/sdb: 7.41 GiB, 7948206080 bytes, 15523840 sectors Disk model: USB SD Reader Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos Disk identifier: 0x8c7e33d1 Device Boot Start End Sectors Size Id Type /dev/sdb1 8192 532480 524289 256M c W95 FAT32 (LBA) /dev/sdb2 540672 15523839 14983168 7.1G 83 Linux
The first partition is the boot partition holding the VideoCore GPU firmware necessary to boot a kernel as well as the Linux kernel image. I will discuss how to get both later. The second partition is for the root filesystem. I mounted them in my host system as described below and the rest of this document follows this convention.
]==> mount /dev/sdb2 /mnt ]==> mkdir /mnt/boot ]==> mount /dev/sdb1 /mnt/boot/
Furthermore, you will need to have the following packages installed in the host system:
]==> apt-get install debootstrap qemu-user-static ]==> apt-get install gcc-aarch64-linux-gnu bison flex python-pyelftools device-tree-compiler
The packages in the first group are necessary to bootstrap the root filesystem, while the ones in the second group are needed to cross-compile the kernel image.
You'll need the
arm64 flavor of Debian. I use the
testing distribution for
pretty much every system I have, but you probably can successfully run any
other. I also like using Midnight Commander, so I include that package in the
target installation. The include specification is a comma-separated list, and
you can declare any package you want there. My network operator is Init7, so I
use their mirror. You should select one that is close to you.
Finally, the target for the installation is
]==> qemu-debootstrap --include=mc --arch=arm64 testing /mnt/ http://mirror.init7.net/debian/
After the bootstrapping process completes, you need to build the kernel image
that can run on the board. The official website
describes the process quite well, but I like building proper
Debian packages instead. The commit with hash
db690083a4 worked fine for me.
]==> cd /tmp ]==> git clone https://github.com/raspberrypi/linux.git ]==> cd linux ]==> make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- bcm2711_defconfig ]==> make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- KBUILD_IMAGE=arch/arm64/boot/Image -j12 bindeb-pkg ]==> make ARCH=arm64 CROSS_COMPILE=/usr/bin/aarch64-linux-gnu- dtbs ]==> cp /tmp/linux/arch/arm64/boot/dts/broadcom/bcm2711-rpi-4-b.dtb /mnt/boot/ ]==> mkdir /mnt/boot/overlays ]==> cp /tmp/linux/arch/arm64/boot/dts/overlays/vc4-fkms-v3d.dtbo /mnt/boot/overlays/ ]==> cp /tmp/linux*deb /mnt
make command configures the kernel for this particular board. The
second one builds the Debian packages with the kernel, the kernel headers, and
the kernel userspace headers. The third one compiles the Device Tree files. The
last four commands install the necessary Device Tree files in the boot partition
and copy the kernel packages to the root filesystem.
You can now
chroot into the root filesystem, install the kernel, create a user
account, and change passwords. We also need to prevent the
package from being updated using
]==> chroot /mnt /usr/bin/qemu-aarch64-static /bin/bash ]==> dpkg -i linux-*deb ]==> rm linux*deb ]==> apt-mark hold linux-libc-dev ]==> useradd -m youruser ]==> passwd youruser ]==> passwd root
You then need to edit the files that play a role in the boot process. The
relevant listings are presented below. Make sure that you got the partition
setup right. You can get the
PARTUUIDs by listing:
ls -l /dev/disk/by-partuuid.
]==> cat /etc/hostname cryptopi ]==> cat /etc/fstab PARTUUID=8c7e33d1-01 /boot vfat defaults 0 2 PARTUUID=8c7e33d1-02 / ext4 defaults,noatime 0 1 ]==> cat /etc/apt/sources.list deb http://mirror.init7.net/debian/ testing main contrib non-free deb-src http://mirror.init7.net/debian/ testing main contrib non-free deb http://security.debian.org testing-security main contrib non-free deb-src http://security.debian.org testing-security main contrib non-free
Finally, all that's left is the installation of the VideoCore GPU firmware files and the configuration of the bootloader. Again, make sure that the partition IDs are right and that you name the kernel image name correctly.
]==> cd /mnt/boot ]==> wget https://github.com/raspberrypi/firmware/raw/master/boot/start4.elf ]==> wget https://github.com/raspberrypi/firmware/raw/master/boot/start4cd.elf ]==> wget https://github.com/raspberrypi/firmware/raw/master/boot/start4db.elf ]==> wget https://github.com/raspberrypi/firmware/raw/master/boot/start4x.elf ]==> wget https://github.com/raspberrypi/firmware/raw/master/boot/fixup4.dat ]==> wget https://github.com/raspberrypi/firmware/raw/master/boot/fixup4cd.dat ]==> wget https://github.com/raspberrypi/firmware/raw/master/boot/fixup4db.dat ]==> wget https://github.com/raspberrypi/firmware/raw/master/boot/fixup4x.dat ]==> cat config.txt enable_uart=1 dtparam=audio=on kernel=vmlinuz-4.19.73-v8+ disable_overscan=1 [pi4] dtoverlay=vc4-fkms-v3d max_framebuffers=2 arm_64bit=1 enable_gic=1 ]==> cat cmdline.txt dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=PARTUUID=8c7e33d1-02 rootfstype=ext4 elevator=deadline fsck.repair=yes quiet rootwait
That's it. You can now unmount the filesystems from the host, insert the microSD card into your RaspberryPi, and the Debian system should boot.
Generally, it's nice to have the network interface start up at the boot time so that you can download stuff from the Internet without a hassle. NetworkManager does that job well. The SSH daemon is also useful for other reasons.
]==> dhclient eth0 ]==> apt-get install network-manager ]==> systemctl enable NetworkManager ]==> apt-get install openssh-server ]==> systemctl enable ssh
You may also want to suppress the annoying Perl warnings about missing locale files and set up your favorite time zone.
]==> apt-get install locales tzdata ]==> dpkg-reconfigure reconfigure locales ]==> dpkg-reconfigure reconfigure tzdata
My Bitcoin node runs fine with all its >250GB of blockchain data, but I have not checked if anything else works at all. In particular, I have not tested the display drivers nor any camera setup. However, random people on the Internet claim that the GPU drivers are now in the kernel, so things should be fine.