Content tagged debian

Intro

It's been quite a long while since the 64-bit CPUs took over the world. It seems that most of the ecosystem moved on, and people stopped paying close attention to the 32-bit compatibility issues. I know I did. It's all fine - compatibility can be quite a pain for very negligible gain. However, some notable software projects are still stuck in the 32-bit-only rut. Furthermore, it seems to be the case even when their primary target platforms have been 64-bit capable for quite a long time now.

One of the projects in the first category is Bitcoin. The Bitcoin developers made their stable release without noticing that the crypto unit tests fail on 32-bit platforms. On the other side of the spectrum, there is the RaspberryPi software stack. It could not run well in 64-bit mode up until a couple of days ago. The good people at Balena Linux distro did the necessary porting work and submitted it to the upstream RPi kernel. This work is so fresh and has been done mainly by a third-party, even though the boards from RaspberryPi 2B 1.2 onwards have an ARMv8 CPU. That's early 2016. I guess that this situation is the result of not pushing the changes to the upstream kernel that seems to be very common in the embedded world and the lack of documentation.

The reason I am writing about this is that I have just hit both of these issues when trying to run a full Bitcoin node on my fresh and new RaspberryPi 4B. I ended up having to install stock Debian on it. It was not exactly hard, but there seemed to have been no instruction on the Internet, so I write down what I did here.

Prerequisites

Except for a RaspberryPi board, you will need a microSD card and a Linux system that can write to this card. Following what Raspbian does, I partitioned my card in the following way:

]==> fdisk /dev/sdb

Welcome to fdisk (util-linux 2.34).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


Command (m for help): p
Disk /dev/sdb: 7.41 GiB, 7948206080 bytes, 15523840 sectors
Disk model: USB SD Reader   
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x8c7e33d1

Device     Boot  Start      End  Sectors  Size Id Type
/dev/sdb1         8192   532480   524289  256M  c W95 FAT32 (LBA)
/dev/sdb2       540672 15523839 14983168  7.1G 83 Linux

The first partition is the boot partition holding the VideoCore GPU firmware necessary to boot a kernel as well as the Linux kernel image. I will discuss how to get both later. The second partition is for the root filesystem. I mounted them in my host system as described below and the rest of this document follows this convention.

]==> mount /dev/sdb2 /mnt
]==> mkdir /mnt/boot
]==> mount /dev/sdb1 /mnt/boot/

Furthermore, you will need to have the following packages installed in the host system:

]==> apt-get install debootstrap qemu-user-static
]==> apt-get install gcc-aarch64-linux-gnu bison flex python-pyelftools device-tree-compiler

The packages in the first group are necessary to bootstrap the root filesystem, while the ones in the second group are needed to cross-compile the kernel image.

System installation

You'll need the arm64 flavor of Debian. I use the testing distribution for pretty much every system I have, but you probably can successfully run any other. I also like using Midnight Commander, so I include that package in the target installation. The include specification is a comma-separated list, and you can declare any package you want there. My network operator is Init7, so I use their mirror. You should select one that is close to you. Finally, the target for the installation is /mnt.

]==> qemu-debootstrap --include=mc --arch=arm64 testing /mnt/ http://mirror.init7.net/debian/

After the bootstrapping process completes, you need to build the kernel image that can run on the board. The official website describes the process quite well, but I like building proper Debian packages instead. The commit with hash db690083a4 worked fine for me.

]==> cd /tmp
]==> git clone https://github.com/raspberrypi/linux.git
]==> cd linux
]==> make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- bcm2711_defconfig
]==> make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- KBUILD_IMAGE=arch/arm64/boot/Image -j12 bindeb-pkg
]==> make ARCH=arm64 CROSS_COMPILE=/usr/bin/aarch64-linux-gnu- dtbs
]==> cp /tmp/linux/arch/arm64/boot/dts/broadcom/bcm2711-rpi-4-b.dtb /mnt/boot/
]==> mkdir /mnt/boot/overlays
]==> cp /tmp/linux/arch/arm64/boot/dts/overlays/vc4-fkms-v3d.dtbo /mnt/boot/overlays/
]==> cp /tmp/linux*deb /mnt

The first make command configures the kernel for this particular board. The second one builds the Debian packages with the kernel, the kernel headers, and the kernel userspace headers. The third one compiles the Device Tree files. The last four commands install the necessary Device Tree files in the boot partition and copy the kernel packages to the root filesystem.

You can now chroot into the root filesystem, install the kernel, create a user account, and change passwords. We also need to prevent the linux-libc-dev package from being updated using apt-mark.

]==> chroot /mnt /usr/bin/qemu-aarch64-static /bin/bash
]==> dpkg -i linux-*deb
]==> rm linux*deb
]==> apt-mark hold linux-libc-dev
]==> useradd -m youruser
]==> passwd youruser
]==> passwd root

You then need to edit the files that play a role in the boot process. The relevant listings are presented below. Make sure that you got the partition setup right. You can get the PARTUUIDs by listing: ls -l /dev/disk/by-partuuid.

]==> cat /etc/hostname 
cryptopi
]==> cat /etc/fstab   
PARTUUID=8c7e33d1-01  /boot           vfat    defaults          0       2
PARTUUID=8c7e33d1-02  /               ext4    defaults,noatime  0       1
]==> cat /etc/apt/sources.list
deb http://mirror.init7.net/debian/ testing main contrib non-free
deb-src http://mirror.init7.net/debian/ testing main contrib non-free

deb http://security.debian.org testing-security main contrib non-free
deb-src http://security.debian.org testing-security main contrib non-free

Finally, all that's left is the installation of the VideoCore GPU firmware files and the configuration of the bootloader. Again, make sure that the partition IDs are right and that you name the kernel image name correctly.

]==> cd /mnt/boot
]==> wget https://github.com/raspberrypi/firmware/raw/master/boot/start4.elf
]==> wget https://github.com/raspberrypi/firmware/raw/master/boot/start4cd.elf
]==> wget https://github.com/raspberrypi/firmware/raw/master/boot/start4db.elf
]==> wget https://github.com/raspberrypi/firmware/raw/master/boot/start4x.elf
]==> wget https://github.com/raspberrypi/firmware/raw/master/boot/fixup4.dat
]==> wget https://github.com/raspberrypi/firmware/raw/master/boot/fixup4cd.dat
]==> wget https://github.com/raspberrypi/firmware/raw/master/boot/fixup4db.dat
]==> wget https://github.com/raspberrypi/firmware/raw/master/boot/fixup4x.dat
]==> cat config.txt 
enable_uart=1
dtparam=audio=on
kernel=vmlinuz-4.19.73-v8+
disable_overscan=1

[pi4]
dtoverlay=vc4-fkms-v3d
max_framebuffers=2
arm_64bit=1
enable_gic=1

]==> cat cmdline.txt
dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=PARTUUID=8c7e33d1-02 rootfstype=ext4 elevator=deadline fsck.repair=yes quiet rootwait

That's it. You can now unmount the filesystems from the host, insert the microSD card into your RaspberryPi, and the Debian system should boot.

Post-installation setup

Generally, it's nice to have the network interface start up at the boot time so that you can download stuff from the Internet without a hassle. NetworkManager does that job well. The SSH daemon is also useful for other reasons.

]==> dhclient eth0
]==> apt-get install network-manager
]==> systemctl enable NetworkManager
]==> apt-get install openssh-server
]==> systemctl enable ssh

You may also want to suppress the annoying Perl warnings about missing locale files and set up your favorite time zone.

]==> apt-get install locales tzdata
]==> dpkg-reconfigure reconfigure locales
]==> dpkg-reconfigure reconfigure tzdata

Conclusion

My Bitcoin node runs fine with all its >250GB of blockchain data, but I have not checked if anything else works at all. In particular, I have not tested the display drivers nor any camera setup. However, random people on the Internet claim that the GPU drivers are now in the kernel, so things should be fine.

Have fun!

Intro

I must be getting old and eccentric. I have recently started working on the Coursera's Scala Specialization. It's all great, but my first realization was that the tools they use are not going to work for me. The problem lies mainly with sbt - their build tool. It fetches and installs in your system God knows what from God knows where to bootstrap itself. I don't trust this kind of behavior in the slightest. I understand that automatic pulling of dependencies and auto-update may save work, but they are also dangerous. I don't even want to mention that they contribute to general bloat and sluggishness that plagues the Java world. You don't need to know what depends on what, so everything uses everything, without a single thought.

Having said all that, I do trust and use QuickLisp. So, go figure.

I would still like to take the class, but I would like to do it using Emacs and command line. Here's what I did.

Prerequisites

You'll need the following packages:

==> sudo apt-get install default-jdk scala ant junit4 scala-mode-el

The assignments they give seem to have a stub for implementation and a bunch of scalatest test suites. I will also want to write some other code to play with things. This is the directory structure I used:

==> find -type f
./01-hello-world/build.xml
./01-hello-world/src/HelloWorld.scala
./02-square-root-newton/build.xml
./02-square-root-newton/src/SquareRoot.scala
./a00-lists/build.xml
./a00-lists/src/example/Lists.scala
./a00-lists/test/example/ListsSuite.scala
./a01-pascal-balance-countchange/build.xml
./a01-pascal-balance-countchange/src/recfun/Main.scala
./a01-pascal-balance-countchange/test/recfun/BalanceSuite.scala
./a01-pascal-balance-countchange/test/recfun/CountChangeSuite.scala
./a01-pascal-balance-countchange/test/recfun/PascalSuite.scala
./build.properties
./build-targets.xml
./lib/get-libs.sh

You can get all this here and will need to run the get-libs.sh script to fetch the two scalatest jar files before you can do anything else.

Ant

I wrote an ant build template that sets up scalac and scalatest tasks and provides compile and test targets. All that the build.xml files in the subdirectories need to do is define some properties and import the template:

1 <project default="compile">
2   <property name="jar.name" value="recfun.jar" />
3   <property name="jar.class" value="recfun.Main" />
4   <property name="tests-wildcard" value="recfun" />
5   <import file="../build-targets.xml" />
6 </project>
  • jar.name is the name of the resulting jar file
  • jar.class is the default class that should run
  • test-wildcard is the name of the package containing the test suites (they are discovered automatically)

You can then run the thing (some output has been omitted for clarity):

]==> ant compile
init:
    [mkdir] Created dir: ./build/classes
    [mkdir] Created dir: ./build-test

compile:
   [scalac] Compiling 1 source file to ./build/classes
      [jar] Building jar: ./build/recfun.jar

]==> ant test
compile-test:
   [scalac] Compiling 3 source files to ./build-test

test:
[scalatest] Discovery starting.
[scalatest] Discovery completed in 127 milliseconds.
[scalatest] Run starting. Expected test count is: 11
[scalatest] BalanceSuite:
[scalatest] - balance: '(if (zero? x) max (/ 1 x))' is balanced
[scalatest] - balance: 'I told him ...' is balanced
[scalatest] - balance: ':-)' is unbalanced
[scalatest] - balance: counting is not enough
[scalatest] PascalSuite:
[scalatest] - pascal: col=0,row=2
[scalatest] - pascal: col=1,row=2
[scalatest] - pascal: col=1,row=3
[scalatest] CountChangeSuite:
[scalatest] - countChange: example given in instructions
[scalatest] - countChange: sorted CHF
[scalatest] - countChange: no pennies
[scalatest] - countChange: unsorted CHF
[scalatest] Run completed in 246 milliseconds.
[scalatest] Total number of tests run: 11
[scalatest] Suites: completed 4, aborted 0
[scalatest] Tests: succeeded 11, failed 0, canceled 0, ignored 0, pending 0
[scalatest] All tests passed.

Submiting the assignments

I could probably write some code to do the assignment submission in python or using ant, but I am too lazy for that. I will use the container handling script that I wrote for another occasion and install sbt in there. The docker/devel sub dir contains a Dockerfile for an image that has sbt installed in it.

==> git clone https://github.com/ljanyst/jail.git
==> cd jail/docker/devel
==> docker build -t jail:v01-dev .

This is the configurations script:

CONT_HOSTNAME=jail-scala
CONT_HOME=$HOME/Contained/jail-scala/home
CONT_NAME=jail:v01-dev
CONT_RESOLUTION=1680x1050

Intro

I have finally received my Kickstarter-backed UP boards. So far they seem great! There are three minor drawbacks, though:

  1. They don't have the exact same shape as Raspberry PI's, so they don't fit the raspberry cases. It's nothing that could not be rectified with small pliers, though.
  2. The audio chip on Cherry Trail (Intel Atom x5 z8350) SoCs is not yet supported by Linux out of the box, so some fiddling with the kernel is necessary.
  3. Debian's UEFI boot configuration does not seem to work from the get-go either.

Boot

You can install Debian Testing using a USB stick. Don't try Jessie, though - the kernel will not detect the MMC card. Things should work fine, except that grub will install itself in /EFI/debian/grubx64.efi on the EFI partition. You will need to move it to /EFI/boot/bootx64.efi manually. It's possible to do it from the UEFI shell using familiar commands.

Media

Kodi installs and works out of the box from Debian Multimedia. Unfortunately, to get the sound working, you will need to recompile the kernel :)

Get the sources and the necessary patches and create the config:

git clone https://github.com/torvalds/linux.git
cd linux
git remote add cht https://github.com/plbossart/sound.git
git fetch cht
git checkout byt-cht-hdmi-v4.7
make oldconfig

You will need to edit .config:

  • to set CONFIG_SYSTEM_TRUSTED_KEYS variable to an empty string
  • to enable CONFIG_HDMI

Then the only thing that's left is building and installing the package:

fakeroot make-kpkg --initrd --append-to-version=-up --revision=1 -j 8 kernel_image kernel_headers
cd ..
sudo dpkg -i linux-image-4.7.0-up+_1_amd64.deb

I wanted to see how efficient it is, so I run the compilation on the board itself. It took roughly 2.5 hours and got very hot. The board can handle perfectly fine the FullHD video files over Samba that Raspberry PI 2 couldn't. The audio quality is much better too. It seems that surround 5.1 actually works. :)

Note 09.11.2017: These drivers have been included into mainline since kernel 4.11. However, you will still need to edit your Alsa settings to choose the default output device. This has worked for me:

]==> cat /etc/asound.conf
defaults.pcm.!card Audio
defaults.ctl.!card Audio
defaults.pcm.!device 2
defaults.ctl.!device 2

Intro

My media center box has recently died tragically of overheating and I decided to replace it with a brand new Cubox-4iPro. While the hardware seems pretty great, the software support is less than perfect, to say the least. Especially if you decide to put, say, Debian Testing on it instead of one of the images prepared by the vendor. Basing on these notes, I was able to install and boot the system, and had quite some fun doing so. Not everything worked as described in the notes and some tweaking needed to be done, so I present here what worked for me.

Preparing the Micro SD card and bootstrapping the system

You need to create at least two partitions: a swap and a root partition. Remember to leave some space at the beginning for the boot loader, 4MB or 8192 sectors should be more than enough. The following layout works fine for my 64GB card.

]==> fdisk /dev/mmcblk0

Command (m for help): p

Disk /dev/mmcblk0: 63.9 GB, 63864569856 bytes
255 heads, 63 sectors/track, 7764 cylinders, total 124735488 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

        Device Boot      Start         End      Blocks   Id  System
/dev/mmcblk0p1            8192     8396799     4194304   82  Linux swap / Solaris
/dev/mmcblk0p2         8396800   124735487    58169344   83  Linux

Then create the file systems and mount the root partition.

]==> mkfs.ext4 /dev/mmcblk0p2
]==> mkswap /dev/mmcblk0p1
]==> mkdir /mnt/tmp
]==> mount /dev/mmcblk0p2 /mnt/tmp/

You will need qemu-user-static and debootstrap packages to install the system on the root partition. Go here to select the appropriate kernel version.

]==> apt-get install qemu-user-static debootstrap

]==> qemu-debootstrap --foreign \
     --include=ntp,ntpdate,less,u-boot,u-boot-tools,linux-image-3.14-2-armmp,bash-completion,fake-hwclock,emacs,mc \
     --exclude=nano --arch=armhf testing /mnt/tmp http://http.debian.net/debian

Installing and setting up the boot loader

Cubox has the Initial Program Loader (IPL) in its NVRAM and you need to put the Secondary Program Loader (SPL) and the primary boot loader (Das U-Boot in our case) at the beginning of the Micro SD card.

]==> dd if=/mnt/tmp/usr/lib/u-boot/mx6_cubox-i/SPL of=/dev/mmcblk0 bs=1K seek=1
39+0 records in
39+0 records out
39936 bytes (40 kB) copied, 0.00685229 s, 5.8 MB/s

]==> dd if=/mnt/tmp/usr/lib/u-boot/mx6_cubox-i/u-boot.img of=/dev/mmcblk0 bs=1K seek=42
292+1 records in
292+1 records out
299804 bytes (300 kB) copied, 0.0451772 s, 6.6 MB/s

You will also need an appropriate DTB file and an U-Boot environment. The DTB (Device Tree Blob) is a database that represents the hardware components in the system, it is provided by the kernel package. You can get the U-Boot environment by slightly massaging the one provided by the flash-kernel package. Finally, you will need to make an environment image using mkimage command (u-boot-tools package).

]==> cp /mnt/tmp/usr/lib/linux-image-3.14-2-armmp/imx6q-cubox-i.dtb /mnt/tmp/dtb

]==> smaug# cat /mnt/tmp/root/boot.cmd

setenv device mmc
setenv partition ${mmcdev}:${mmcpart}
setenv bootargs 'root=/dev/mmcblk0p2 rootfstype=ext4 rootwait console=ttymxc0,115200n8 console=tty1'

image_locations='/boot/ /'
for pathprefix in ${image_locations}
do
  load ${device} ${partition} ${loadaddr} ${pathprefix}vmlinuz \
  && load ${device} ${partition} ${fdt_addr} ${pathprefix}dtb \
  && load ${device} ${partition} ${ramdiskaddr} ${pathprefix}initrd.img \
  && echo "Booting Debian ${kvers} from ${device} ${partition}..." \
  && bootz ${loadaddr} ${ramdiskaddr}:${filesize} ${fdt_addr}
done

]==> mkimage -A arm -O linux -T script -C none -n "Initial u-boot script" -d /mnt/tmp/root/boot.cmd /mnt/tmp/boot/boot.scr
Image Name:   Initial u-boot script
Created:      Tue Aug 12 22:59:31 2014
Image Type:   ARM Linux Script (uncompressed)
Data Size:    576 Bytes = 0.56 kB = 0.00 MB
Load Address: 00000000
Entry Point:  00000000
Contents:
   Image 0: 568 Bytes = 0.55 kB = 0.00 MB

Applying basic settings

Before you can boot into the system, you need to provide the file system information, root password, console settings, host name, apt sources and such.

]==> cat /mnt/tmp/etc/fstab
/dev/mmcblk0p2 /               ext4    errors=remount-ro 0       1
/dev/mmcblk0p1 none            swap    defaults          0       0
]==> chroot /mnt/tmp passwd root
]==> echo 'T0:23:respawn:/sbin/getty -L ttymxc0 115200 vt100' >> /mnt/tmp/etc/inittab
]==> echo 'cubox' > /mnt/tmp/etc/hostname
]==> cat /mnt/tmp/etc/apt/sources.list
deb http://security.debian.org/ testing/updates main contrib non-free
deb-src http://security.debian.org/ testing/updates main contrib non-free

deb http://ftp2.fr.debian.org/debian// testing-updates main contrib non-free
deb-src http://ftp2.fr.debian.org/debian/ testing-updates main contrib non-free

deb http://ftp2.fr.debian.org/debian testing main contrib non-free
deb-src http://ftp2.fr.debian.org/debian testing main contrib non-free

The system does not have a real time clock, but you can cheat and preserve the time at least between reboots with good enough accuracy. To do this you can, for instance, set the current time to the last modification time of /var/log/syslog as early in the boot sequence as possible. Download this script and put it in /mnt/tmp/etc/init.d then run:

]==> chmod 755 /etc/init.d/rtcemu
]==> chroot /mnt/tmp /bin/bash
]==> update-rc.d rtcemu defaults
]==> touch /var/log/syslog

Serial console

As of writing this, the HDMI output does not really work out of the box, so you will need to access the serial console to boot into the system and make it accessible over the network. You can use a Micro USB cable and screen for this purpose, like this:

]==> screen /dev/ttyUSB0 115200

Booting

You are now ready to boot the box. Insert the Micro SD card, attach the power cable and see the system boot in your screen session. When it starts the boot count-down, stop it by pressing enter and type the following:

setenv mmcpart 2
saveenv
boot

Doing this changes the boot partition to 2 and saves the environment on the card so that you won't have to re-do this every time you reboot the system.

Post-boot settings

Now you need to configure the network, set up time zones, locale settings, keyboard layout and install some useful packages, like network-manager and openssh-server.

]==> dhclient eth0
]==> dpkg-reconfigure tzdata
]==> apt-get update
]==> apt-get install console-data keyboard-configuration locales
]==> dpkg-reconfigure console-data
]==> dpkg-reconfigure keyboard-configuration
]==> dpkg-reconfigure locales
]==> service keyboard-setup restart
]==> apt-get install network-manager openssh-server
]==> update-rc.d network-manager defaults
]==> update-rc.d ssh defaults

flash-kernel is an utility that can hook into dpkg and make the newly installed kernels visible to U-Boot.

]==> apt-get install flash-kernel
]==> cat /etc/default/flash-kernel
LINUX_KERNEL_CMDLINE="root=/dev/mmcblk0p2 rootfstype=ext4 rootwait console=ttymxc0,115200n8 console=tty1"

You have an operational base system now!

What's next?

So, what is next? Plenty of fun! :) The device needs a kernel that can leverage all its features and then xserver and xbmc, it's supposed to be a media center box after all. Stay tuned.