Desktop on a Stick

Ingredients

The first step is to get a Pi to test with. You’ll need a Pi 4, Pi 400, or CM4 with at least 4GB of RAM. At the time of writing, that’s a slightly tricky proposition given the supply shortages. The vital rpilocator can help here, but it would appear (at least according to Eben, who probably knows what he’s talking about!) that the shortage should begin to clear during the second half of 2023.

The second step is to get an image we can play with. I’m going to be using the recently released Ubuntu 23.04 (Lunar Lobster) Server for Raspberry Pi image, specifically the arm64 variant. You can use rpi-imager to flash this to a spare SD card, or even SSD drive attached to a USB adapter.

Historically, Ubuntu images (including the Ubuntu for Raspberry Pi images) have been built with a system called livecd-rootfs. This could charitably be described as a rough collection of dozens of hooks (and hacks) in the form of shell scripts, all of them plastered onto the already crufty base of livecd-rootfs, itself a gigantic mess of shell script.

The result is exactly the hellish nightmare of obscure regex-loaded nonsense that you might imagine. Worse still, many of the hacks inject configuration files into the resulting image that are unowned. That is to say, no Debian package owns these injected files, and that causes all sorts of fun when it comes to upgrades.

My friend William, before he left, made a valiant start on sorting out this mess by re-writing the old ubuntu-image tool. In concert with this effort, my small contribution was to start migrating hacks from livecd-rootfs into the ubuntu-settings package so that in Lunar (and beyond), as much configuration as possible would be directly under the control of the package management system (this contribution made a screeching U-turn when, at the last minute, it was decided we were going to use livecd-rootfs for lunar after all, but that’s another story!).

This is one of the major reasons it’s now much simpler to spin up a desktop image with cloud-init. Instead of having a write a ton of configuration to add all the necessary tweaks livecd-rootfs used to do, we can just include one extra package and (mostly) be done.

Preparing your kitchen

Once your SD card (or drive) is flashed, eject it, then re-insert it. Regardless of the OS you’re using (Windows, Mac OS, Linux), you should see the boot partition (labelled “system-boot”) appear [1]. We’re going to be tweaking several configuration files, specifically:

• config.txt — this is the Pi bootloader configuration
• cmdline.txt — this is the Linux kernel command line
• user-data — this is the main cloud-init configuration file
• network-config — this is the cloud-init network configuration

We’ll start with config.txt because the changes here are universal: they’ll be required no matter which desktop we’re intending to use:

1. Find the enable_uart=1 line (it’s on line 28 by default) and comment it out by inserting # at the start of the line. This simply disables the serial console which isn’t strictly necessary, but if the serial console is enabled, the Pi’s GPU clock is locked to low speed and obviously we want the GPU clock to float higher for our desktop.
2. At the end of the file add dtoverlay=vc4-kms-v3d to load the “full” KMS device-tree overlay (it should be in the [all] section at the end of the file)

Next we need to tweak the kernel command line in cmdline.txt. Again, all the changes here are common to all desktops. This file consists of a single line of text; don’t introduce any line breaks when editing it:

1. Delete console=serial0,115200 at the start. This just removes the tty associated with the serial console which we disabled above in config.txt.
2. Insert zswap.enabled=1 zswap.zpool=z3fold zswap.compressor=zstd at the start. This activates the zswap system that we use on the desktop.

These changes more or less boil down to the base differences between the Ubuntu Pi server and desktop images. Most of the other differences (since Lunar, as discussed above) come down to the selection of packages that are installed …

Picking your flavour

It’s time to learn what we can do with cloud-init. The most obvious thing we want is to have a particular desktop package installed. This is pretty simple, and achieved by adding the following block to the user-data file on the boot partition:

packages:
  - lubuntu-desktop
  - ubuntu-raspi-settings-desktop

This tells cloud-init that, on first boot, we want it to try and install two extra packages:

• lubuntu-desktop is the basis of the Lubuntu desktop images; we’ll substitute other desktop packages here later
• ubuntu-raspi-settings-desktop contains all the little tweaks for the Pi desktop (ensuring there’s a swap-file, disabling suspend in Gnome, inserting the correct modules for zswap into the initrd, setting appropriate permissions for I2C and SPI interfaces, etc.)

In addition to these we should also add the following settings (these aren’t mandatory, but highly recommended):

package_update: true
package_upgrade: true
package_reboot_if_required: true

The first line tells cloud-init we want it to run apt update to refresh the set of available packages. This is the most important line as, without it, cloud-init will likely fail at some point because the apt index shipped on the image will be out of date. The next line demands that apt upgrade should be run to ensure all packages are up to date. The final line indicates that, if any package indicates a reboot is required, we should carry one out (this is absolutely the case when installing a desktop, as a display manager will be installed at the very least).

Next, while some of the desktops available install a browser by default (by depending on the Firefox apt package, which in turns installs the Firefox snap), several don’t as their official images include the Firefox (or Chromium) snap directly without the apt “wrapper”. To avoid winding up with a desktop without a browser, we’ll add another section to install the Firefox snap explicitly:

snap:
  commands:
    - snap install --channel=latest/stable/ubuntu-23.04 firefox

I’m not entirely sure why the snap section is so different from packages, or why the full snap install <package> form is required. It does appear from the documentation that the snap.commands section simply allows arbitrary commands to be specified (canonical-livepatch is included in the examples), but in that case why call it a snap section?

The exceedingly long channel name causes the snap to follow a branch of a channel that may include per-release quirks. Or … something like that. I’m still not clear on the whole “channel” concept in snaps.

Locale-ly sourced

On the official Ubuntu Desktop for Raspberry Pi images, we have a first time setup process [2] which guides the user through locale selection and initial user creation. We don’t have that here, so we’d better get cloud-init to handle that instead. Firstly the locale, timezone, and keyboard settings:

locale: en_GB.UTF-8
timezone: Europe/London
keyboard:
  model: pc105
  layout: gb
  options: ctrl:nocaps

To find out the valid values for these settings, you can use the following commands on an existing Ubuntu desktop (I say desktop specifically because the available locales on server installations tend to be extremely minimal) [3]:

locale:

localectl list-locales

timezone:

timedatectl list-timezones

keyboard:

model:

localectl list-x11-keymap-models

layout:

localectl list-x11-keymap-layouts

variant:

localectl list-x11-keymaps-variants [layout]

options:

localectl list-x11-keymaps-option

Adding personality

Next we need to create, and customize, the initial user. By default, cloud-init will create a user named ubuntu with a locked password, which also has password-less sudo access, and a bunch of default group memberships (for things like direct video and audio access, etc). The default group memberships are fine, but I’d like a slightly more personalized username, and one with password-required sudo rights. This can be accomplished with the following block:

user:
  name: "dave"
  gecos: "Dave Jones"
  plain_text_passwd: "raspberry"
  lock_passwd: false
  sudo: "ALL=(ALL:ALL) ALL"
  ssh_import_id:
    - lp:waveform

This changes the default username to “dave”, and uses my name in the gecos field (which traditionally recorded a whole bunch of things like display name, office number, telephone number, and so forth but these days tends to just be used for the display name). The sudo setting specifically lacks the NOPASSWD flag, requiring a password for use. The ssh_import_id setting lists users from which to import public SSH keys (use “lp:” prefix for Launchpad usernames, and “gh:” for Github).

The block also sets the initial password to the plain-text string “raspberry” and ensures password login is possible (lock_passwd is true by default which means the only way to login is with SSH or another auxiliary mechanism, but that’s not so useful on the desktop!). This is horribly insecure! We could do something slightly more secure here by providing a hashed password instead. First we’d run mkpasswd to generate the hash:

$ mkpasswd --method=SHA-512 --rounds=4096
Password:
$6$rounds=4096$JZRfK1tM.xiWZtR5$XpMvuj2reJr.....lI0T4Z/

Then we would use this with hashed_passwd instead of plain_text_passwd:

user:
  name: "dave"
  gecos: "Dave Jones"
  hashed_passwd: "$6$rounds=4096$JZRfK1tM.xiWZtR5$XpMvuj2reJr.....lI0T4Z/"
  lock_passwd: false
  sudo: "ALL=(ALL:ALL) ALL"
  ssh_import_id:
    - lp:waveform

However, don’t be lulled into a false sense of security. As the cloud-init documentation notes:

While hashed_password is better than plain_text_passwd, using [a password] in user-data represents a security risk as user-data could be accessible by third-parties depending on your cloud platform

To be clear: user-data is a world-readable file on the Ubuntu Pi images (it has to be because it’s on a FAT partition). There is no absolutely secure method of setting the password via cloud-init (at least on the Ubuntu Pi images); the secure thing to do here is login and then change your password. Consider this password initial and temporary.

Baking in hacks

Finishing off our user-data file we’re going to make one final rather hacky (but useful) change. At the time of writing, Firefox starts under the XWayland layer. This leads to rather poor performance under certain circumstances. To force it to load under Wayland directly, we’d need to set the MOZ_ENABLE_WAYLAND environment variable. Ideally we would do this only in our user’s environment, not system wide. This could be done by appending a line to our user’s ~/.profile script. However, this doesn’t exist initially (and won’t until cloud-init creates our user). Thankfully, this presents no problem!

write_files:
  - path: /home/dave/.profile
    defer: true
    append: true
    content: |
      export MOZ_ENABLE_WAYLAND=1

The powerful (but obviously slightly dangerous) write_files key allows us to write to arbitrary files on first boot. Here we specify we’d like to write to /home/dave/.profile (adjust for your custom username as necessary). We also inform cloud-init that the write should be deferred which means this will take place after user creation. Further, we specify that we wish to append (not overwrite), and finally we provide the lines we’d like to append.

Going shopping

Naturally, in order to obtain the packages we’ve requested we’re going to need some network connectivity. If your Pi is going to be connected by Ethernet, you’re good to go; the default configuration will just use your Ethernet connection. However, if you’re going to be relying on WiFi instead we need to be a bit of surgery in the last file we mentioned earlier, network-config.

Firstly we’ll delete the ethernets section, then uncomment the wifis section and specify the local access point and password:

network:
  version: 2
  wifis:
    wlan0:
      dhcp4: true
      optional: true
      access-points:
        "my-wifi-ssid":
          password: "my-wifi-password"

It is also useful to specify the regulatory-domain here (note that this needs to be specified under the wlan0 key, which is the reason for the inclusion of the parent keys below):

network:
  wifis:
    wlan0:
      regulatory-domain: GB

The valid regulatory domains can be found in the Linux kernel source Specifically, the two-letter code after the “country” line, which are almost entirely the ISO-3166 standard two-letter country codes. The special “00” domain (the “world regulatory domain”) is the default, but relying on this can lead to poor WiFi performance, particularly in 5GHz setups where many channels have restrictions in the world domain.

The first byte is with the eye

At the end of all this tinkering, you should have files that look something like the following on your boot partition (lines you may wish to pay specific attention to are highlighted):

 1[all]
 2kernel=vmlinuz
 3cmdline=cmdline.txt
 4initramfs initrd.img followkernel
 5
 6[pi4]
 7max_framebuffers=2
 8arm_boost=1
 9
10[all]
11# Enable the audio output, I2C and SPI interfaces on the GPIO header. As these
12# parameters related to the base device-tree they must appear *before* any
13# other dtoverlay= specification
14dtparam=audio=on
15dtparam=i2c_arm=on
16dtparam=spi=on
17
18# Comment out the following line if the edges of the desktop appear outside
19# the edges of your display
20disable_overscan=1
21
22# If you have issues with audio, you may try uncommenting the following line
23# which forces the HDMI output into HDMI mode instead of DVI (which doesn't
24# support audio output)
25#hdmi_drive=2
26
27# Enable the serial pins
28#enable_uart=1
29
30# Autoload overlays for any recognized cameras or displays that are attached
31# to the CSI/DSI ports. Please note this is for libcamera support, *not* for
32# the legacy camera stack
33camera_auto_detect=1
34display_auto_detect=1
35
36# Config settings specific to arm64
37arm_64bit=1
38dtoverlay=dwc2
39
40[cm4]
41# Enable the USB2 outputs on the IO board (assuming your CM4 is plugged into
42# such a board)
43dtoverlay=dwc2,dr_mode=host
44
45[all]
46dtoverlay=vc4-kms-v3d

1zswap.enabled=1 zswap.zpool=z3fold zswap.compressor=zstd dwc_otg.lpm_enable=0 console=tty1 root=LABEL=writable rootfstype=ext4 rootwait fixrtc quiet splash

Please note that cmdline.txt must consist of a single line of text. Ignore any wrapping of text above.

 1#cloud-config
 2
 3hostname: lubuntu
 4
 5locale: en_GB.UTF-8
 6timezone: Europe/London
 7
 8keyboard:
 9  model: pc105
10  layout: gb
11  options: ctrl:nocaps
12
13user:
14  name: "dave"
15  lock_passwd: false
16  gecos: "Dave Jones"
17  plain_text_passwd: "raspberry"
18  sudo: "ALL=(ALL:ALL) ALL"
19  ssh_import_id:
20    - lp:waveform
21
22write_files:
23  - path: /home/dave/.profile
24    append: true
25    defer: true
26    content: |
27      export MOZ_ENABLE_WAYLAND=1
28
29package_update: true
30package_upgrade: true
31package_reboot_if_required: true
32
33packages:
34  - lubuntu-desktop
35  - ubuntu-raspi-settings-desktop
36
37snap:
38  commands:
39    - snap install firefox

You may be wondering if the ordering of these blocks matters. The answer is “no”. The ordering of actions in cloud-init does not depend on the ordering of the configuration file (and nor should it). The system is intended to be declarative; we are describing the state we wish to attain, not how to obtain it.

This is why the write_files section has options like defer which adjust the timing of these actions. By default, write_files entries will always occur early on so that written files can be available to other actions.

 1network:
 2  version: 2
 3  wifis:
 4    wlan0:
 5      regulatory-domain: GB
 6      dhcp4: true
 7      optional: true
 8      access-points:
 9        "my-wifi-ssid":
10          password: "my-wifi-password"

With all those written, safely eject your card (or SSD drive), plug it into your Pi and let it boot. This will take a … very … very … long … time. Installing an entire hierarchy of desktop packages in this manner is not a quick process, and the first boot will probably take at least 1 hour (I didn’t time every installation during this test, but the cloud-init logs for the Lubuntu run indicated setup on the microSD card took roughly 1½ hours).

However, once it’s complete, it should automatically reboot and you should find yourself at a shiny new graphical desktop login! There’s still some horror to sort out here (in particular the clash of networking configuration stacks), but we’ll deal with that next time when we’ll also take a look at comparing the current crop of desktops on Ubuntu, and see what’s changed in the last few years. We’ll also compare them to the official Gnome desktop, and learn a few more cloud-init tricks …