It's Pi all the way down... - desktop

Making Mantic Magic

2023-10-12T00:00:00+01:00

Meet the Minotaur!

It’s around this time of year that a new Ubuntu release comes out (including images for the Raspberry Pi), and shortly afterwards I pen something on the subject of how best to tweak it to get the most out of it on your Pi. Past subjects have included adding a swap file … but that’s now done out of the box. Or enabling the “zswap” facility … but that’s now on by default too. Erm … let’s see, how to flash the image to an SSD? Nope, that’s as easy as an SD card these days, thanks to rpi-imager. Disabling multipath to gain a bit more RAM on the Pi Zero? Nope, default now too.

The list is indeed shrinking fast, but it’s not completely empty yet and the arrival of the Pi 5 has certainly left a little opportunity for post-release tweakage!

Speaking of the 5 …

Presto!

What’s it like? How is the Pi 5, and how is Ubuntu on the Pi 5?

Simply put, it’s an awesome piece of Pi. I’ll go so far as to say it’s the first Pi where, if your computing needs are fairly typical, it will meet them as well as any cheap PC.

I use Pis for pretty much all my work. All my coding, packaging, emailing, talking to colleagues online, and much of my browsing is done on a Pi 4. But there it is: “much of”. Not “all of”. Playing videos on Youtube? Painful. A video call involving a dozen colleagues? Out of the question. For those I turn to a PC. Often at home, the honking huge thing which doubles as a gaming platform, but out on the road my little laptop does these duties.

When I call my laptop “little”, I’m not kidding. I like my diminutive, and above all cheap, computers. I really don’t like lugging around a thousand quid’s worth of easily breakable or nickable gear. So my current laptop is an Acer Travelmate Spin B1 which has these positively mind-blowing specs:

Component	Specification
CPU	Intel Pentium N4200 (ooooh!)
Graphics	Integrated Intel Graphics
RAM	4GB (yes, my Pi has more)
Storage	64GB eMMC (again, the Pi has a bigger, faster SSD)
Display	11.6” 1080p (you guessed it, my Pi has a bigger monitor)

Yet, for all its diminutive capacity, it still breezes past my main Pi 4 at boot time, and is an eminently more capable desktop machine [1]. In fact, I run two Pi 4s for my work; one runs the Ubuntu desktop and the other a customized Ubuntu server image with kmscon as the console. The vast majority of my work (including all email) is done on the latter because the console is so much faster than the desktop.

So … enough waffling and delay. What about the Pi 5?

It beats the laptop. Hands down. I know my laptop is nothing to write home about, but this Pi is, being the first Pi to roundly beat one of my PCs at being a “good desktop”. It boots faster (about 20 seconds from cold for Ubuntu; we’re not at the blazing 7 seconds that RaspiOS manages, but Ubuntu’s a rather beefier desktop environment). It runs faster (everything launches faster, and generally “feels” faster than on the laptop). It’s considerably more expandable (remember the storage on my laptop is a soldered-on eMMC module!). And yet, it’s cheaper.

The “cheaper” comparison obviously sounds unfair; the laptop includes a keyboard, touchpad, battery, screen and so on, and moreover it’s older technology (newer technology is typically cheaper per unit of performance than older). When I bought it refurbished several years ago, it still cost about 300 quid, so let’s tot up my Raspberry Pi equivalent:

The Pi 5 I’ve been testing on is the 8GB model (kindly provided a couple of months ago by the folk at Raspberry Pi), which retails for 80 quid
Add in a good power supply (12 quid)
Reasonable keyboard (20 quid [2])
Mouse (10 quid)
Monitor (80 quid for a 24” FHD)
Good SD card (15 quid for a 128GB SanDisk)

217 of your pounds sterling. Yes, definitely “cheaper”, and recall we’re comparing to a refurbished laptop, not a new one. It’s not as portable as the laptop, but 83 quid probably buys a fair sized power-bank! Alright, that’s still not a fair comparison but I’m going to make it anyway because my Pis really do accompany me on the road :-)

Dave’s “portable” setup.

Abracadabra!

What should you do to get the most out of Ubuntu on your Pi 5? Honestly, not a great deal. Anyone who’s used Ubuntu desktop on the Pi 4 extensively will be yelling things like “active cooling!” and “SSD boot for performance!” at the screen, but frankly these are optional on the 5 and absolutely unnecessary to have a system that works “well”.

The pre-release Pi 5 sent to me a few months back (thanks Gordon!) didn’t include the fancy new cooling fan [3]. The board does get warm, particularly under load, and it’s not too difficult to get it to throttle (watching a video fullscreen in Firefox will do the job). But do I notice it throttling? No! And once the video finishes (or the package finishes building, or what have you) and I go back to doing less computationally intensive things, the board’s back to an unthrottled temperature within a couple of seconds … without any active cooling.

The other thing the Pi 5 didn’t arrive with was the official power supply [4]. This is a rather more important piece as, in order to reach its full potential, the Pi 5 does have rather higher power demands. Specifically, if you want the USB ports to deliver enough juice to run external storage (and thus to boot from an SSD drive), you need a supply capable of delivering 5 amps at 5 volts. The vast majority of USB-PD compatible supplies don’t manage this (certainly none of the ones I owned could). There are a few third-party power supplies that do (for example, the Radxa Power PD 30W) but my recommendation to most would be to simply buy a Raspberry Pi supply.

I’ll write a bit more about this in a future article but an important aspect here is that the official supply has a “captive cable”. This means that at the “business end” (that sticks into your Pi), it will deliver 5V. If your supply requires an external cable, it’ll presumably deliver its rated 5V at its socket, but that doesn’t guarantee it’ll still be 5V at the business end of the cable you’re inserting. Furthermore, to carry 5A, USB cables need an e-marker chip.

Anyway, suffice it to say, I’m still running my 5 off my old 3A capable supply (an official power supply is on order!), and even though it may not power anything beefy off the USB ports, it’s still managed everything else without any brown-outs.

Cool as a very odd cucumber.

What about storage? Much the same applies. Yes, I run my Pi 4 on an SSD, and for doing serious work on the Pi 5 I would do exactly the same. But whereas on the Pi 4 this was about gaining capacity and performance, on the Pi 5 it’s really just down to extra capacity (and possibly reliability). The SD card interface is so much faster on the 5 (and the IO bandwidth to other components like memory so much bigger), that there really isn’t a hard requirement to have an SSD for good desktop performance. A decent SD card is perfectly sufficient.

In summary, a throttled, under-powered, SD-card driven Pi 5 is still a considerably more comfortable and responsive machine than an unthrottled, fully-powered, SSD-booted Pi 4. And it’s still capable of beating my laptop!

Ala Peanut Butter Sandwiches?

So … there’s nothing to do? No hardware to add, no software to tinker with to get the most out of Ubuntu on your shiny new Pi? Perish the thought! Where would we be without the need of a bit of tinkering? [5]

There’s a few major things that we couldn’t get into the release in time. The first is the upstream patches for libcamera (this is the reason for the camera module plugged into the board in the shot above — still testing!). This is intended to be delivered as an SRU in Mantic, which isn’t too bad given that libcamera isn’t included in the base image so users need to download it anyway.

The second is the mesa patches for full graphical performance. These did make it into the base image, so most applications on the desktop do indeed get the full benefit of the new Vulkan-compliant drivers. But some desktop applications (and one rather notable one in particular) don’t use the system’s mesa drivers because they’re shipped as snaps, so they use the libraries included in the corresponding content or base snaps (most of which are still from the 22.04 era).

Obviously this includes Firefox.

He’s not helping.

For the most part, this doesn’t matter too much. Browsing the web and even playing video is still fine because the CPU’s capable of just powering through (the Pi 5 really does have a beast of a processor!). However, fire up the WebGL aquarium sample and it becomes blindingly obvious. It’ll barely manage a paltry 2fps, if you’re lucky. Replace the Firefox snap with a deb so it’s using the system mesa libraries (I won’t go over the instructions again here, but you can derive them from this post if you really want to) and all of a sudden that jumps up to a much more respectable 40fps or so (at least on my 1080p monitor).

Naturally, the plan is to include the updated mesa drivers in the base snaps as soon as we’re able after launch, so hopefully this should be a short-lived state of affairs. Also, as noted, this doesn’t appear to affect the vast majority of web browsing, only rather niche things like WebGL.

The other missing bit is a power monitoring daemon, which is something I’ll be prioritising in the next cycle. The new power management chip on the Pi 5 allows the OS to determine whether the previous reboot was due to a power brownout. This is fantastic as brownout issues are notoriously variable and difficult to diagnose, so having the hardware tell you that this was definitely the cause, should deal with a whole pile of Heisenbugs.

Finally, the other major change with the Pi 5 is the means by which GPIO pins are controlled. No longer is the SoC itself driving the GPIOs. The new RP1 “southbridge” is now in charge instead. This means that the traditional GPIO libraries like RPi.GPIO no longer work. I’ve written about this previously and thankfully Ubuntu’s pretty well placed to weather the storm here. We switched the GPIO Zero back-end over back in hirsute (21.04) and added rpi-lgpio (a RPi.GPIO compatible shim) back in kinetic (22.10) to ease the transition.

These pieces needed a few updates [6] to deal with the Pi 5’s new hardware, but that’s all included in mantic. With a bit of luck most users either won’t notice the change, or won’t have much difficulty in getting their circuits to work on their shiny new Pi 5.

That’s all for now. Go grab your copy of Mantic Minotaur (it should be in rpi-imager by the time I publish this) and have fun!

[1]	Running Windows on it was as awful as you might imagine, but Ubuntu is still light enough to be perfectly useable on it

[2]	At this point, anybody who knows me is spluttering their tea over the screen; “twenty quid? Sure you’re not missing a zero there, Dave?”

[3]	There weren’t enough to give one to each alpha tester at that point, and it wasn’t necessary for Ubuntu’s compatibility work

[4]

Again, there weren’t enough to go around. However, I should add that they did very kindly (and usefully!) ship a Pico Probe with the board which was really useful in debugging things, especially since it attaches to the separate debug port now, meaning I didn’t have to unplug things from the GPIO pins just to get a serial console

[5]	That’s rhetorical :)

[6]	Try `pinout` on the new Pi 5!

The Ubuntu Desktop

2023-08-09T00:00:00+01:00

See Also: The intro post which links to all the other flavour posts.

In this post, we’ll be comparing the official Ubuntu desktop release with all the flavours we’ve been baking from cloud-init configurations, on a Raspberry Pi.

Contents

Ubuntu
Sort it out!
Popping and locking
Verdict

Ubuntu

Finally we come to the comparison with the official desktop. Obviously I don’t need to include any cloud-init bits here because … you just install the desktop image on your card and that’s it.

The official desktop has a few other simplicity tricks up its sleeve too: no need to explicitly select a Wayland-based desktop, you get one by default. The first-time setup wizard also handles configuring things like the keyboard layout and initial user without having to deal with YAML. So … points for simplicity?

The default applications are a mix of the brilliant (the aforementioned Rhythmbox) and the … not so brilliant. In the screenshot above you can see Totem failing to play anything (LP: #1998782 is the Ubuntu bug, but the upstream bug has more details), so it was back to sudo apt install vlc. The incorrect audio selection bug (LP: #1993347) also popped up but is trivial as ever to deal with.

Everything else is mostly fine: good terminal, fine web-browser, stuff opens happily even on unmounted network shares, though (as noted before) dragging files from an archive to extract them doesn’t work (LP: #1923033). Though this is ultimately due to the Wayland transition, it’s notable that it does work in Kubuntu, even when running Plasma under Wayland.

One other difference worth noting is that on this distribution I stuck with using the snap version of Firefox because we may as well do a comparison of stuff “as it ships” here (which we can’t really do with the other flavours). Despite being a snap, the Wayland support works (once the MOZ_ENABLE_WAYLAND=1 setting is exported), and since LP: #1974064 was fixed [1] opening links from a standalone PDF reader works too.

Speaking of Wayland, all the desktop animations are generally smooth and don’t lag. With the exception of the start menu.

Oh dear, it’s rant time …

Sort it out!

The full-screen zoom-out animation is just too large to do smoothly even under Wayland (sometimes it manages it after a few attempts). However, this doesn’t bother me too much because, once it’s deigned to appear, I can never find stuff in it anyway.

Some flavours have a categorical start menu: Lubuntu, Xubuntu, Kubuntu, and Studio. It’s a bit dated, but it works: find the category you want, scroll through the alphabetical list, and pick the application you want. Others take a more smart-phone-y approach and just have a big alphabetical list of everything installed to scroll through: Budgie, Unity … GNOME? GNOME has one big list. But it’s not alphabetical:

It’s not in most-recently-used order either. I have fired up Videos here, but not GVim for instance. Maybe something weird like installation order? Let’s install something and see. I’ve picked GIMP which, after installation, appears at the end of the list on the second screen, after all the other apparently alphabetically arranged items:

But, if we logout and login again, our start menu has re-arranged and now GIMP is before GVim, in apparently alphabetical order. But no! “Cheese” is now before “Additional Drivers” although prior to the reboot it came later in the list!

Okay, I give up. I’ve no idea what the order is here. Maybe it’s “whatever you want it to be”? You can re-order things manually by dragging them around, or drag them over each other to form groups (such as the pre-grouped “Utilities”). But this means that if I want a sensible layout (which the default … isn’t), I have to laboriously drag things around until they’re “just so”. I have to put work in (something no other flavour has demanded). Or if you just like them alphabetical then simply … erm … install an extension?

In essence the starter is a strange mash-up of “smart-phone home screen” and “smart-phone all-applications list”, made all the more smart-phone-y by scrolling sideways because … well, just because. The thing is … this isn’t a bug. This is genuinely GNOME’s idea of “how a start menu should work”!

The mitigation to this is that most flavours these days, including GNOME (but even some which favour a categorised menu like Kubuntu) allow you to search by just opening the start menu, typing a few letters of what you want, then hitting Enter when it’s the first in the list. This is how I tend to use the start menu on GNOME because it’s the only way it makes any sense. But, for this purpose I don’t care about the icons, and that in turns means I don’t care about giant full-screen zooming animations that lag the entire menu.

Okay, rant over.

Popping and locking

Oh, and the screen-saver? Blanks and locks the screen, but fails to suspend the monitor. I’ve dug into this before (LP: #1998716) and all I came away with was a new-found appreciation for how bloody complicated something as apparently simple as a screen-saver is, once you begin to understand all that it does.

Verdict

It’s not lightweight, but it is pretty, reasonably responsive, with a decent selection of apps, and despite a few Wayland transition issues (like drag and drop) it’s mostly functional … except for that start menu. Still, it can’t be that bad as it has been my main desktop for the last few years.

The problem is … now I’ve used Kubuntu …

[1]	during the writing of this series in fact — this is another bit I’ve had to go back and re-write!

The Ubuntu Studio

2023-08-08T00:00:00+01:00

See Also: The intro post which links to all the other flavour posts. Skip down here for the configurations used.

In this post, we’ll be looking at creating an Ubuntu Studio desktop with a cloud-init configuration, on a Raspberry Pi.

Contents

Ubuntu Studio
Verdict
Configuration

Ubuntu Studio

Originally, this was not a flavour I was planning to cover. After all Ubuntu Studio is intended for multimedia creation which is some of the heaviest-duty work you can put any computer through. However, I was also aware they’d recently switched to Plasma (from KDE) as a basis, and that meant Wayland was supported. After Kubuntu’s striking performance, I was intrigued to see how (or if!) it’d work on a Pi.

For Ubuntu Studio, the packages section in user-data contains an apparent multitude of packages:

packages:
  - ubuntustudio-desktop
  - ubuntustudio-audio
  - ubuntustudio-graphics
  - ubuntustudio-photography
  - ubuntustudio-video
  - ubuntustudio-publishing
  - ubuntu-raspi-settings-desktop
  - firefox

The various ubuntustudio-* packages other than ubuntustudio-desktop are all optional depending on what you want to use. For example, if you’re only interested in audio work, leave out the graphics, photography, video, and publishing meta-packages. I would recommend leaving out video because doing video-editing on a Pi is … “optimistic” to say the least. However, one oddity of Ubuntu Studio is that it does “assume” a certain set of installed packages in the default quick launcher. If you exclude certain meta-packages you may find a number of blank “?” icons in the quick launch bar because of “missing” applications. You can un-pin these items without consequence.

One other note on default applications: Ubuntu Studio typically ships with FreeShow (which seems to be a dead link at the time of writing) from the corresponding snap. Unfortunately we can’t include that here as it’s another snap that’s only built for amd64.

There’s plenty in these meta-packages that won’t work on a Pi even if it will install without complaint. For instance, OBS Studio and Blender both require a GPU with OpenGL 3.3 support (the Pi’s only supports up to 2.1). You can attempt to override this with MESA_GL_VERSION_OVERRIDE=3.3 in your environment, but don’t expect much in the way of performance (and also don’t be surprised if several things, like the surfaces in the Blender preview, don’t display at all, assuming you’re patient enough to wait for it to start).

That said, we should talk about the performance because, once again, we’re dealing with a Plasma desktop running under Wayland (assuming you selected the Wayland option on the login screen, which I highly recommend). And, once again, the performance (for a Pi) is great. Despite it being an environment replete with little UI animations, they all seem to play smoothly. The interface never feels laggy in responding to clicks or movements. Sure, there’s some delay when launching things but that’s not the desktop interface; that’s the SD card.

In use Ubuntu Studio is a bit different. For starters, it was the only flavour here to correctly select the HDMI audio output by default. Perhaps this is something to do with the rather more complex audio stack? This is no more than a guess on my part, however.

Studio is by far the largest of the flavours here. With all the meta-packages included above it weighs in at a positively obese 17GB on the SD card, so don’t even think about trying this on anything less than a 32GB card (unless you leave out some meta-packages). Speaking of storage, the partitions on the SD card showed up as removable in the file browser (same as in Kubuntu).

As on Kubuntu, the terminal was the the typically excellent and feature-full Konsole, and the file-browser is again Dolphin. Still, it feels a bit churlish to look at my usual set of applications for Ubuntu Studio. No-one’s going to be installing Studio just to play music or fiddle with the terminal; it’s a flavour for creators. So I scoured the house for all the interesting gear I could find to plug in and see how things went …

First off, given we’re dealing with a novel display stack (Wayland), how about trying to colour-calibrate the monitor? I dug out my trusty old ColorHug2 and strapped it to the screen, fired up DisplayCAL and got to work. No issues generating the profile: hardware was recognized, the screen-saver was suppressed for the duration of calibration, and although the calculations took a while (it is a Pi), it was ultimately successful at generating a profile. Unfortunately, I couldn’t get the profile to install at the end of it (complaints about a timeout, though after a bit of digging I think there’s something else going on). Nor, for that matter, did the swapped-red-and-blue test profile appear to work, so there’s something else missing here. Oh well.

What about drawing? I helped myself to the ancient Wacom Intuos 3 tablet attached to my other half’s Mac Mini, fired up Krita and drew some bits that 6 year-old me would’ve been truly proud of. Ahem. Needless to say, the tablet Just Worked, including all the fun configuration options for optional hot-keys, pressure response curves and tool switching depending on the pen tip. I can’t say this is terribly surprising, though, as it’s always worked on every Linux system I’ve tried it on for the last 10 years or so.

How about music composition? I nicked my daughter’s Yamaha keyboarad, found the one remaining USB-B cable in the house (that wasn’t busy serving a UPS), and hooked it up as a MIDI source. After failing to figure out Ardour, I managed to rekindle some of my LMMS skills and, with some awkward fumbling around the sound output configuration, got it recording from (and playing back through) the keyboard. My formerly formidable piano skills have sadly bit-rotted over the last 30 years. But I still managed to record a passable rendition of the opening of Scott Joplin’s The Entertainer … treble only, at a positively sedate 45bpm, then the bass line on a separate track, then sped it all up to 70 to pretend I could still play.

On the publishing side of things I gave the excellent MuseScore a whirl, including the keyboard as MIDI input. It performed happily and I managed to crank out a vaguely acceptable score for the first bit of The Entertainer again [1]. Printing remotely worked happily, as did scanning (once I’d configured SANE to look on the right host for the scanner).

I gave the video editing and rendering bits a brief try, but they were pretty much as one would expect: basically unusable without much beefier hardware.

Finally, I tried out Scribus as well and was rather surprised to find it very “laggy” when dragging things out on the canvas. I pondered if it’s one of the applications currently suffering from Wayland transition issues and ran it instead with QT_QPA_PLATFORM=xcb and that largely fixed things. Still, I didn’t dig any further (as I’d already burned far too much time playing with this flavour).

But now for the burning question: what about the screen-saver?

It managed to lock the screen (no suspend), and then crashed. It didn’t crash back to the desktop (good marks for security), but also didn’t display a “here’s how to get out of this” message like Kubuntu. I managed to manually use the loginctl trick from Kubuntu to figure out an unlock from another console, but I can imagine an ordinary user resorting to the three-finger shuffle here. Ouch.

Verdict

This isn’t your regular flavour, and it’s certain overkill for many. But if you’re the creative type it may well be the flavour for you. Provided your expectations of the platform are reasonable (this is not the box for Blender, OBS Studio, or kdenlive) it’s useable for basic music creation, composition, writing or art.

Some things aren’t responsive in all circumstances (when drawing with a tablet you have to be a bit careful of what’s visible vs the complexity of the brushes), but with a bit of patience and effort it works! Studio also has a really nice menu detailing where to get further help, from documentation to community support.

It looks like there’s still some kinks to work out (particularly around the Wayland transition) but yet again I’m very impressed by a KDE-based flavour.

Configuration

For Ubuntu Studio, the configurations I used were as follows. For the boot configuration, highlighted lines are those changed from the lunar defaults:

config.txt

 1[all]
 2kernel=vmlinuz
 3cmdline=cmdline.txt
 4initramfs initrd.img followkernel
 5
 6[pi4]
 7max_framebuffers=2
 8arm_boost=1
 9
10[all]
11# Enable the audio output, I2C and SPI interfaces on the GPIO header. As these
12# parameters related to the base device-tree they must appear *before* any
13# other dtoverlay= specification
14dtparam=audio=on
15dtparam=i2c_arm=on
16dtparam=spi=on
17
18# Comment out the following line if the edges of the desktop appear outside
19# the edges of your display
20disable_overscan=1
21
22# If you have issues with audio, you may try uncommenting the following line
23# which forces the HDMI output into HDMI mode instead of DVI (which doesn't
24# support audio output)
25#hdmi_drive=2
26
27# Enable the serial pins
28#enable_uart=1
29
30# Autoload overlays for any recognized cameras or displays that are attached
31# to the CSI/DSI ports. Please note this is for libcamera support, *not* for
32# the legacy camera stack
33camera_auto_detect=1
34display_auto_detect=1
35
36# Config settings specific to arm64
37arm_64bit=1
38dtoverlay=dwc2
39
40[cm4]
41# Enable the USB2 outputs on the IO board (assuming your CM4 is plugged into
42# such a board)
43dtoverlay=dwc2,dr_mode=host
44
45[all]
46dtoverlay=vc4-kms-v3d

For the kernel command line, the entire file must consist of a single line of text so I would suggest simply copying this wholesale:

cmdline.txt

1zswap.enabled=1 zswap.zpool=z3fold zswap.compressor=zstd dwc_otg.lpm_enable=0 console=tty1 root=LABEL=writable rootfstype=ext4 rootwait fixrtc quiet splash

For the cloud-init configuration, the highlighted lines are those that you may wish to change after copying the content. As mentioned above, you may wish to trim the list of meta-packages highlighted:

user-data

 1#cloud-config
 2
 3hostname: studio-pi
 4
 5locale: en_GB.UTF-8
 6timezone: Europe/London
 7
 8keyboard:
 9  model: pc105
10  layout: gb
11
12user:
13  name: "dave"
14  lock_passwd: false
15  gecos: "Dave Jones"
16  plain_text_passwd: raspberry
17  sudo: "ALL=(ALL:ALL) ALL"
18
19ssh_import_id:
20  - lp:waveform
21
22apt:
23  sources:
24    mozillateam:
25      source: 'ppa:mozillateam/ppa'
26
27write_files:
28  - path: /etc/apt/preferences.d/firefox
29    content: |
30      Package: firefox*
31      Pin: release o=LP-PPA-mozillateam
32      Pin-Priority: 501
33
34      Package: firefox*
35      Pin: release o=Ubuntu
36      Pin-Priority: -1
37  - path: /home/dave/.profile
38    append: true
39    defer: true
40    content: |
41      export MOZ_ENABLE_WAYLAND=1
42
43package_update: true
44package_upgrade: true
45package_reboot_if_required: true
46
47packages:
48  - ubuntustudio-desktop
49  - ubuntustudio-audio
50  - ubuntustudio-graphics
51  - ubuntustudio-photography
52  - ubuntustudio-video
53  - ubuntustudio-publishing
54  - ubuntu-raspi-settings-desktop
55  - firefox

Finally, the networking configuration. I used Ethernet for my experiments, and given the number of packages that need installing I’d generally recommend that too. Please see the intro post for important information on adjusting the network configuration to Network Manager post installation:

network-config

1network:
2  version: 2
3  ethernets:
4    eth0:
5      dhcp4: true
6      optional: true

[1]	Hang on, if you’re so gung-ho about hating WYSIWYG, shouldn’t you be using LilyPond here? Yes, but that’s not part of the Ubuntu Studio audio seed :)

The Kubuntu Flavour

2023-08-05T00:00:00+01:00

See Also: The intro post which links to all the other flavour posts. Skip down here for the configurations used.

In this post, we’ll be looking at creating a Kubuntu desktop with a cloud-init configuration, on a Raspberry Pi.

Contents

Kubuntu
Verdict
Configuration

Kubuntu

Kubuntu is the KDE-based spin of Ubuntu. For those unfamiliar, KDE is the other “big” desktop player besides GNOME, and is based on the Qt toolkit. KDE’s always had a more “traditional” feel in its desktop (almost staid, some might say). The start bar (with a classical categorised start menu) goes at the bottom, followed by the application switcher, and system tray. Applications all have a reasonably common look and feel, and all applications have every configuration option you could ever hope for scattered throughout their numerous pages of preferences.

This is in fairly stark contrast to GNOME which, of late, has been more experimental and has pursued a deliberate path of “making it simple”. To put it another way, if you sat your elderly mother in front of a modern GNOME system, it might take them a while to figure out how to get going. But on KDE, they’d be up and running right out the gate [1].

The last time I tried Kubuntu on a Pi, it was laughably slow, and I was expecting no different this time. I could not have been more wrong …

Kubuntu is notably the only other desktop environment here that supports Wayland out of the box. It defaults to X11, but you can easily select “Desktop Session: Plasma (Wayland)” from the bottom of the KDM login page, and this selection will persist for future logins. Speaking of logins, there’s a bizarrely long delay before the desktop shows up after login. This appears peculiar to KDE but if you put up with it, the wait is worthwhile!

When running under Wayland, the environment is buttery-smooth. All the little animations from the fade-in of the desktop, to the opening slide of the start menu proceed without jitters or jerks. Don’t expect any speed up in opening large applications. This is still a Pi, and I’m running these experiments off SD cards which top out at 20Mb/s transfer on a Pi 4. Nonetheless, I was astonished to find that the desktop I expected to be the heavy-weight laggard was up there with the best of them in responsiveness!

The bundled applications largely work well. The file browser is the capable Dolphin which was happy browsing and opening things from unmounted network shares, and has a nifty “multiple-selection” feature (look for the “plus” when hovering over items). One oddity is that, as under Lubuntu, it considered the partitions on the SD card “removable” (though it mercifully didn’t offer an eject button for the root partition). One other thing it notably managed was to extract files from an archive with drag’n’drop. This has been broken on GNOME (and thus Ubuntu, LP: #1923033) for years now.

As before, the default audio output is incorrectly selected, but again that’s a trivial work-around. Video playback is handled by the Haruna media player. Unfortunately this wasn’t happy playing things from unmounted network shares (despite them being accessible from the Open File menu). Otherwise, it proved one of the more capable bundled players, handling chapter markers and subtitles with aplomb. The bundled music player is Elisa which is a simple, attractive, and intuitive affair, but sadly doesn’t support playing (or indexing) anything other than a mounted file-system.

Firefox worked well, but now we finally come to something I alluded to at the start. When installed as a snap it doesn’t seem able to use the “full” Wayland mode (reverting to XWayland protocol instead). This is the reason for installing the deb, and also the reason for me re-writing a whole pile of stuff here [2]!

The screen-saver works … then sometimes crashes. At this point I was getting rather used to flakiness in screen-savers but what surprised me was that, having crashed, it brought up a nice and detailed set of instructions on securely unlocking the desktop from another terminal. This … seems like an improvement?

Verdict

What a difference from last time!

Great performance (after a slow initial login), combined with KDE’s legendary configurability. Some reliability issues still (screen-saver, not-so-removable storage!), but this is a genuinely useful desktop on a Pi now.

There’s also a ton of nice little UI tweaks that made it genuinely pleasant to use for several days. The start menu encapsulates all styles (categorical, one big list, and searchable), the system tray has myriad capabilities that make it far more useful than most (e.g. selecting the audio output, or bluetooth devices straight from the tray widgets without launching other things).

Configuration

For Kubuntu, the configurations I used were as follows. For the boot configuration, highlighted lines are those changed from the lunar defaults:

config.txt

 1[all]
 2kernel=vmlinuz
 3cmdline=cmdline.txt
 4initramfs initrd.img followkernel
 5
 6[pi4]
 7max_framebuffers=2
 8arm_boost=1
 9
10[all]
11# Enable the audio output, I2C and SPI interfaces on the GPIO header. As these
12# parameters related to the base device-tree they must appear *before* any
13# other dtoverlay= specification
14dtparam=audio=on
15dtparam=i2c_arm=on
16dtparam=spi=on
17
18# Comment out the following line if the edges of the desktop appear outside
19# the edges of your display
20disable_overscan=1
21
22# If you have issues with audio, you may try uncommenting the following line
23# which forces the HDMI output into HDMI mode instead of DVI (which doesn't
24# support audio output)
25#hdmi_drive=2
26
27# Enable the serial pins
28#enable_uart=1
29
30# Autoload overlays for any recognized cameras or displays that are attached
31# to the CSI/DSI ports. Please note this is for libcamera support, *not* for
32# the legacy camera stack
33camera_auto_detect=1
34display_auto_detect=1
35
36# Config settings specific to arm64
37arm_64bit=1
38dtoverlay=dwc2
39
40[cm4]
41# Enable the USB2 outputs on the IO board (assuming your CM4 is plugged into
42# such a board)
43dtoverlay=dwc2,dr_mode=host
44
45[all]
46dtoverlay=vc4-kms-v3d

For the kernel command line, the entire file must consist of a single line of text so I would suggest simply copying this wholesale:

cmdline.txt

1zswap.enabled=1 zswap.zpool=z3fold zswap.compressor=zstd dwc_otg.lpm_enable=0 console=tty1 root=LABEL=writable rootfstype=ext4 rootwait fixrtc quiet splash

For the cloud-init configuration, the highlighted lines are those that you may wish to change after copying the content:

user-data

 1#cloud-config
 2
 3hostname: kubuntu-pi
 4
 5locale: en_GB.UTF-8
 6timezone: Europe/London
 7
 8keyboard:
 9  model: pc105
10  layout: gb
11
12user:
13  name: "dave"
14  lock_passwd: false
15  gecos: "Dave Jones"
16  plain_text_passwd: raspberry
17  sudo: "ALL=(ALL:ALL) ALL"
18
19ssh_import_id:
20  - lp:waveform
21
22apt:
23  sources:
24    mozillateam:
25      source: 'ppa:mozillateam/ppa'
26
27write_files:
28  - path: /etc/apt/preferences.d/firefox
29    content: |
30      Package: firefox*
31      Pin: release o=LP-PPA-mozillateam
32      Pin-Priority: 501
33
34      Package: firefox*
35      Pin: release o=Ubuntu
36      Pin-Priority: -1
37  - path: /home/dave/.profile
38    append: true
39    defer: true
40    content: |
41      export MOZ_ENABLE_WAYLAND=1
42
43package_update: true
44package_upgrade: true
45package_reboot_if_required: true
46
47packages:
48  - kubuntu-desktop
49  - ubuntu-raspi-settings-desktop
50  - firefox

network-config

1network:
2  version: 2
3  ethernets:
4    eth0:
5      dhcp4: true
6      optional: true

[1]	I leave it for the reader to judge what this implies about GNOME’s drive for simplicity …

[2]

I’d started off this series doing everything “by the book” with Firefox from a snap, then after discovering that only the deb-based install supported this, wound up re-doing everything partly to ensure all the flavours had the best performance I could give them (full Wayland support if available without too much hacking), and secondly that they were on a level playing field (all snaps or all debs for Firefox).

The Ubuntu Unity Flavour

2023-08-04T00:00:00+01:00

See Also: The intro post which links to all the other flavour posts. Skip down here for the configurations used.

In this post, we’ll be looking at creating an Ubuntu Unity desktop with a cloud-init configuration, on a Raspberry Pi.

Contents

Ubuntu Unity
Verdict
Configuration

Ubuntu Unity

A blast from the past! This was one I didn’t manage to test last time, but I’ve a major soft spot for good ol’ Unity, having spent many years with it as my desktop on the PC back in the pre-Bionic days. So let’s give it a spin on the Pi!

For those unfamiliar with Unity, it has some quite different ideas of its own: the start menu and quick launcher are on the left of the screen (rather than the traditional top or bottom). If I recall correctly (and I’m sure someone will correct me in the comments if I’m not) this was because most screens have a wide-screen aspect ratio, so if you’re going to “waste” space with non-application stuff, better to do it horizontally than vertically (which sounds pretty reasonable to me).

In later iterations, and indeed in Unity 7 that we’re using here, it also borrows a few tid-bits from MacOS X: the window icons (minimize, maximize, close) are on the top left, and application menus appear in the bar at the top of the window rather than within the application window. Again, I think I’m right in stating this was (at least partially) in aid of not wasting vertical real-estate.

The selection of bundled applications was excellent. VLC for video playing, and Rhythmbox for music. Couldn’t ask for better. The file browser is the excellent Nemo from the Cinnamon desktop. Everything was generally reliable and worked … eventually.

Unfortunately, this brings us to the first major issue: performance. As with all the alternative desktops so far there’s no Wayland support yet, but even discounting that performance was painfully slow. I did note that, as under Budgie, cups-browsed was eating 100% of a CPU core again. But even after killing that, things still felt very sluggish, especially after the snappy responses of Lubuntu and Xubuntu.

Integration was also spotty. The menus for most things appeared in the top bar. But not Firefox (arguably the most important application on the desktop). One might argue that Firefox doesn’t have a menu bar any-more, that it’s succumbed to the modern obsession with hiding everything behind a “burger” button but the menus are still there: hit Alt+F and you’ll see (in fact several of the menus aren’t available from the “burger” button, which is probably why).

However, the screen-saver? Works! This is the only other flavour to, out of the box, reliably suspend my monitor and lock the desktop with no crashes (Lubuntu being the other).

Verdict

I’d really love to recommend Unity. Its interface ideas may be novel and quite different to most other flavours, but they make a good deal of sense (far more than GNOME’s do, to me anyway). However, while the reliability is great, the performance just isn’t good enough for use on a small platform.

The theme (whether light or dark) made it rather hard to distinguish window edges of inactive applications. Looking at the screenshot above, can you see where the file-browser ends, and the music player or the browser begins? There are also some strange edges to menu corners as if the environment wanted to draw shadows there, but forgot:

This all gave me an impression that things were not rendering “as intended”. Is there meant to be a shadow border around windows?

Configuration

For Ubuntu Unity, the configurations I used were as follows. For the boot configuration, highlighted lines are those changed from the lunar defaults:

config.txt

 1[all]
 2kernel=vmlinuz
 3cmdline=cmdline.txt
 4initramfs initrd.img followkernel
 5
 6[pi4]
 7max_framebuffers=2
 8arm_boost=1
 9
10[all]
11# Enable the audio output, I2C and SPI interfaces on the GPIO header. As these
12# parameters related to the base device-tree they must appear *before* any
13# other dtoverlay= specification
14dtparam=audio=on
15dtparam=i2c_arm=on
16dtparam=spi=on
17
18# Comment out the following line if the edges of the desktop appear outside
19# the edges of your display
20disable_overscan=1
21
22# If you have issues with audio, you may try uncommenting the following line
23# which forces the HDMI output into HDMI mode instead of DVI (which doesn't
24# support audio output)
25#hdmi_drive=2
26
27# Enable the serial pins
28#enable_uart=1
29
30# Autoload overlays for any recognized cameras or displays that are attached
31# to the CSI/DSI ports. Please note this is for libcamera support, *not* for
32# the legacy camera stack
33camera_auto_detect=1
34display_auto_detect=1
35
36# Config settings specific to arm64
37arm_64bit=1
38dtoverlay=dwc2
39
40[cm4]
41# Enable the USB2 outputs on the IO board (assuming your CM4 is plugged into
42# such a board)
43dtoverlay=dwc2,dr_mode=host
44
45[all]
46dtoverlay=vc4-kms-v3d

For the kernel command line, the entire file must consist of a single line of text so I would suggest simply copying this wholesale:

cmdline.txt

1zswap.enabled=1 zswap.zpool=z3fold zswap.compressor=zstd dwc_otg.lpm_enable=0 console=tty1 root=LABEL=writable rootfstype=ext4 rootwait fixrtc quiet splash

For the cloud-init configuration, the highlighted lines are those that you may wish to change after copying the content:

user-data

 1#cloud-config
 2
 3hostname: unity-pi
 4
 5locale: en_GB.UTF-8
 6timezone: Europe/London
 7
 8keyboard:
 9  model: pc105
10  layout: gb
11
12user:
13  name: "dave"
14  lock_passwd: false
15  gecos: "Dave Jones"
16  plain_text_passwd: raspberry
17  sudo: "ALL=(ALL:ALL) ALL"
18
19ssh_import_id:
20  - lp:waveform
21
22apt:
23  sources:
24    mozillateam:
25      source: 'ppa:mozillateam/ppa'
26
27write_files:
28  - path: /etc/apt/preferences.d/firefox
29    content: |
30      Package: firefox*
31      Pin: release o=LP-PPA-mozillateam
32      Pin-Priority: 501
33
34      Package: firefox*
35      Pin: release o=Ubuntu
36      Pin-Priority: -1
37  - path: /home/dave/.profile
38    append: true
39    defer: true
40    content: |
41      export MOZ_ENABLE_WAYLAND=1
42
43package_update: true
44package_upgrade: true
45package_reboot_if_required: true
46
47packages:
48  - ubuntu-unity-desktop
49  - ubuntu-raspi-settings-desktop
50  - firefox

network-config

1network:
2  version: 2
3  ethernets:
4    eth0:
5      dhcp4: true
6      optional: true

The Ubuntu Budgie Flavour

2023-08-03T00:00:00+01:00

See Also: The intro post which links to all the other flavour posts. Skip down here for the configurations used.

In this post, we’ll be looking at creating an Ubuntu Budgie desktop (with Pi-specific customizations!) with a cloud-init configuration.

Contents

Ubuntu Budgie
Verdict
Configuration

Ubuntu Budgie

The Budgie desktop is (was?) based on GNOME but with a more “traditional” desktop feel. The Ubuntu Budgie remix is styled with a brief nod towards MacOS X with a Dock-like launcher (Plank) centred at the bottom of the screen. However, that’s as far as the Mac-allusions go. There’s no application menu in the bar at the top, and the window icons are at the top-right instead of top-left.

Despite the GNOME heritage, there’s no Wayland support yet (although a bit of googling does suggest that it’s a high priority for the next release). On first login, I did need to select “Budgie Desktop” in the login manager (GDM) instead of the “Ubuntu Desktop” default, which was a little odd, but it turned out there was a reason for that which we’ll come back to later …

The specific package used to install the Ubuntu Budgie remix (ubuntu-budgie-desktop-raspi) also includes a nice little Pi-specific customization tool that launches on first login. However, I think some of its options may be redundant at this point (for instance, when using the “full” KMS overlay there’s no need to specify a GPU memory size, even if you are using a camera via the newer libcamera stack).

Most of the terminals in the various flavours are much of a muchness; all support multiple tabs, all have scroll-back, and all are essentially decent at their job. However, Budgie’s pick for a terminal (Tilix) stands out as being the only GTK-based terminal to support tiled panes, that I’m aware of [1]. Last time I tried it (in 2020) the terminal (which may or may not have been Tilix, I can’t recall) crashed pretty reliably, but this time around it was solid as any of the rest, and a stand-out feature of the environment.

Budgie seems to have a bit of a thing about tiling generally. The desktop’s got corner tiling by default (useful on a big monitor), but also has some fascinating shortcuts for ⅖-⅗ tiling, custom (shortcut activated) layouts, and arrangement of windows in arbitrarily sized grids. On my little monitor, I don’t really need anything more than side-by-side tiling but I can imagine this being very handy on large-scale high-DPI monitors.

Sadly, the reliability of the console wasn’t reflected elsewhere. Things felt a bit sluggish at times and eventually I figured out cups-browsed was sitting around eating 100% of one of the CPU cores. This might be a known issue (LP: #2018504), but that’s marked “fix released” on Lunar and this was running on a fully up-to-date system. I’ll need to try and reproduce it to see if that bug needs more attention.

Video playback also proved tricky to test; Budgie bundles the same default video player as Xubuntu, Parole, but this crashed the Budgie window manager process on the couple of occasions I tried it (it happily kept on playing the video, but the desktop behind flickered out of existence and then restarted!). The bundled Music player is Lollypop and, while it seems a capable player, it didn’t seem to have any support for any of the music sharing protocols supported on the home server, so I wound up mounting an NFS share for the purposes of testing. This worked, but it struggled a bit with the sheer quantity of albums, eating a good half a gig of RAM by the time it was done indexing. The audio output default also needed adjusting (as on the other desktops).

And the screen-saver? Didn’t appear to do anything at all at first (screen never blanked, no suspend, no lock). This turned out to be an issue with gnome-shell-common being installed. With that package purged (and … ubuntu-budgie-desktop along with it!), the screen-saver managed to lock the desktop correctly, but still failed to suspend the monitor (same as the official GNOME desktop). Interestingly, this also uninstalled the GDM3 login greeter, and replaced it with the slick greeter (familiar to users of Unity), which resolved having to select “Budgie Desktop” at login.

Still, this begs the question why gnome-shell-common is a transitive dependency of ubuntu-budgie-desktop (purging it does remove the meta-package)? Is there a reason for this, or is it a bug?

Verdict

It’s attractive (nice start menu!), reasonably responsive for a modern desktop, and has some novel applications (nice to see some Pi specific customizations). However, it needs a bit of tweaking out of the box for better reliability. I don’t like removing seeds that supposedly define the system!

The tiling capabilities look very interesting for those on large monitors, though.

Configuration

For Ubuntu Budgie, the configurations I used were as follows. For the boot configuration, highlighted lines are those changed from the lunar defaults:

config.txt

 1[all]
 2kernel=vmlinuz
 3cmdline=cmdline.txt
 4initramfs initrd.img followkernel
 5
 6[pi4]
 7max_framebuffers=2
 8arm_boost=1
 9
10[all]
11# Enable the audio output, I2C and SPI interfaces on the GPIO header. As these
12# parameters related to the base device-tree they must appear *before* any
13# other dtoverlay= specification
14dtparam=audio=on
15dtparam=i2c_arm=on
16dtparam=spi=on
17
18# Comment out the following line if the edges of the desktop appear outside
19# the edges of your display
20disable_overscan=1
21
22# If you have issues with audio, you may try uncommenting the following line
23# which forces the HDMI output into HDMI mode instead of DVI (which doesn't
24# support audio output)
25#hdmi_drive=2
26
27# Enable the serial pins
28#enable_uart=1
29
30# Autoload overlays for any recognized cameras or displays that are attached
31# to the CSI/DSI ports. Please note this is for libcamera support, *not* for
32# the legacy camera stack
33camera_auto_detect=1
34display_auto_detect=1
35
36# Config settings specific to arm64
37arm_64bit=1
38dtoverlay=dwc2
39
40[cm4]
41# Enable the USB2 outputs on the IO board (assuming your CM4 is plugged into
42# such a board)
43dtoverlay=dwc2,dr_mode=host
44
45[all]
46dtoverlay=vc4-kms-v3d

For the kernel command line, the entire file must consist of a single line of text so I would suggest simply copying this wholesale:

cmdline.txt

1zswap.enabled=1 zswap.zpool=z3fold zswap.compressor=zstd dwc_otg.lpm_enable=0 console=tty1 root=LABEL=writable rootfstype=ext4 rootwait fixrtc quiet splash

For the cloud-init configuration, the highlighted lines are those that you may wish to change after copying the content:

user-data

 1#cloud-config
 2
 3hostname: budgie-pi
 4
 5locale: en_GB.UTF-8
 6timezone: Europe/London
 7
 8keyboard:
 9  model: pc105
10  layout: gb
11
12user:
13  name: "dave"
14  lock_passwd: false
15  gecos: "Dave Jones"
16  plain_text_passwd: raspberry
17  sudo: "ALL=(ALL:ALL) ALL"
18
19ssh_import_id:
20  - lp:waveform
21
22apt:
23  sources:
24    mozillateam:
25      source: 'ppa:mozillateam/ppa'
26
27write_files:
28  - path: /etc/apt/preferences.d/firefox
29    content: |
30      Package: firefox*
31      Pin: release o=LP-PPA-mozillateam
32      Pin-Priority: 501
33
34      Package: firefox*
35      Pin: release o=Ubuntu
36      Pin-Priority: -1
37  - path: /home/dave/.profile
38    append: true
39    defer: true
40    content: |
41      export MOZ_ENABLE_WAYLAND=1
42
43package_update: true
44package_upgrade: true
45package_reboot_if_required: true
46
47packages:
48  - ubuntu-budgie-desktop-raspi
49  - ubuntu-raspi-settings-desktop
50  - firefox

network-config

1network:
2  version: 2
3  ethernets:
4    eth0:
5      dhcp4: true
6      optional: true

[1]	I specify GTK-based because that tittering you can hear is from KDE’s Konsole users, who’ve had this feature for yonks.

The Xubuntu Flavour

2023-08-02T00:00:00+01:00

See Also: The intro post which links to all the other flavour posts. Skip down here for the configurations used.

In this post, we’ll be looking at creating a Xubuntu desktop with a cloud-init configuration, on a Raspberry Pi.

Contents

Xubuntu
Verdict
Configuration

Xubuntu

Xubuntu is another lightweight flavour of Ubuntu (similar to Lubuntu in this respect), but based on XFCE (a GTK based environment). Subjectively, I couldn’t discern much difference in performance between Lubuntu and Xubuntu. Other than the positioning of the application bar at the top of the screen (compared to Lubuntu’s traditional bottom placement), things felt pretty similar.

On the functionality side of things, the incorrect audio output appeared as in Lubuntu, but was trivial to work around. Out of the box, music playback from the network worked thanks to the inclusion of the excellent Rhythmbox [1]. Video playback worked well with the Parole player, but unfortunately it didn’t seem able to playback from an SMB network share unless it was mounted.

This is one of those “integration” things; the file browser (Thunar), and most file-open dialogs, will happily browse to an SMB share and will appear to open the file. But then an error appears or, worse, nothing happens. Of course, the solution in this case is simple: sudo apt install vlc and it Just Works.

On the subject of bugs, remember I mentioned screen-savers? The lock-screen would happily kick in and blank the monitor, but wouldn’t suspend it (presumably LP: #1998716 again?). However, attempting to unlock (wiggling the mouse or hitting a key to bring up the login screen) gave the poor thing conniptions. The password prompt would briefly flicker into life, then the screen would blank again. Then flicker into life, then blank. Password entry wouldn’t work while all this was going on (something crashing?). After this happened several (4 or 5?) times, it seemed to get a grip on itself and allow login once more, but it’s rather broken! I’ll try and find some time to dig out some more useful info for a bug report (like whether this is specific to my monitor).

Everything else was pretty much as expected: I could read PDFs, print things, open links, browse the web, manipulate the file-system, use the terminal, all without issue, and all reasonably smoothly.

One amusing stand-out I did notice: the choice of bundled desktop games! On most desktops, this is pretty bland fare. Some variation of Klondike solitaire, maybe a Minesweeper clone, perhaps a Sudoku implementation. But not here! No, Xubuntu bundles one single game package [2], but it’s packed with more fiendishly fun entertainment than all the other default desktop gaming packages put together: Simon Tatham’s Puzzles (if you haven’t played these before, you owe it to yourself to sudo apt install sgt-launcher and give them a whirl).

Verdict

It’s pretty (rather more so that Lubuntu in my opinion), lightweight, responsive and reasonably functional. However, it’s got a few more bugs than Lubuntu (the screen-saver particularly) so given the choice I’d probably go with Lubuntu for a light-weight choice. Still … a brilliant selection of games!

Configuration

For Xubuntu, the configurations I used were as follows. For the boot configuration, highlighted lines are those changed from the lunar defaults:

config.txt

 1[all]
 2kernel=vmlinuz
 3cmdline=cmdline.txt
 4initramfs initrd.img followkernel
 5
 6[pi4]
 7max_framebuffers=2
 8arm_boost=1
 9
10[all]
11# Enable the audio output, I2C and SPI interfaces on the GPIO header. As these
12# parameters related to the base device-tree they must appear *before* any
13# other dtoverlay= specification
14dtparam=audio=on
15dtparam=i2c_arm=on
16dtparam=spi=on
17
18# Comment out the following line if the edges of the desktop appear outside
19# the edges of your display
20disable_overscan=1
21
22# If you have issues with audio, you may try uncommenting the following line
23# which forces the HDMI output into HDMI mode instead of DVI (which doesn't
24# support audio output)
25#hdmi_drive=2
26
27# Enable the serial pins
28#enable_uart=1
29
30# Autoload overlays for any recognized cameras or displays that are attached
31# to the CSI/DSI ports. Please note this is for libcamera support, *not* for
32# the legacy camera stack
33camera_auto_detect=1
34display_auto_detect=1
35
36# Config settings specific to arm64
37arm_64bit=1
38dtoverlay=dwc2
39
40[cm4]
41# Enable the USB2 outputs on the IO board (assuming your CM4 is plugged into
42# such a board)
43dtoverlay=dwc2,dr_mode=host
44
45[all]
46dtoverlay=vc4-kms-v3d

For the kernel command line, the entire file must consist of a single line of text so I would suggest simply copying this wholesale:

cmdline.txt

1zswap.enabled=1 zswap.zpool=z3fold zswap.compressor=zstd dwc_otg.lpm_enable=0 console=tty1 root=LABEL=writable rootfstype=ext4 rootwait fixrtc quiet splash

For the cloud-init configuration, the highlighted lines are those that you may wish to change after copying the content:

user-data

 1#cloud-config
 2
 3hostname: xubuntu-pi
 4
 5locale: en_GB.UTF-8
 6timezone: Europe/London
 7
 8keyboard:
 9  model: pc105
10  layout: gb
11
12user:
13  name: "dave"
14  lock_passwd: false
15  gecos: "Dave Jones"
16  plain_text_passwd: raspberry
17  sudo: "ALL=(ALL:ALL) ALL"
18
19ssh_import_id:
20  - lp:waveform
21
22apt:
23  sources:
24    mozillateam:
25      source: 'ppa:mozillateam/ppa'
26
27write_files:
28  - path: /etc/apt/preferences.d/firefox
29    content: |
30      Package: firefox*
31      Pin: release o=LP-PPA-mozillateam
32      Pin-Priority: 501
33
34      Package: firefox*
35      Pin: release o=Ubuntu
36      Pin-Priority: -1
37  - path: /home/dave/.profile
38    append: true
39    defer: true
40    content: |
41      export MOZ_ENABLE_WAYLAND=1
42
43package_update: true
44package_upgrade: true
45package_reboot_if_required: true
46
47packages:
48  - xubuntu-desktop
49  - ubuntu-raspi-settings-desktop
50  - firefox

network-config

1network:
2  version: 2
3  ethernets:
4    eth0:
5      dhcp4: true
6      optional: true

[1]

Rhythmbox is probably my favourite part of the entire GNOME project. It’s genuinely superb: supports every protocol and format I’ve ever needed (and several I haven’t), has a UI that’s minimal but fantastically functional, and most of all fast. Want to navigate your way through ~8000 songs spread over ~500 albums? No problem, even on a Pi! It’s one of the four applications that I reliably fire up every single day.

[2]	That I could see; the seed seems to suggest it ships the GNOME implementations of Mines and Sudoku too but they didn’t appear in the start menu.

The Lubuntu Flavour

2023-08-01T00:00:00+01:00

See Also: The intro post which links to all the other flavour posts. Skip down here for the configurations used.

In this post, we’ll be looking at creating a Lubuntu desktop with a cloud-init configuration, on a Raspberry Pi.

Contents

Lubuntu
Verdict
Configuration

Lubuntu

Lubuntu is a lightweight flavour of Ubuntu, based on LXQt (which is the desktop environment that RaspiOS is based upon too). It’s certainly the lightest in terms of resource usage from all the desktops I’m comparing, and one of the most responsive too.

It’s a bit prettier than when I last saw it 3 years ago, but still doesn’t have the polish of the “full” desktop suites like GNOME or KDE. The interface feels very responsive but it’s notable that there’s no Wayland support yet so the performance of graphically heavy things is not entirely smooth.

One issue immediately jumped out, which was the incorrect selection of the default audio output (the 3.5mm jack output which wasn’t connected, instead of HDMI). This is a known issue (LP: #1993347), and one that occurs on almost every other flavour too. Thankfully it’s a trivial work-around (just pick the HDMI output instead) and the selection persists across reboots, so it’s a one-time fix.

The applications on Lubuntu are understandably minimal, but surprisingly good! Most desktop environments try and bundle “native” applications for specific purposes, like a video player. By “native” I mean applications that stick strictly within that desktop’s base toolkit (whether that’s GTK or Qt, and their corresponding media bits), so that the distribution of that desktop doesn’t bloat itself out with additional libraries (or whole frameworks) for a single application. This means you’ll see things like the Parole or Totem video players … right before you open a command line, sudo apt install vlc, and never look back.

Lubuntu, by contrast, just installs VLC saving you a whole step! That said, VLC is all you get for media playback here and, capable though VLC is with music, it’s not set up as an album browser. I’m too used to having something to do just that. The distro seed giveth, and the distro seed … leaveth stuff out.

There were a couple of minor oddities. The two partitions on the SD card show up as “removable” (which they fairly obviously … aren’t) on the system tray widget. After one system upgrade and a reboot, the wallpaper disappeared inexplicably. However, the screen-saver worked perfectly. Locked the screen and put the monitor in power-saving mode. Why do I mention this? Firstly, I’ve had a long-standing issue with the official GNOME desktop not putting my monitor into power-saving mode (LP: #1998716). Secondly … screen-saver breakage became a bit of a theme the further I went on with this series …

Otherwise, things mostly worked well. The included file browser is PCManFM, which is capable (and fast!). It’ll also be familiar to users of RaspiOS as it’s the default file browser there too. No issues opening things, including over unmounted network shares, no problems printing or scanning over the network, and Firefox worked as normal.

Verdict

It’s not the prettiest, but it is lightweight, responsive, and has a nice choice of default apps. In practice it feels functional enough that I’d be happy to use it as my desktop, but I’d definitely miss the ability to easily “tile” applications (maybe there’s a shortcut for this that I didn’t find!).

Configuration

For Lubuntu, the configurations I used were as follows. For the boot configuration, highlighted lines are those changed from the lunar defaults:

config.txt

 1[all]
 2kernel=vmlinuz
 3cmdline=cmdline.txt
 4initramfs initrd.img followkernel
 5
 6[pi4]
 7max_framebuffers=2
 8arm_boost=1
 9
10[all]
11# Enable the audio output, I2C and SPI interfaces on the GPIO header. As these
12# parameters related to the base device-tree they must appear *before* any
13# other dtoverlay= specification
14dtparam=audio=on
15dtparam=i2c_arm=on
16dtparam=spi=on
17
18# Comment out the following line if the edges of the desktop appear outside
19# the edges of your display
20disable_overscan=1
21
22# If you have issues with audio, you may try uncommenting the following line
23# which forces the HDMI output into HDMI mode instead of DVI (which doesn't
24# support audio output)
25#hdmi_drive=2
26
27# Enable the serial pins
28#enable_uart=1
29
30# Autoload overlays for any recognized cameras or displays that are attached
31# to the CSI/DSI ports. Please note this is for libcamera support, *not* for
32# the legacy camera stack
33camera_auto_detect=1
34display_auto_detect=1
35
36# Config settings specific to arm64
37arm_64bit=1
38dtoverlay=dwc2
39
40[cm4]
41# Enable the USB2 outputs on the IO board (assuming your CM4 is plugged into
42# such a board)
43dtoverlay=dwc2,dr_mode=host
44
45[all]
46dtoverlay=vc4-kms-v3d

For the kernel command line, the entire file must consist of a single line of text so I would suggest simply copying this wholesale:

cmdline.txt

1zswap.enabled=1 zswap.zpool=z3fold zswap.compressor=zstd dwc_otg.lpm_enable=0 console=tty1 root=LABEL=writable rootfstype=ext4 rootwait fixrtc quiet splash

For the cloud-init configuration, the highlighted lines are those that you may wish to change after copying the content:

user-data

 1#cloud-config
 2
 3hostname: lubuntu-pi
 4
 5locale: en_GB.UTF-8
 6timezone: Europe/London
 7
 8keyboard:
 9  model: pc105
10  layout: gb
11
12user:
13  name: "dave"
14  lock_passwd: false
15  gecos: "Dave Jones"
16  plain_text_passwd: raspberry
17  sudo: "ALL=(ALL:ALL) ALL"
18
19ssh_import_id:
20  - lp:waveform
21
22apt:
23  sources:
24    mozillateam:
25      source: 'ppa:mozillateam/ppa'
26
27write_files:
28  - path: /etc/apt/preferences.d/firefox
29    content: |
30      Package: firefox*
31      Pin: release o=LP-PPA-mozillateam
32      Pin-Priority: 501
33
34      Package: firefox*
35      Pin: release o=Ubuntu
36      Pin-Priority: -1
37  - path: /home/dave/.profile
38    append: true
39    defer: true
40    content: |
41      export MOZ_ENABLE_WAYLAND=1
42
43package_update: true
44package_upgrade: true
45package_reboot_if_required: true
46
47packages:
48  - lubuntu-desktop
49  - ubuntu-raspi-settings-desktop
50  - firefox

network-config

1network:
2  version: 2
3  ethernets:
4    eth0:
5      dhcp4: true
6      optional: true

We Serve All Flavours!

2023-07-31T00:00:00+01:00

See Also: The prior post in which we learned about the wonders of automated configuration with cloud-init.

It’s that time of the decade, again: desktop round-up time! As of writing this introductory paragraph, I’ve now finished poking, prodding, and reviewing each of the Ubuntu flavours. However, the article turned into yet another small-novella sized piece and the only sensible way I can think to publish this is by making this one an introductory piece, with a few final cloud-init hints, and an index to the other articles which will be published once a day over the coming week (and a bit).

My apologies, dear reader, for keeping you on tenterhooks again …

Contents

Adding Sprinkles
99 Flakes
The Taste Test
The Flavours

Adding Sprinkles

In the interests of “integration”, and because I wish to demonstrate yet more capabilities of cloud-init, I’m going to pre-emptively work around a long-standing bug (LP: #1974064) which I frequently run into, preventing links in PDFs from opening in the browser. But only if that browser is a snap … as Firefox is. Furthermore (without giving too much away from future posts in this series) there still appear to be some integration issues between the snapped Firefox and Wayland based environments in some of the desktops. So, in the interests of a level playing field for everyone we’re going to get cloud-init to install Firefox from a deb on first boot.

We need to add an apt source (the Firefox stable PPA), and add some configuration to pin its items higher than the archive’s. Thankfully, this is pretty simple to derive from all those guides out there showing how to do this on an already running system:

apt:
  sources:
    mozillateam:
      source: 'ppa:mozillateam/ppa'

write_files:
  - path: /etc/apt/preferences.d/firefox
    content: |
      Package: firefox*
      Pin: release o=LP-PPA-mozillateam
      Pin-Priority: 501

      Package: firefox*
      Pin: release o=Ubuntu
      Pin-Priority: -1

Now we can ditch that weird snap section in our configuration, and just stick firefox in the list of packages to install.

One other tip I’m going to add here, but which I won’t be adding to the experimental configurations as published, because the vast majority of people won’t want this, is configuration for an apt proxy (specifically apt-cacher-ng). To be clear, for most people I don’t recommend running this (a couple of times a month it sends my server into a bit of a tizz and needs kicking), but if you’re going to be repeatedly installing the same large packages (as I frequently do for packaging work, or in this case when installing several desktop packages which have similar bases) it can save a huge amount of time and bandwidth. Anyway, this is what the relevant snippet looks like:

apt:
  conf: |
    Acquire::http { Proxy "http://myproxy.local:3142"; }

Obviously, replace myproxy.local with the address of your local proxy.

Below is the full configuration we’ll be using (minus the apt-cacher configuration), with a fairly obvious placeholder for the desktop package. I’ll publish full configurations (including the config.txt, cmdline.txt, and network-config, largely derived from the prior post) in each section in case you want to play along at home. Just place these on the boot partition of an Ubuntu Server for Pi (23.04 or later) card [1] and adjust the highlighted lines to your tastes:

user-data

 1#cloud-config
 2
 3hostname: miss-piggy
 4
 5locale: en_GB.UTF-8
 6timezone: Europe/London
 7
 8keyboard:
 9  model: pc105
10  layout: gb
11  options: ctrl:nocaps
12
13user:
14  name: "dave"
15  lock_passwd: false
16  gecos: "Dave Jones"
17  plain_text_passwd: raspberry
18  sudo: "ALL=(ALL:ALL) ALL"
19
20ssh_import_id:
21  - lp:waveform
22
23apt:
24  sources:
25    mozillateam:
26      source: 'ppa:mozillateam/ppa'
27
28write_files:
29  - path: /etc/apt/preferences.d/firefox
30    content: |
31      Package: firefox*
32      Pin: release o=LP-PPA-mozillateam
33      Pin-Priority: 501
34
35      Package: firefox*
36      Pin: release o=Ubuntu
37      Pin-Priority: -1
38  - path: /home/dave/.profile
39    append: true
40    defer: true
41    content: |
42      export MOZ_ENABLE_WAYLAND=1
43
44package_update: true
45package_upgrade: true
46package_reboot_if_required: true
47
48packages:
49  - YOUR-DESKTOP-PACKAGE-HERE
50  - ubuntu-raspi-settings-desktop
51  - firefox

Bear in mind your first boot is going to install a lot of things, and it will take at least an hour (more like an hour and a half) even on a good A1-compliant SD card.

99 Flakes

One wrinkle I forgot to mention in the prior post is that of networking. The server images we’re basing this on use netplan, and start off by having it “render” the network configuration via networkd (part of systemd).

However, every Ubuntu desktop flavour ultimately winds up using Network Manager for its network configuration. While netplan is capable of rendering a configuration via Network Manager, that’s probably not what you ultimately want as your network configuration wouldn’t be editable via the Network Manager GUI (only the netplan YAML configuration files).

Why not change the initial netplan configuration to render through Network Manager instead? This won’t work because at first boot, the card is just Ubuntu Server for Pi and doesn’t contain Network Manager. That only gets installed along with the chosen desktop flavour, but that act requires a network to already be configured. Dependency cycles!

The fix is simple enough. Once the desktop has booted, and you’re all snugly logged in for the first time, run the following to delete the generated netplan configuration and re-apply the (now empty) settings:

$ sudo rm /etc/netplan/50-cloud-init.yaml
$ sudo netplan apply

At this point networkd will stop “managing” the interfaces and Network Manager will take them over. In the case of a simple DHCP-based Ethernet connection, it should reconfigure automatically and you’ll be back up and running in seconds. However, if you performed your initial set up over WiFi you’ll now need to go into the GUI network settings for your flavour (wherever that is — it can usually be found from the system-tray though) and re-enter your WiFi credentials.

This is one of the ways that this Frankenstein-like desktop setup differs from the official desktop releases. There are a few others (dual seeds, user creation differences, root-fs sizing, probably a few others that I’ve forgotten!), and if anyone’s interested I can go through them, but this is the only one that’s affected the use of the desktop during my explorations.

The Taste Test

What should I look for in a desktop? I can’t define this universally, so I’ll just go with what works for me. My personal workflow is probably a strange mix of deeply anachronistic, and semi-modern. On the one hand, I spend the vast majority of my time in a terminal window. On the other, I eschew office applications in favour of doing office-y work in a web-browser. Google Sheets for any spreadsheet work (though I have been known to dip into sc for trivial bits), and when I absolutely have to touch a word-processor [2], Google Docs.

I could almost get away with working on a Chromebook, but not quite. If I have PDFs to deal with (and between Canonical’s hiring push, and various hardware data-sheets, I get through quite a few of these), I use an honest-to-goodness standalone PDF reader, instead of the (usually horribly slow) built-in PDF readers in browsers. Sometimes I even … brace yourself … print them [3]!

I also play most of my music, served from the home server, in a standalone application, and the same goes for most films I watch (there are some good web-based systems for this, but browsers are heavy and I work on a Pi so I tend to try and limit my use of them to text-based duties). Most of my file manipulation is done from the command line, but sometimes for simpler stuff (especially involving external storage) I’ll resort to the file browser.

So, I’ll be looking for:

A decent terminal — this is pretty much a given; it’d hardly be a Linux distro without a decent terminal!
A decent browser — again, pretty much a given as we’ll be installing Firefox anyway.
Some decent media applications — music player, video player, a standalone PDF viewer, preferably a good file browser for managing it all.
Integration — I want to be able to click a link from my terminal, or in a PDF, and have it open in the web-browser. I’d like to be able to open files, archives, and videos straight from the file-browser, even from network shares (ideally without messing around with network mounts). The home server has all sorts of services for this (NFS, SMB, DLNA, etc. all advertised over avahi) so there should be plenty of opportunity here. I also want to be able to print stuff without jumping through tons of hoops (again, the printers here are network-accessible and advertised over avahi).
Performance — it’s a Pi, so I can’t expect the world here, but the video player should handle 720p or 1080p sources in full-screen, smoothly. The music player should play without stuttering, and generally things shouldn’t feel (too) sluggish. The browser is an exception here: modern browsers are ridiculously (over-)complex beasts and I’ll allow some lee-way for this.
Reliability — The more bugs I run into, the more points I’ll dock. Erm … are we doing points? Well, any bugs I run into I’ll point out. And possibly report, if I can figure out what to file ‘em against!

Now, with all that extra gubbins out of the way, on with the tests!

The Flavours

These are the flavours I’m intending to cover, which will become links to the relevant article:

Lubuntu — the LXQt flavour
Xubuntu — the XFCE flavour
Ubuntu Budgie — the Budgie flavour
Ubuntu Unity — the Unity flavour
Kubuntu — the KDE flavour
Ubuntu Studio — the creator’s flavour
Ubuntu — comparison to the official GNOME-based image

[1]	Side note: theoretically this should work on either armhf or arm64, however I’m only testing arm64 here as it’s all we officially support on the desktop images anyway.

[2]

In case it’s not obvious, I am firmly of the opinion that WYSIWYG is the work of the devil, and the One True Way is that of the markup language, beautifully processed by 100% natural, hand-crafted, hypo-allergenic, non-genetically modified, organically-certified algorithms into whatever output format you may prefer.

[3]	I have rarely felt as old as I did a few weeks ago when a younger colleague casually inquired, “does anyone still have a printer?” Yes! In my utter extravagance I even have two. One of them does colour!

Desktop on a Stick

2023-06-30T00:00:00+01:00

Updated to include the per-release quirks channel for the snap.

TL;DR: Skip down here for the final configurations.

It’s that time of the decade, again: desktop round-up time! But thanks to a genius notion from Oliver (one of our product managers), I’m going to be doing it a bit differently. Previously, my procedure for checking out the various Ubuntu desktops on a Raspberry Pi looked something like this:

Flash a server image (go make coffee)
Boot the server image (twiddle thumbs for a minute or two while it sorts out the first time setup)
apt install lubuntu-desktop (go make more coffee)
Reboot, play with it a bit, install a bunch of stuff (yet more coffee)

To avoid the inevitable caffeine poisoning that results from repeating this procedure for Lubuntu, Xubuntu, Kubuntu, and Budgie, this time we’re going to use … drum roll please …

The (original) cloud-init logo with the words “cloud-init” adjacent. The logo is the usual Ubuntu-orange circle, within which appears a very geometrically simplified representation of the fingers of the left hand. The index finger is extended and appears to be touching the edge of a large white spark (which in no way suggests someone electrocuting themselves by touching a live surface). Or it could be a worm with a large anime hair-do. I’m not sure.

If that sounds like a bit of an anti-climax, that’s just because you haven’t played with cloud-init enough. Yes, its one-shot nature makes it painful to iterate when you get things wrong. Yes, the documentation is painfully obscure in places. However, once you get a grip on it, it’s quite astonishingly flexible. Hopefully, by reading this you can bypass some of that pain and start from a more useful point!

But before we get to the fun bits, we need to get the basics done …

Ingredients

The first step is to get a Pi to test with. You’ll need a Pi 4, Pi 400, or CM4 with at least 4GB of RAM. At the time of writing, that’s a slightly tricky proposition given the supply shortages. The vital rpilocator can help here, but it would appear (at least according to Eben, who probably knows what he’s talking about!) that the shortage should begin to clear during the second half of 2023.

The second step is to get an image we can play with. I’m going to be using the recently released Ubuntu 23.04 (Lunar Lobster) Server for Raspberry Pi image, specifically the arm64 variant. You can use rpi-imager to flash this to a spare SD card, or even SSD drive attached to a USB adapter.

Warning

A word of caution: don’t try this with Jammy (22.04). There’s a reason I’m using Lunar (23.04) for this experiment. If you don’t care about the reasons, skip to the next section. Otherwise, read on …

Historically, Ubuntu images (including the Ubuntu for Raspberry Pi images) have been built with a system called livecd-rootfs. This could charitably be described as a rough collection of dozens of hooks (and hacks) in the form of shell scripts, all of them plastered onto the already crufty base of livecd-rootfs, itself a gigantic mess of shell script.

The result is exactly the hellish nightmare of obscure regex-loaded nonsense that you might imagine. Worse still, many of the hacks inject configuration files into the resulting image that are unowned. That is to say, no Debian package owns these injected files, and that causes all sorts of fun when it comes to upgrades.

My friend William, before he left, made a valiant start on sorting out this mess by re-writing the old ubuntu-image tool. In concert with this effort, my small contribution was to start migrating hacks from livecd-rootfs into the ubuntu-settings package so that in Lunar (and beyond), as much configuration as possible would be directly under the control of the package management system (this contribution made a screeching U-turn when, at the last minute, it was decided we were going to use livecd-rootfs for lunar after all, but that’s another story!).

This is one of the major reasons it’s now much simpler to spin up a desktop image with cloud-init. Instead of having a write a ton of configuration to add all the necessary tweaks livecd-rootfs used to do, we can just include one extra package and (mostly) be done.

Preparing your kitchen

Once your SD card (or drive) is flashed, eject it, then re-insert it. Regardless of the OS you’re using (Windows, Mac OS, Linux), you should see the boot partition (labelled “system-boot”) appear [1]. We’re going to be tweaking several configuration files, specifically:

config.txt — this is the Pi bootloader configuration
cmdline.txt — this is the Linux kernel command line
user-data — this is the main cloud-init configuration file
network-config — this is the cloud-init network configuration

We’ll start with config.txt because the changes here are universal: they’ll be required no matter which desktop we’re intending to use:

Find the enable_uart=1 line (it’s on line 28 by default) and comment it out by inserting # at the start of the line. This simply disables the serial console which isn’t strictly necessary, but if the serial console is enabled, the Pi’s GPU clock is locked to low speed and obviously we want the GPU clock to float higher for our desktop.
At the end of the file add dtoverlay=vc4-kms-v3d to load the “full” KMS device-tree overlay (it should be in the [all] section at the end of the file)

Next we need to tweak the kernel command line in cmdline.txt. Again, all the changes here are common to all desktops. This file consists of a single line of text; don’t introduce any line breaks when editing it:

Delete console=serial0,115200 at the start. This just removes the tty associated with the serial console which we disabled above in config.txt.
Insert zswap.enabled=1 zswap.zpool=z3fold zswap.compressor=zstd at the start. This activates the zswap system that we use on the desktop.

These changes more or less boil down to the base differences between the Ubuntu Pi server and desktop images. Most of the other differences (since Lunar, as discussed above) come down to the selection of packages that are installed …

Picking your flavour

It’s time to learn what we can do with cloud-init. The most obvious thing we want is to have a particular desktop package installed. This is pretty simple, and achieved by adding the following block to the user-data file on the boot partition:

packages:
  - lubuntu-desktop
  - ubuntu-raspi-settings-desktop

This tells cloud-init that, on first boot, we want it to try and install two extra packages:

lubuntu-desktop is the basis of the Lubuntu desktop images; we’ll substitute other desktop packages here later
ubuntu-raspi-settings-desktop contains all the little tweaks for the Pi desktop (ensuring there’s a swap-file, disabling suspend in Gnome, inserting the correct modules for zswap into the initrd, setting appropriate permissions for I2C and SPI interfaces, etc.)

In addition to these we should also add the following settings (these aren’t mandatory, but highly recommended):

package_update: true
package_upgrade: true
package_reboot_if_required: true

The first line tells cloud-init we want it to run apt update to refresh the set of available packages. This is the most important line as, without it, cloud-init will likely fail at some point because the apt index shipped on the image will be out of date. The next line demands that apt upgrade should be run to ensure all packages are up to date. The final line indicates that, if any package indicates a reboot is required, we should carry one out (this is absolutely the case when installing a desktop, as a display manager will be installed at the very least).

Next, while some of the desktops available install a browser by default (by depending on the Firefox apt package, which in turns installs the Firefox snap), several don’t as their official images include the Firefox (or Chromium) snap directly without the apt “wrapper”. To avoid winding up with a desktop without a browser, we’ll add another section to install the Firefox snap explicitly:

snap:
  commands:
    - snap install --channel=latest/stable/ubuntu-23.04 firefox

I’m not entirely sure why the snap section is so different from packages, or why the full snap install <package> form is required. It does appear from the documentation that the snap.commands section simply allows arbitrary commands to be specified (canonical-livepatch is included in the examples), but in that case why call it a snap section?

The exceedingly long channel name causes the snap to follow a branch of a channel that may include per-release quirks. Or … something like that. I’m still not clear on the whole “channel” concept in snaps.

Locale-ly sourced

On the official Ubuntu Desktop for Raspberry Pi images, we have a first time setup process [2] which guides the user through locale selection and initial user creation. We don’t have that here, so we’d better get cloud-init to handle that instead. Firstly the locale, timezone, and keyboard settings:

locale: en_GB.UTF-8
timezone: Europe/London
keyboard:
  model: pc105
  layout: gb
  options: ctrl:nocaps

Note

You may want to leave off that “options:” line. I include it to put Ctrl in the right place (i.e. where Caps Lock is) and because I have no need of a Caps Lock key.

To find out the valid values for these settings, you can use the following commands on an existing Ubuntu desktop (I say desktop specifically because the available locales on server installations tend to be extremely minimal) [3]:

locale:

localectl list-locales

timezone:

timedatectl list-timezones

keyboard:

model:: localectl list-x11-keymap-models
layout:: localectl list-x11-keymap-layouts
variant:: localectl list-x11-keymaps-variants [layout]
options:: localectl list-x11-keymaps-option

Adding personality

Next we need to create, and customize, the initial user. By default, cloud-init will create a user named ubuntu with a locked password, which also has password-less sudo access, and a bunch of default group memberships (for things like direct video and audio access, etc). The default group memberships are fine, but I’d like a slightly more personalized username, and one with password-required sudo rights. This can be accomplished with the following block:

user:
  name: "dave"
  gecos: "Dave Jones"
  plain_text_passwd: "raspberry"
  lock_passwd: false
  sudo: "ALL=(ALL:ALL) ALL"
  ssh_import_id:
    - lp:waveform

This changes the default username to “dave”, and uses my name in the gecos field (which traditionally recorded a whole bunch of things like display name, office number, telephone number, and so forth but these days tends to just be used for the display name). The sudo setting specifically lacks the NOPASSWD flag, requiring a password for use. The ssh_import_id setting lists users from which to import public SSH keys (use “lp:” prefix for Launchpad usernames, and “gh:” for Github).

The block also sets the initial password to the plain-text string “raspberry” and ensures password login is possible (lock_passwd is true by default which means the only way to login is with SSH or another auxiliary mechanism, but that’s not so useful on the desktop!). This is horribly insecure! We could do something slightly more secure here by providing a hashed password instead. First we’d run mkpasswd to generate the hash:

$ mkpasswd --method=SHA-512 --rounds=4096
Password:
$6$rounds=4096$JZRfK1tM.xiWZtR5$XpMvuj2reJr.....lI0T4Z/

Then we would use this with hashed_passwd instead of plain_text_passwd:

user:
  name: "dave"
  gecos: "Dave Jones"
  hashed_passwd: "$6$rounds=4096$JZRfK1tM.xiWZtR5$XpMvuj2reJr.....lI0T4Z/"
  lock_passwd: false
  sudo: "ALL=(ALL:ALL) ALL"
  ssh_import_id:
    - lp:waveform

However, don’t be lulled into a false sense of security. As the cloud-init documentation notes:

While hashed_password is better than plain_text_passwd, using [a password] in user-data represents a security risk as user-data could be accessible by third-parties depending on your cloud platform

To be clear: user-data is a world-readable file on the Ubuntu Pi images (it has to be because it’s on a FAT partition). There is no absolutely secure method of setting the password via cloud-init (at least on the Ubuntu Pi images); the secure thing to do here is login and then change your password. Consider this password initial and temporary.

Baking in hacks

Finishing off our user-data file we’re going to make one final rather hacky (but useful) change. At the time of writing, Firefox starts under the XWayland layer. This leads to rather poor performance under certain circumstances. To force it to load under Wayland directly, we’d need to set the MOZ_ENABLE_WAYLAND environment variable. Ideally we would do this only in our user’s environment, not system wide. This could be done by appending a line to our user’s ~/.profile script. However, this doesn’t exist initially (and won’t until cloud-init creates our user). Thankfully, this presents no problem!

write_files:
  - path: /home/dave/.profile
    defer: true
    append: true
    content: |
      export MOZ_ENABLE_WAYLAND=1

The powerful (but obviously slightly dangerous) write_files key allows us to write to arbitrary files on first boot. Here we specify we’d like to write to /home/dave/.profile (adjust for your custom username as necessary). We also inform cloud-init that the write should be deferred which means this will take place after user creation. Further, we specify that we wish to append (not overwrite), and finally we provide the lines we’d like to append.

Going shopping

Naturally, in order to obtain the packages we’ve requested we’re going to need some network connectivity. If your Pi is going to be connected by Ethernet, you’re good to go; the default configuration will just use your Ethernet connection. However, if you’re going to be relying on WiFi instead we need to be a bit of surgery in the last file we mentioned earlier, network-config.

Firstly we’ll delete the ethernets section, then uncomment the wifis section and specify the local access point and password:

network:
  version: 2
  wifis:
    wlan0:
      dhcp4: true
      optional: true
      access-points:
        "my-wifi-ssid":
          password: "my-wifi-password"

It is also useful to specify the regulatory-domain here (note that this needs to be specified under the wlan0 key, which is the reason for the inclusion of the parent keys below):

network:
  wifis:
    wlan0:
      regulatory-domain: GB

The valid regulatory domains can be found in the Linux kernel source Specifically, the two-letter code after the “country” line, which are almost entirely the ISO-3166 standard two-letter country codes. The special “00” domain (the “world regulatory domain”) is the default, but relying on this can lead to poor WiFi performance, particularly in 5GHz setups where many channels have restrictions in the world domain.

The first byte is with the eye

At the end of all this tinkering, you should have files that look something like the following on your boot partition (lines you may wish to pay specific attention to are highlighted):

config.txt

 1[all]
 2kernel=vmlinuz
 3cmdline=cmdline.txt
 4initramfs initrd.img followkernel
 5
 6[pi4]
 7max_framebuffers=2
 8arm_boost=1
 9
10[all]
11# Enable the audio output, I2C and SPI interfaces on the GPIO header. As these
12# parameters related to the base device-tree they must appear *before* any
13# other dtoverlay= specification
14dtparam=audio=on
15dtparam=i2c_arm=on
16dtparam=spi=on
17
18# Comment out the following line if the edges of the desktop appear outside
19# the edges of your display
20disable_overscan=1
21
22# If you have issues with audio, you may try uncommenting the following line
23# which forces the HDMI output into HDMI mode instead of DVI (which doesn't
24# support audio output)
25#hdmi_drive=2
26
27# Enable the serial pins
28#enable_uart=1
29
30# Autoload overlays for any recognized cameras or displays that are attached
31# to the CSI/DSI ports. Please note this is for libcamera support, *not* for
32# the legacy camera stack
33camera_auto_detect=1
34display_auto_detect=1
35
36# Config settings specific to arm64
37arm_64bit=1
38dtoverlay=dwc2
39
40[cm4]
41# Enable the USB2 outputs on the IO board (assuming your CM4 is plugged into
42# such a board)
43dtoverlay=dwc2,dr_mode=host
44
45[all]
46dtoverlay=vc4-kms-v3d

cmdline.txt

1zswap.enabled=1 zswap.zpool=z3fold zswap.compressor=zstd dwc_otg.lpm_enable=0 console=tty1 root=LABEL=writable rootfstype=ext4 rootwait fixrtc quiet splash

Please note that cmdline.txt must consist of a single line of text. Ignore any wrapping of text above.

user-data

 1#cloud-config
 2
 3hostname: lubuntu
 4
 5locale: en_GB.UTF-8
 6timezone: Europe/London
 7
 8keyboard:
 9  model: pc105
10  layout: gb
11  options: ctrl:nocaps
12
13user:
14  name: "dave"
15  lock_passwd: false
16  gecos: "Dave Jones"
17  plain_text_passwd: "raspberry"
18  sudo: "ALL=(ALL:ALL) ALL"
19  ssh_import_id:
20    - lp:waveform
21
22write_files:
23  - path: /home/dave/.profile
24    append: true
25    defer: true
26    content: |
27      export MOZ_ENABLE_WAYLAND=1
28
29package_update: true
30package_upgrade: true
31package_reboot_if_required: true
32
33packages:
34  - lubuntu-desktop
35  - ubuntu-raspi-settings-desktop
36
37snap:
38  commands:
39    - snap install firefox

You may be wondering if the ordering of these blocks matters. The answer is “no”. The ordering of actions in cloud-init does not depend on the ordering of the configuration file (and nor should it). The system is intended to be declarative; we are describing the state we wish to attain, not how to obtain it.

This is why the write_files section has options like defer which adjust the timing of these actions. By default, write_files entries will always occur early on so that written files can be available to other actions.

network-config

 1network:
 2  version: 2
 3  wifis:
 4    wlan0:
 5      regulatory-domain: GB
 6      dhcp4: true
 7      optional: true
 8      access-points:
 9        "my-wifi-ssid":
10          password: "my-wifi-password"

With all those written, safely eject your card (or SSD drive), plug it into your Pi and let it boot. This will take a … very … very … long … time. Installing an entire hierarchy of desktop packages in this manner is not a quick process, and the first boot will probably take at least 1 hour (I didn’t time every installation during this test, but the cloud-init logs for the Lubuntu run indicated setup on the microSD card took roughly 1½ hours).

However, once it’s complete, it should automatically reboot and you should find yourself at a shiny new graphical desktop login! There’s still some horror to sort out here (in particular the clash of networking configuration stacks), but we’ll deal with that next time when we’ll also take a look at comparing the current crop of desktops on Ubuntu, and see what’s changed in the last few years. We’ll also compare them to the official Gnome desktop, and learn a few more cloud-init tricks …

[1]	If you’re using Linux, you’ll also see the root partition (“writable”) appear, but you can ignore this

[2]	`oem-config`, derived from the `ubiquity` installer. There are moves afoot to replace this at some point (given `ubiquity` is being replaced with `subiquity`), but nothing concrete as yet

[3]

For those violently allergic to systemd, the equivalent commands the listing locales, timezones, and keyboard settings are:

locale:: locale -a
timezone:: (cd /usr/share/zoneinfo/posix; find -type f -or -type l -printf '%P\n')
keyboard:: Read various sections in /usr/share/X11/xkb/rules/base.lst

Making Jammy less Dodgy

2022-07-22T00:00:00+01:00

It’s now several months after jammy’s release, and thus long past time since I should’ve posted something about it. I’ve been holding back on this because, despite being an LTS (in fact, the first Ubuntu LTS desktop for the Pi), I can’t say I’m happy with it.

Yet.

Still, that’s why we don’t enable upgrades for LTS users until the first point release (planned for August 4th at the time of writing). So let’s go through the painful bits and see what early adopters can rectify before I get my nose back to the grindstone and try and fix all the bits I’ve messed up!

In some ways, this may be considered a spiritual successor to my earlier post (and even earlier post) on the Pi desktops. Or to put it another way, consider this the current “Dave recommends you run your Ubuntu Pi this way” post (until I supersede it again)!

Note

I’m going to take the unusual step of stating quite explicitly that the following (like everything on this site) is my personal opinion. This is my recommendation for running your Ubuntu Pi, not anyone else’s (including Canonical).

So, with disclaimers thoroughly disclaimed, let us begin …

Snap!

It’s the serial complaint: the default browser (Firefox) is now a snap. I’m not going to get into the fiery debate over snaps here (more beer is required for that!). Nor am I going to address the start-up speed issues (they’ve been furiously hacked upon and thoroughly covered elsewhere; in fact I was a small cog in the team that did some of that work). Besides, I’m more concerned with the performance of the browser after it’s started (a browser isn’t something I fire up more than once a day), and in that regard I’ve no particular complaints about the performance of the snapped version over the deb version.

And yet, complaints I have …

There are currently issues with the Firefox snap which, personally, I consider deal-breakers. In particular LP: #1741074 “chrome-gnome-shell extension fails to detect native host connector” (aka the “native messaging issue”). The bug’s title may sound relatively innocuous but in practice this translates to “I can’t use KeepassXC” (my password database of choice) with my browser.

I would fallback to KeepassXC’s awesome auto-type feature, but that doesn’t work under Wayland (to be clear: this isn’t exactly a defect, more a design choice; the linked thread has more details). Hence, at least with the version of jammy as released, my only option to use my (now pretty extensive, and certainly indispensable) password database is to copy and paste usernames and passwords via the clipboard … which just about any process can view. Obviously, this is a fairly major step backwards in security and not one I’m willing to put up with.

I should stress this is another issue that is top of the list and being furiously attacked with many sharp objects. Once it’s fixed, I’ll have no specific qualms about returning to the snap (after all, Mozilla seem to prefer it as a means of distribution). But what to do in the meantime?

Switch to the flatpak? No good: the aforementioned bug applies to any confined (snap or otherwise) browser. Besides … the Firefox flatpak is only available for amd64!

Switch to Chromium? It’s snapped too. That said, I’m not sure I would even if I could: uBlock Origin (an extension I consider damned near mandatory for anyone wanting to browse the modern web on anything less than a beefy Ryzen with 32GB of RAM) is better in Firefox than in Chromium.

What about full-fat Chrome? Not an option; Google only distribute Chrome packages for amd64 too.

Note

Rant: frankly, this is one of my biggest misgivings about snap, flatpak, and all these distribution methods that purport to put “distribution in the hands of the developers”: they all suck at providing for anything other than PC architectures leaving stuff like the Pi (or RISC-V, or any new architecture) high and dry.

So, until this is fixed, let’s just use the deb:

$ sudo add-apt-repository ppa:mozillateam/ppa
... lots of output, press Enter ...
$ cat << EOF | sudo tee /etc/apt/preferences.d/firefox
Package: firefox*
Pin: release=LP-PPA-mozillateam
Pin-Priority: 501

Package: firefox*
Pin: release o=Ubuntu
Pin-Priority: -1
EOF
$ sudo apt purge firefox
$ sudo snap remove firefox
$ sudo apt install firefox

Crackle!

Another cereal complaint (I make no excuses for the puns here): the introduction of systemd-oomd! In fact, I was rather glad of this one but I also think it’s less of an issue on the Pi than it is the PC, for reasons I’ll go into …

Note

This section is partly redundant because, while writing it, one of my colleagues (Nick, who despite being fresh-faced and new, jumped headlong into the fray) pushed a new configuration that substantially solves things. See LP: #1972159 for the full story.

However, there’s still some useful stuff in here for users on smaller memory systems (including non-Pis!) so I’ve left it in.

Firstly, a little context. For those not aware, systemd-oomd is a new facility for killing memory hogging processes (now activated by default on our desktop images). Previously, this was solely the domain of the dreaded OOM killer in the Linux kernel. While this certainly did the job of killing memory hogs, it was typically only capable of doing so after the system had descended into “swap hell”. In other words, only once something ridiculously fat had pushed everything else on the system to page itself out to disk, did the kernel OOM killer wake up and stalk the process table looking for potential victims.

An artist’s impression of “swap hell”

This typically meant that, while things did get remedied eventually, it took several minutes of the machine “thrashing” for that to happen. In the noughties, I might listen for the audible cue of an HDD thrashing its heads to have a clue this was going on, but this is now and SSDs (or SD cards) don’t provide quite the same feedback!

As a rough rule of thumb, people will assume something has crashed after roughly 5 seconds of the UI appearing to freeze. If that sounds short, consider how many seconds it would take before you assumed a crash if even your mouse pointer is frozen (or jumpy to the point that someone might reasonably assume something was “wrong”). In practice this typically meant that, long before the kernel OOM killer got involved, the user’s itchy power-button finger got involved first.

To be fair, systemd-oomd does fix several issues with the kernel’s OOM killer. The most notable being that a modern application may have many “processes” and (being process-oriented) the kernel’s OOM killer would only kill the heaviest individual process (regardless of whether it was a mere component in a larger system). That potentially left the rest of the application in a unknown / unstable state. By contrast, systemd-oomd will kill the whole cgroup consistently (cgroup in this context can be read as “application” … sort of).

It also acts before the system ever gets near “swap hell”, basing its actions on the system reaching some (fairly arbitrary) thresholds like 90% of both RAM and swap being utilized (to activate), and pressure stall information (to determine what to act upon).

Unfortunately there are some major issues:

Most browsers (presumably in acknowledgement of how much fat is dripping off the silk of the web these days) already have mechanisms to “unload” their heavier processes (usually individual tabs) when they detect the system is “under load”. However, being fine-tuned to the individual application, these usually kick in later than systemd-oomd will (or to put it another way, by the time they would kick in, systemd-oomd has already killed the whole browser).
systemd-oomd has no user interaction or even notification. One second you’re working in your browser and the next: blip. It’s gone. No warning, no chance to intervene, and not so much as an apology after the fact!

On the upside, it all happens very quickly: there’s no wading through the treacle of swap-thrashing. But it feels … arbitrary, unpredictable, and frankly “flaky”.

There is a simple solution: get more RAM! On my machines with 8GB (or more) of RAM, I simply haven’t encountered the issue. It’s only on machines with 4GB or less that this seems to be a serious issue (at least, that’s my personal experience — I’m sure others with different workloads may find differently). Still, that includes the Pi 400 I use as a desktop, and also my tiddly laptop (which has 4GB of RAM and a Celeron processor — I like my machines small and cheap!).

And this is where things (surprisingly!) turned out better on the Pi than on the PC:

On my little 4GB laptop (which doesn’t really get much use, only when I’m away from the house), I noticed Firefox frequently getting killed especially when working in Google Docs. Curiously, I didn’t notice the same on the Pi 400 (one of my two main workhorses that gets used a lot more than the laptop).

Then I recalled that we’d activated zswap (see earlier post) by default on jammy for the Pi desktop images. My PC laptop had the same RAM size (4GB), the same swap size (1GB), but while zswap was active on the Pi, it wasn’t on the laptop. That’s not to say the Pi was perfect: there were two occasions when Firefox got killed, but that was in stark contrast to the laptop on which it happened half a dozen times in a single day.

Obviously the first thing I did was activate zswap on my laptop and, after a reboot suddenly things were much better. I did still manage to get Firefox killed during a subsequent test run, but after then expanding the swap to 2GB on both machines, I managed a full day of work on the laptop without a single systemd-oomd activation.

So (unusually for this site), here’s a recommendation for any jammy desktop users on small PCs out there (Pi users, you can ignore this bit):

$ sudo -i
# sed -i \
> -e '/GRUB_CMDLINE_LINUX_DEFAULT/ s/splash/splash zswap.enabled=1 zswap.compressor=zstd zswap.zpool=z3fold/' \
> /etc/default/grub
# echo zstd >> /etc/initramfs-tools/modules
# echo z3fold >> /etc/initramfs-tools/modules
# update-initramfs -u
# update-grub

The above will activate zswap by adding zswap.enabled=1 zswap.compressor=zstd zswap.zpool=z3fold to the kernel’s command line in the GRUB configuration, and adding the necessary modules to the initramfs.

Next, I’d also recommend (to both small PC users and Pi users), to bump the default swap-file up to 2GB. After doing this, I haven’t suffered a single systemd-oomd kill on my Pi 400 (again, for my particular workload — bear in mind I spend most of my time in lightweight terminal things, so adjust accordingly). The following should be run on a freshly booted system as the first thing we’re going to do is disable the existing swap before expanding it:

$ sudo swapoff /swapfile
$ sudo fallocate -l 2G /swapfile
$ sudo mkswap /swapfile
$ sudo chmod 600 /swapfile
$ sudo swapon /swapfile

What have we actually accomplished with all this?

Let’s take the example of my 4GB laptop. The defaults for zswap’s configuration will reserve 20% of all RAM for compressed pages, which is roughly 800MB. Those compressed pages will (with the z3fold allocator) store up to 3 uncompressed pages each. So that 800MB of RAM can act as up to 2.4GB of (suspiciously fast) swap. And beyond that we also have 2GB of (typical, slow) disk-based swap.

Visually what we’ve done is this:

Three bars illustrating the original state of the memory layout (4GB RAM, 1GB swap), the new physical layout (3.2GB RAM, 800MB zswap, 2GB disk swap), and the new “effective” layout (3.2GB RAM, 2.4GB zswap, 2GB disk swap)

Don’t be fooled: we’ve reduced the amount of available RAM from 4GB to 3.2GB. This isn’t good, but it’s the trade-off you make with compressed RAM systems of any sort. We’ve also sacrificed another gig of disk space to swap. However, we’ve increased the available swap from 1GB (by default) to about 4.4GB, of which more than half is very fast compressed-memory swap and all for a (fairly) minimal memory cost.

Furthermore, we’ve also moved from a configuration where there’s 4GB of very fast pages, and 1GB of really slow pages to one where there’s 3.2GB of very fast pages, 2.4GB of fairly quick pages, and 2GB of really slow pages. In other words, we’ve now got a “smoother” progression of memory the kernel can stick pages into. This helps a system under load go a bit slower while remaining reasonably responsive, instead of suddenly hitting a brick wall of swap.

Soggy Cornflakes

One mea-culpa I should offer here: I managed to forget something rather important on the Ubuntu Pi desktop images for jammy.

While zswap is activated, and the swap-file is created (now on boot by the mkswap.service unit so we no longer ship a gigabyte of “nothing” in the image itself) … I managed to forget to add the “z3fold” and “zstd” modules to the initramfs modules list. As a result, zswap is still working on the Pi desktop images, but it’s currently falling back to the default “zbud” (2-page) allocator, and “lzo” compression.

LP: #1977764 is tracking this and I’ll get it fixed as soon as I reasonably can (unfortunately, while it sounds trivial it’s actually a messy one — read the bug if you want the full story).

Pop!

That about covers things for the desktop side. What about the server image? Overall I’m actually pretty happy with the server side of things, but one thing has been irking me whenever I’ve played with the lovely little Pi Zero 2W. There’s precious little memory available at runtime, particularly on the arm64 images (which it seems the vast majority of our users prefer).

There’s been some effort recently to identify things installed by default which shouldn’t be, with the goal of reducing the “base” memory footprint of our images and I thought I’d have a look at what’s running by default on the Pi server images to see if there’s anything we could cull there.

First, let’s see how much free memory we have on a fresh boot:

$ free -h
               total        used        free      shared  buff/cache   available
Mem:           414Mi       152Mi        33Mi       3.0Mi       228Mi       242Mi
Swap:             0B          0B          0B

Hmmm, only 242MB available. Okay, what’s eating it?

Note

A quick clarification about “free” vs “available”. Ignore “free”. This is the amount of RAM actually unused, but unused RAM is wasted RAM. The kernel has a duty to minimize free RAM by filling anything unused with disk (and other useful) caches which can be evicted trivially in the case that more RAM is required. The “available” metric indicates the amount of RAM that is available “on demand” should userland processes request it. It’s roughly (but not precisely) equal to “free” + “buff/cache”.

Or to put it another way: “available” tells you how much RAM is available for use; “free” is the metric which tells you how much RAM you’ve wasted your money on ;)

We’ll use the ps command exclude everything under PID 2 which is the kernel (--ppid 2 -p 2 -N). We’ll turn on the hierarchical view (-H) to get a clue of what belongs to what (this won’t show every inter-process dependency, but it can be a rough guide), and the “full” output (-F) so we can get the memory usage info (amongst other things; I’ll exclude some of the columns below for the sake of text wrapping):

$ ps --ppid 2 -p 2 -NHF
UID          PID    PPID     SZ   RSS PSR TTY   CMD
root           1       0  41786 11364   2 ?     /sbin/init fixrtc splash
root         374       1  12020 12572   0 ?       /lib/systemd/systemd-journald
root         416       1  72416 25672   3 ?       /sbin/multipathd -d -s
root         429       1   5998  6332   2 ?       /lib/systemd/systemd-udevd
root         576       1   4087  9732   3 ?       /sbin/wpa_supplicant -c /run/netpla
systemd+     588       1  22160  6292   3 ?       /lib/systemd/systemd-timesyncd
systemd+     634       1   4097  7704   1 ?       /lib/systemd/systemd-networkd
systemd+     636       1   6270 12224   1 ?       /lib/systemd/systemd-resolved
message+     668       1   2280  4220   3 ?       @dbus-daemon --system --address=sys
root         673       1  20492  3344   2 ?       /usr/sbin/irqbalance --foreground
root         677       1   8217 17916   1 ?       /usr/bin/python3 /usr/bin/networkd-
root         678       1  59035  8220   3 ?       /usr/libexec/polkitd --no-debug
syslog       680       1  55508  4392   3 ?       /usr/sbin/rsyslogd -n -iNONE
root         684       1  366205 27060  0 ?       /usr/lib/snapd/snapd
root         687       1   6002  7164   2 ?       /lib/systemd/systemd-logind
root         691       1  98379 11280   3 ?       /usr/libexec/udisks2/udisksd
root         695       1   3813  3440   0 ?       /sbin/wpa_supplicant -u -s -O /run/
root         712       1   1727  2456   0 ?       /usr/sbin/cron -f -P
root         726       1  61827 12368   1 ?       /usr/sbin/ModemManager
root         736       1   1409   808   1 ttyS0   /sbin/agetty -o -p -- \u --keep-bau
root         744       1  27480 19884   0 ?       /usr/bin/python3 /usr/share/unatten
root         755       1   3787  8428   1 ?       sshd: /usr/sbin/sshd -D [listener]
root        1660     755   4530  9716   0 ?         sshd: ubuntu [priv]
ubuntu      1804    1660   4661  7592   1 ?           sshd: ubuntu@pts/0
ubuntu      1805    1804   2152  4852   2 pts/0         -bash
ubuntu    355020    1805   2495  2844   3 pts/0           ps --ppid 2 -p 2 -NHF
root         881       1    560   168   2 ?       /usr/bin/hciattach /dev/serial1 bcm
root         899       1   2386  4468   3 ?       /usr/lib/bluetooth/bluetoothd
ubuntu      1004       1   4559  9924   2 ?       /lib/systemd/systemd --user
ubuntu      1005    1004  42795  5800   2 ?         (sd-pam)
root        1766       1   1398   812   2 tty6    /sbin/agetty -o -p -- \u --noclear
root        1777       1   2417  3976   1 tty1    /bin/login -p --
ubuntu    227377    1777   2174  4856   2 tty1      -bash

Okay, lots to go through here. RSS (Resident Set Size) is the most interesting column here (it’s not wholly accurate but good enough for our purposes). What’s the fattest thing in RAM? Ah … snapd (at ~26MB). On my fat 8GB Pi 4 I’m happy to leave that because I use the LXD snap (a lot!). However, this is a little Zero 2W and I only need debs on this, so out it goes:

$ sudo apt purge snapd
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following package was automatically installed and is no longer required:
  squashfs-tools
Use 'sudo apt autoremove' to remove it.
The following packages will be REMOVED:
  snapd*
0 upgraded, 0 newly installed, 1 to remove and 0 not upgraded.
After this operation, 84.1 MB disk space will be freed.
Do you want to continue? [Y/n]
... lots of complaints about read-only stuff ...
rm: cannot remove '/snap/core20/1408/var/log': Read-only file system
rm: cannot remove '/snap/core20/1408/var/mail': Read-only file system
rm: cannot remove '/snap/core20/1408/var/opt': Read-only file system
rm: cannot remove '/snap/core20/1408/var/snap': Read-only file system
rm: cannot remove '/snap/core20/1408/var/spool/mail': Read-only file system
rm: cannot remove '/snap/core20/1408/var/tmp': Read-only file system
rm: cannot remove '/snap/core20/1408/writable': Read-only file system
Cannot remove directory /snap
Removing extra snap-confine apparmor rules
Removing snapd cache
Removing snapd state
dpkg: warning: while removing snapd, directory '/snap' not empty so not removed
$ sudo apt autoremove --purge
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following packages will be REMOVED:
  squashfs-tools*
0 upgraded, 0 newly installed, 1 to remove and 0 not upgraded.
After this operation, 408 kB disk space will be freed.
Do you want to continue? [Y/n]
(Reading database ... 100500 files and directories currently installed.)
Removing squashfs-tools (1:4.5-3build1) ...
Processing triggers for man-db (2.10.2-1) ...

At this point, given it couldn’t remove everything under /snap I’d just reboot to clear up any remaining mounts and sudo rm -fr /snap (I don’t recall having to do this before; I’ll try and remember to investigate this further at some point).

$ sudo reboot
... wait to get back to the login ...
$ sudo rm -fr /snap

Now, what’s next? “multipathd” (~25MB). This is a piece of the default server installation for dealing with multipath disk IO. However, unless you’re running a Compute Module with some fairly serious storage attached to it, this is likely pointless on your Pi. Unfortunately, removing this is probably not a good idea:

$ sudo apt purge multipath-tools
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following packages were automatically installed and are no longer required:
  kpartx libsgutils2-2 liburcu8 sg3-utils sg3-utils-udev
Use 'sudo apt autoremove' to remove them.
The following packages will be REMOVED:
  multipath-tools* ubuntu-server* ubuntu-server-raspi*
0 upgraded, 0 newly installed, 3 to remove and 0 not upgraded.
After this operation, 1179 kB disk space will be freed.
Do you want to continue? [Y/n] n
Abort.

Notice how it wanted to remove the “ubuntu-server” package? That’s usually not a good sign! However, perhaps we can disable it somehow:

$ systemctl cat multipathd.service
# /lib/systemd/system/multipathd.service
[Unit]
Description=Device-Mapper Multipath Device Controller
Before=iscsi.service iscsid.service lvm2-activation-early.service
Before=local-fs-pre.target blk-availability.service shutdown.target
Wants=systemd-udevd-kernel.socket
After=systemd-udevd-kernel.socket
After=multipathd.socket systemd-remount-fs.service
DefaultDependencies=no
Conflicts=shutdown.target
ConditionKernelCommandLine=!nompath
ConditionKernelCommandLine=!multipath=off
ConditionVirtualization=!container

[Service]
... etc ...

[Install]
... etc ...

Ah ha! Adding nompath or the slightly more obvious multipath=off to the kernel command line should disable this service. Well, that’s easily accomplished:

$ sudo sed -i -e 's/$/ multipath=off/' /boot/firmware/cmdline.txt
$ sudo reboot

I’m sorely tempted to add this by default to the Pi images, but I need to dig a bit further into whether multipath has any genuine use-cases on the Pi.

Next on the list? The Python-based “unattended-upgrades-shutdown” process (~19MB). It’s not a great idea to disable this (it ensures the computer doesn’t shut down in the middle of unattended-upgrades). However, it’s annoying that it hangs around the whole time eating RAM and only actually does anything at shutdown. Still, it’s the subject of an existing ticket, LP: #1955084 so we’ll just leave that one for now.

Next? Another Python process: the “networkd-dispatcher” daemon (~17MB). If you like your WiFi connection, leave this one alone! Next up, it’s “systemd-journald” (~12MB). That’s also doing useful work, so we’ll leave that one alone too.

Then … “ModemManager”. Hang on? Modem Manager? Am I back in the 90s?! Oh, it’s for GSM modems. Still, I’m not actually using a GSM modem on this Pi so why is that there? In fact, it’s the subject of another ticket, LP: #1981109 (you may note it’s another one filed by Steve, another colleague who has been ruthlessly hunting down the cruft of late). Anyway, given the details in that ticket this one can be safely excised (it’s not even a real dependency, just a “recommendation” of another package)!

$ sudo apt purge modemmanager --autoremove
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following packages will be REMOVED:
  libtcl8.6* modemmanager* tcl* tcl8.6* usb-modeswitch* usb-modeswitch-data*
0 upgraded, 0 newly installed, 6 to remove and 0 not upgraded.
After this operation, 8595 kB disk space will be freed.
Do you want to continue? [Y/n]
(Reading database ... 100491 files and directories currently installed.)
Removing usb-modeswitch (2.6.1-3ubuntu2) ...
Removing tcl (8.6.11+1build2) ...
Removing tcl8.6 (8.6.12+dfsg-1build1) ...
Removing libtcl8.6:arm64 (8.6.12+dfsg-1build1) ...
Removing modemmanager (1.18.6-1) ...
Unknown option: runtime
Removing usb-modeswitch-data (20191128-4) ...
Processing triggers for man-db (2.10.2-1) ...
Processing triggers for dbus (1.12.20-2ubuntu4) ...
Processing triggers for libc-bin (2.35-0ubuntu3) ...
(Reading database ... 100105 files and directories currently installed.)
Purging configuration files for tcl8.6 (8.6.12+dfsg-1build1) ...
Purging configuration files for usb-modeswitch (2.6.1-3ubuntu2) ...
Purging configuration files for modemmanager (1.18.6-1) ...
Processing triggers for dbus (1.12.20-2ubuntu4) ...

Next up, “systemd-resolved” (~12MB). We shouldn’t touch that one (DNS resolution and caching is generally useful).

Next? “udisks2” (~11MB). This is a useful service, handling things like auto-mounting USB storage devices that have been inserted. However, I’m not going to be doing that on this Pi Zero 2W so I don’t need it running all the time. I’m not going to uninstall it as it’s potentially useful and, even after disabling it (which will prevent it auto-starting), the udisksctl command can implicitly start it back up again, should I find I need it.

$ sudo systemctl stop udisks2
$ sudo systemctl disable udisks2
Removed /etc/systemd/system/graphical.target.wants/udisks2.service

At this point we’re into diminishing returns so let’s reboot to a clean start and see what free says now.

$ sudo reboot
... wait to get back to the login ...
$ free -h
               total        used        free      shared  buff/cache   available
Mem:           414Mi       103Mi       165Mi       2.0Mi       146Mi       299Mi
Swap:             0B          0B          0B

That’s looking better! Nearly 300MB free now. Finally, another tweak we can use to grab a bit more memory. This one is only useful if you have no intention of using GPU functionality (no camera, no video decoding / encoding acceleration, etc). However, I don’t need any graphics on this particular Pi (in fact there won’t be any display plugged into it) so let’s reduce the GPU memory split all the way down to the minimum (16MB).

$ sudo -i
$ echo "gpu_mem=16" >> /boot/firmware/config.txt
$ sudo reboot
... wait to get back to the login ...
$ free -h
               total        used        free      shared  buff/cache   available
Mem:           462Mi       101Mi       214Mi       2.0Mi       146Mi       348Mi
Swap:             0B          0B          0B

And there we have it; over 100MB extra available memory from our starting point. Bear in mind we have disabled some functionality to achieve this, so these aren’t all changes that could (or should!) be made universally. However, from the tickets we’ve encountered along the way there’s definitely some improvements that can still be made in the base image.

When HDMI freezes over

2021-10-13T00:00:00+01:00

Well, it’s time for another Ubuntu release, and so time for another slew of blog posts from me! In this particular case though, I need to start with a mea culpa on the state of the Ubuntu Desktop for Raspberry Pi release.

The bug, the bodge, and the deadline

Shortly before release, a bug in HDMI output handling (initially part of another bug) was discovered which vexed us right up to the release itself. The symptoms ranged from the annoying (display output freezes after a certain amount of use, usually a couple of hours in my case) to the critical (display output freezes during boot, in the case of a colleague).

The determining factor eventually turned out to be something to do with high refresh-rate monitors. With my eye-sight, I’ve never wanted or needed anything beyond good ol’ 1080p at 60Hz … and so that’s what all my monitors are. But younger colleagues (with better eye-sight) have fancier displays with higher frame-rates and on these the issue was most definitely critical.

We quickly found a work-around (which is in the release notes) which was simply to replace the “kms” overlay in config.txt with the “fkms” overlay, but then the race was on to fix things before release.

The kernel team worked tirelessly to try and come up with a fix, but it quickly became apparent this was an upstream issue too, and that it was unlikely we’d find a fix on our own. Eventually it was obvious that the bug was going to make it into the release images, and so the debate switched to whether to release with the fkms workaround in place.

To bodge, or not to bodge

As will be obvious to anyone who’s tried the desktop release with a high-refresh rate monitor, we released without the workaround in place (and that’s the royal we, because ultimately this was my decision so if you’re going to yell at anyone, yell at me — comments below!).

By way of (lengthy) explanation, here was my reasoning:

We have regular meetings with the Pi Foundation (or more precisely the engineers at Raspberry Pi Trading Ltd. — just to appease Ben’s nagging of my constant conflation of these entities!). My understanding from these meetings is the “kms” overlay is considered the “future direction of development”. As such, it is under active development in the upstream kernel.
The “fkms” overlay is considered legacy, and while it is supported in the kernel in use in Raspberry Pi OS, if it happens to break in the development kernel (which is where the Ubuntu kernel patches are sourced from), that’s acceptable.
We have a mechanism for modifying the boot configuration during a release upgrade, but no such mechanism currently exists for kernel upgrades. The modifications performed during release upgrades are not perfect (there are edge cases we cannot reasonably account for), but a release upgrade is a sufficiently major operation that a certain amount of breakage might be considered reasonable in the case of esoteric configurations. The same cannot be said of kernel upgrades.
Other (minor) points against “fkms”: audio doesn’t operate correctly (choppy) under “fkms” without the “tsched=0” workaround, and video displays screen-tearing. These are obviously non-critical, but it’s a sub-par experience compared to fixing “kms” properly.

To summarise: we could switch people to the “fkms” overlay during the impish release upgrade (and obviously modify the release image to use “fkms” too) but that would leave us in a position where a future kernel upgrade during the release broke things (in fact, our testing with a few interim upstream kernels suggested this was already the case), and there was no good mechanism for fiddling with the boot configuration during a regular apt update, apt upgrade procedure.

Alternatively, we could release with the “kms” overlay knowing full well that would break badly on certain high-fps monitors, but with a release noted work-around that wasn’t terribly complex (editing one line in a text file, which was accessible on all platforms), and the hope that we could fix the kernel in a (quickly released) update. Obviously this would still entail pain for people having to use the workaround to even boot (in order to install such an upgrade). But it would also mean that such people would have experience of installing the workaround and thus would have little difficulty in removing it themselves, without having to bodge a postinst script into the kernel to try removing it (probably imperfectly).

Conclusion … ?

As it turns out, we may have a fix quicker than I’d hoped: as I write this final section, I’m just waiting on another SD card to flash to play with a testing kernel which may fix at least part of the issue (I wasn’t kidding when I said the kernel team were working tirelessly on this!). There are also a couple of patches that appear to stabilize things completely, at least on my old 1080p 60Hz monitor.

Anyway, to all those affected, all I can say for now is “sorry” that this release isn’t everything I’d hoped it to be, and that the release includes this issue. That’s on me: I could’ve made the call to switch to “fkms” for the release, but I genuinely think that although this is obviously the more painful course, it’s the right one for a better experience in the end.

6 Months with the Pi Desktop

2021-04-17T00:00:00+01:00

See Also: A look at the Jammy desktop image.

In a deep movie trailer voice: “Previously on Dave’s blog“… “Oh no! My PC is no more! Whatever shall I do?!” “Oooh, a Pi 400 …”

Well, it’s been about 6 months since I started dog-fooding the Ubuntu desktop on a Raspberry Pi (and since I last managed to write a blog post … ahem). I’d originally intended to go back to using a big, fat, Watt-guzzling PC as my main development machine at some point in this cycle. However, a combination of global silicon shortages (if the damned crypto-miners could let me buy a single decent graphics card that’d be … nice?), and some major improvements in the Pi desktop meant I wound up spending the entire hirsute cycle using the little Pi 400 as my primary desktop.

The Hippopotamus in the Room

The biggest issue with this, as discussed in the previous installment, was video playback. We didn’t have video acceleration on the Ubuntu Desktop for Pi and this meant that, at the start of the hirsute development cycle, I still had to resort to my little laptop (a tiny Celeron powered thing) for video conferencing. However, at some point (a couple of months ago? I forget the precise timing as all sorts of work-arounds have been required at one stage or another) things came together, the planets aligned, the Gods of the Desktop smiled upon their lowly worshippers and Wayland+KMS started working.

Suddenly … things got much better.

For the past several weeks I’ve been using my little Pi 400 as my sole desktop. The laptop has been ditched entirely. HD YouTube playback works happily. Video conferencing over Google Meet works perfectly (no Zoom, but that’s because there’s no ARM builds of the client, ho hum), and the desktop generally feels responsive and slick.

There’s definitely a few kinks still to work out, but nothing show-stopping that’s made me dig the laptop out of storage or prevented me from Getting Stuff Done.

Making Hippos Fly

Above is a shot of my little desktop setup. Some optimizations to note:

I don’t have a fancy monitor. It’s bog-standard old 1080p HDMI thing. Personally, I can’t tell the difference between 1080p and 4K without squinting through my glasses anyway, but from a performance perspective having a 1080p monitor means no scaling is required to make the desktop readable. 200% scaling is fast under Gnome, but the result is frankly too big for my liking, and fractional scaling causes a noticeable performance hit. Choose your screen wisely!
Use the HDMI0 port. On the Pi 4, it’s next to the USB-C power port. On the Pi 400, it’s the one next to the SD card slot. In certain configurations, audio playback through HDMI only works from HDMI0.
It’s booting off an SSD connected to one of the USB3 ports (that little black box sat on the base of the monitor). We introduced support for booting Ubuntu over USB in groovy, and hirsute should work just as well. Simply flash the image to your USB drive and you’re good to go. But do note that you may need to enable USB boot in your Pi’s boot order first (this is one-time setup so if your Pi can already boot off USB, you’re good to go).
You can’t attach a Pi Camera Module to the Pi 400, so I’ve got an old Logitech C920 attached to one of the USB ports.
The Pi 400 only has 4GB of RAM. For lightweight desktops like LXDE this is okay, but Gnome shell is quite a bit heavier. The Ubuntu Desktop image for Pi does come with a 1GB swap file but honestly you probably want to extend this and I highly recommend using zswap (more on that below).
One advantage the Pi 400 does have over the Pi 4 is it’s much better at running cool. In turn, this means overclocking it is entirely viable even without any active cooling. At present I have the following lines added to my config.txt file on the boot partition, and the Pi 400 has been running with no heat / throttling issues for months:
```
arm_freq=2000
over_voltage=6
```
You might get away with these settings on a Pi 4, but you’d likely want some active cooling for that setup.

Swap, swap, glorious zswap!

As a rough rule of thumb, all systems should have swap. It doesn’t matter if your machine is a beast with 128GB of RAM, having some swap (not much, just a gig or two, say) is still useful as it permits the kernel to ditch pages that are never (or extremely rarely) used and use the RAM saved for disk cache (and there’s no such thing as too much of that!).

However, on a Pi 400 running a hefty desktop environment under 4GB of RAM it’s downright vital. As mentioned above, the images ship with a 1GB swap file (some may argue it would be preferable to create it on first boot, but it compresses trivially in the image, and this ensures that when people flash it they effectively “reserve” room for the necessary swap space). Ideally though (unless you’re on a beast like the Pi 4 8GB model), you probably want to expand this to something a bit bigger.

This can be easily done from the command line but I’d caution to do this when you’ve first booted the system and before you’ve launched anything else as the first step is to (temporarily) disable the swap. The following commands will add 1GB of size to the default swap-file on the desktop image:

$ sudo swapoff /swapfile
$ sudo fallocate -l 2g /swapfile
$ sudo mkswap /swapfile
$ sudo swapon /swapfile

Note

You could trivially add a second swap-file and this wouldn’t necessitate disabling the existing one, but that then means getting into editing the /etc/fstab file to add it and this is a bit easier to document. Use whichever means you are comfortable with, but the important thing here is that 1GB of swap probably isn’t quite enough on a system with 4GB of RAM.

Next, I would highly recommend enabling “zswap“. This is one of several forms of virtual memory compression provided by the Linux kernel. The other commonly used one is “zram“. I’m not enough of an expert to opine on the fine detail of these systems, but my rough rule of thumb is as follows:

If you have no swap-file at all, zram is preferable. It provides a pseudo swap-file which is actually a compressed area of RAM and hence requires no swap-file backing it.
However, if you have a swap-file, zswap is preferable. It acts as a write-back cache for the swap-file and can in certain circumstances avoid pages ever hitting the disk at all.

Don’t worry about the computational overhead of these systems. CPU speed and memory bandwidth have grown considerably faster than bulk storage IO bandwidth has. As a result, memory compression systems that (to some degree) prevent or reduce disk IO are almost always a win.

Anyway, because we definitely have a swap-file, we’re going to use zswap here. It’s trivial to enable, but we’re going to customize it a bit to get some more out of it. To enable zswap you simply need to add the parameter zswap.enabled=1 to your kernel command line:

$ sudo sed -i -e 's/$/ zswap.enabled=1/' /boot/firmware/cmdline.txt

This is sufficient to enable zswap; if you were to reboot at this point it would already be working. If you don’t want to get into things that are a bit more complex, then feel free to stop at this point — you’ll already have plenty of benefit.

Still here? Okay, zswap’s defaults are, if anything, a bit conservative. The default allocator (zbud) is limited to 2 compressed objects per page. A superior allocator (z3fold) permits 3 objects per page boosting the typical compression ratio. Furthermore, the default compression algorithm is “lzo” which is nice and fast, but doesn’t provide great compression. Other options include “lz4”, “lz4hc” (LZ4 with “high compression”), and “zstd”. Personally, I’ve gone for “lz4”; “zstd” is also tempting but remember we’re balancing compression with speed here, and despite zstd being better at compressing things, lz4 is very fast.

Unfortunately, we can’t “just” enable these as we enabled zswap above. We need to make sure that the modules for these features are included in the initramfs at boot time:

$ sudo -i
# echo lz4 >> /etc/initramfs-tools/modules
# echo z3fold >> /etc/initramfs-tools/modules
# update-initramfs -u

Then simply add some more parameters to your kernel command line:

# sed -i -e 's/$/ zswap.compressor=lz4/' /boot/firmware/cmdline.txt
# sed -i -e 's/$/ zswap.zpool=z3fold/' /boot/firmware/cmdline.txt

Now reboot and you should have your super-whizzy compressed RAM setup complete! To check everything is set correctly (after rebooting) you can run the following:

$ grep -R . /sys/module/zswap/parameters
/sys/module/zswap/parameters/same_filled_pages_enabled:Y
/sys/module/zswap/parameters/enabled:Y
/sys/module/zswap/parameters/max_pool_percent:20
/sys/module/zswap/parameters/compressor:lz4
/sys/module/zswap/parameters/zpool:z3fold
/sys/module/zswap/parameters/accept_threshold_percent:90

If things aren’t right, have a look at /boot/firmware/cmdline.txt in a text editor.

To see how zswap is performing:

$ sudo grep -R . /sys/kernel/debug/zswap
/sys/kernel/debug/zswap/same_filled_pages:3418
/sys/kernel/debug/zswap/stored_pages:242039
/sys/kernel/debug/zswap/pool_total_size:353652736
/sys/kernel/debug/zswap/duplicate_entry:0
/sys/kernel/debug/zswap/written_back_pages:0
/sys/kernel/debug/zswap/reject_compress_poor:368
/sys/kernel/debug/zswap/reject_kmemcache_fail:0
/sys/kernel/debug/zswap/reject_alloc_fail:0
/sys/kernel/debug/zswap/reject_reclaim_fail:0
/sys/kernel/debug/zswap/pool_limit_hit:0

I’ve even tried this on a Pi 4 2GB model and, while it’s not as nice as a Pi 400 (in particular due to the SoC running hotter), it makes it a pretty viable desktop at least for light usage. It was perfectly capable of firing up Firefox and playing an HD YouTube video, while also running LibreOffice Writer, and a terminal with vim, all on a $35 2GB Pi!

Note

I should note we don’t officially support using the desktop image on a 2GB Pi, but personally I think this little optimization makes it a reasonable platform for the Ubuntu desktop.

Hirsute Hippo’s due for release on April 22nd, and the release notes are definitely worth a quick peek.

A Week with the Pi Desktop

2020-10-22T00:00:00+01:00

In a previous post I mentioned I’d be using a Pi as my exclusive desktop for a week, mostly to validate that it actually worked (and that our guesstimate that 4Gb RAM models were a reasonable “base”), but partly just for the experience!

Here’s a blow-by-blow account of how it went (there’s a TL;DR at end if you just want to scroll there :)…

Monday

I’d spent a few hours of the prior week setting up the system as I needed for work so, come this morning, everything was pretty much as my desktop normally is. Just on a Pi.

The Pi in question was a model 4B with 4Gb of RAM. It was booting off an ancient 120Gb SSD I had lying around from an old PC (in that small black case on the floor) and the eagle-eyed will note that there’s absolutely no active cooling or so much as a heat-sink installed on it in this picture. This is deliberate: I was reasonably sure it was going to thermally throttle at some point in the day, but I wanted to see how often that happened and how bad it was in practice. In other words, trying to answer the question: is this usable on an otherwise “naked” Pi?

Other than that, it was plugged into all the stuff I usually have plumbed into my beefy (but aging, and now somewhat dead) Core i7.

First tweaks:

Audio from rhythmbox was choppy. I’d encountered this before and the hints from the “Audio” section of my earlier notes on the Pi desktop soon sorted this out. I’ll see what we can do about fixing this in the final image but given where it is I don’t think we can layer another file on top of it to implement this just for the Pi. This will probably be something I just have to stick in the release notes.
The wrong audio output device was always selected on boot (should be “Analog Output - Built-in Audio”, but it was always “Multichannel Output - Built-in Audio”). On a hunch, I tried disabling the “module-switch-on-connect” lines in that same file (/etc/pulse/default.pa) and the problem was solved, although later I discovered this did mean that audio devices plugged into USB weren’t automatically selected. However, in my case, that’s ideal as I didn’t want them selected. Again, this may wind up being a release note.
No swap. Not an issue initially, but I figured I’d be needing it sooner or later so I set up a 1Gb swap file and some monitoring scripts to regularly report things like system load, swap used, SoC temperature, etc. then added them to my steadily growing tmux configuration. Will see what can be done about fixing this for the release.

The first real test of the day: a meeting in Google Meet! I didn’t have any camera plumbed in at this point as I hadn’t succeeded in getting the camera module working properly with it yet, but I plugged in my nice USB microphone (that’s the USB audio device I mentioned above, which I don’t want to use for output, just input), fired up Chromium (installed from the snap-store) and somewhat nervously connected…

My group were evidently feeling somewhat shy (!) and everyone had their camera feed off, so it was just audio for this first challenge. Still, there were 5 people in the call so this was a reasonable first test. My audio apparently worked perfectly, and I had no issues hearing anyone at my end. All pretty smooth so far!

The Pi had been idling at ~60°C but that temperature slowly began to climb during the call. Within about 10 minutes it was at the 78°C mark (the Pi 4 throttles at 80°C) and I was starting to get a bit nervous. It took quite a while to get actually hit 80°C (maybe another 5 minutes) and at that point there was an occasional drop-out from the audio when others were speaking but otherwise the thermal throttling worked remarkably smoothly and everything stayed connected and working! So far, so good.

After the call was over, I dove into some other work and wound up compiling a couple of packages, writing up some notes and answering a few e-mails. The compiling all went fine, no swapping, no throttling, but the e-mail… This proved to be the next big challenge. Within minutes of firing up a GMail tab, swap space was being eaten into. It really is quite the memory hog!

Next tweaks:

I activated zswap (appended zswap.enabled=1 to /boot/firmware/cmdline.txt) to see if that alleviated the 50% of my 1Gb swap space that’d been eaten. It seemed to (hovered around 20-30% after loading everything back up after rebooting to check my config), but these things are difficult to measure. YMMV.

Now, the next big challenge of the day. Another Google Meet call, but this time one that usually had 10+ participants and was very likely to involve video streams. I opted to try this one in Firefox instead (just to see if it would work). Within a minute of connecting it was obvious the Pi was struggling; the SoC temperature hit 80°C (the throttle point) within the first minute, and the video streams were mostly frozen. That said, at all points audio from the call was playing smoothly. With a fair bit of patience (very slow UI!), I disabled the incoming video feeds so the Pi just had audio to deal with.

Then the next challenge: using a shared Google Doc in another tab with the call still going. I was highly dubious the Pi would manage this given it was already into thermal throttling and swapping but I opened the tab anyway. It wasn’t quick to load but to my amazement it worked! I could see others editing away and could make a few edits myself. It wasn’t smooth, it did have a fair bit of cursor lag, but at all points the audio of the call kept playing mostly smoothly (a few crackles here and there, but nothing to affect comprehension), and I could actually work semi-productively. Remember, this was on the smallest of the Pis supported, with no thermal management at all and no shutting down anything I was already running.

Impressions at the end of the day: it’s slightly uncanny. Several times during the day I had to remind myself that I wasn’t on Ammy (my old Core i7). To be clear: I’m not claiming they’re in any way comparable on performance because they’re not. It’s trivial to find something that shows a massive performance difference (like Google Meet calls, or using GMail), but for most of the time, playing some music in the background while I hacked away at something in vim under a terminal window there was effectively no difference.

Then I’d need to flash a fresh image onto an SD card to test something, and suddenly I’d find myself reaching down to stick the card in the flash-reader that used to be in one of Ammy’s 5.25” bays and realize: nope, I’m on the Pi, and I need to use that nice little card-reader-slash-usb-hub I’d stuck on the desk instead.

To my genuine surprise, there wasn’t one thing the Pi failed at all day. Bear in mind this was an uncooled, more or less “stock” 4Gb Pi. Sure, the Google Meet calls weren’t great, and the GMail UI was hilariously laggy in some places … but they worked! Nothing crashed, nothing hung, and I had a day about as productive as any normal day.

If there’s one major difference that I’m just noticing it’s that, as I write this at 8 o’clock in the evening, my fingers are cold.

This may not sound too surprising: my office is in an unheated loft extension, it’s mid-October, and I’m in the North of the UK, so it’s pretty chilly outside. But normally Ammy, with her mammoth heat-sink, myriad fans, and frankly silly graphics card, has done a pretty good job of keeping things toasty and warm up here. She’s usually more than capable of raising the temperature in this room up to 30°C (in summer, things get ridiculously hot up here!). But the Pi has nothing like that thermal output. I might have to get some gloves…

Tuesday

Played around a bit with the configuration today to test some extra interfaces:

Unplugged my usual Microsoft Intellimouse and tried out a bluetooth-based mouse instead. After some considerable pain getting it paired (which may be because it was previously paired to another machine), it’s working happily.
Paired the Pi with my mobile phone and tried out bluetooth-based tethering. Totally uneventful: just worked.
Also fitted a Pimoroni fan-shim to see what difference that made. Thankfully since this image doesn’t have U-Boot there were no problems with the fan-shim interrupting it.

To configure the temperature limit on the fan-shim, I simply added the following line to /boot/firmware/config.txt:

dtoverlay=gpio-fan,gpiopin=18,temp=65000

The only thing that brought the Pi into throttling territory yesterday were the calls in Google Meet, so I was interested to see what difference that made today. For the first call of the day there were only a few people with cameras on, but that was enough to quickly push the Pi’s temperature up.

Sure enough, the fan-shim kept things nicely below 65°C throughout the call. However, while this meant the audio was perfect throughout, I still wound up having to switch off most of the incoming video streams to keep things operating smoothly (Google Meet has a nice option to show a single standard-def stream at a time which seemed to work well). On this call, I was still audio only.

Then it was back to some compiling and documentation work, no problems here and everything went smoothly.

Next call: at this point while I’d managed to get a camera module installed, there was a problem in that the cable isn’t that long and I didn’t think people would particularly enjoy a view from under my desk (ahem). So, instead I plugged in the USB web-cam that normally sits on my monitor. I knew this was likely to give the Pi a hard time as USB web-cams typically chuck out an MJPEG stream and it would be up to Chromium to convert that to VP8 (or VP9?) which it wouldn’t use the GPU for. Sure enough, within a few seconds of connecting it was obvious the Pi was struggling (apparently my audio was choppy and dropping out), so I disabled my video feed and carried on as normal.

Finally: needed to modify some bits in JIRA (which we use for organizing the team’s work). JIRA was … hilariously slow. Nearly a full minute to just load the web-page. That said, JIRA is hilariously, painfully, slow on just about anything I’ve used (including Ammy) so I shouldn’t be surprised there. It was useable with a good dose of patience, but I’m thankful it’s not something I spend all my time in (can’t see my boss using it with a Pi :).

Verdict for the day: bluetooth was fine; the fan-shim is probably worthwhile for heavy workloads; USB web-cams are not going to be happy on video calls - use a camera module if at all possible (though cable length may be an issue).

Wednesday

Switched up to a Pi4 8Gb today. I was satisfied with the 4Gb’s performance with a decent dose of swap-file but was now interested in whether the 8Gb would swap at all and whether the (hypothetical) lack of swap would lead to any serious performance difference.

I transferred the fan-shim to this Pi so I could compare just the difference in RAM rather than anything else. Everything else was just unplugged from the old Pi and plugged into the new Pi. This is one of the things I love about the Pi: transferring all your storage between them is trivial (whether SD card or USB hard-drive based, it’s a lot easier than messing around inside a PC tower!).

There was some small faff getting the EEPROM configured to handle USB boot, but I had an SD card already setup to deal with that so it didn’t take more than a minute or two, then we were off.

The day was frankly uneventful. The usual mix of meets, compiling, emailing, etc. Only one thing of note really: no swapping. With a judicious set of JIRA, Gmail, and other hogs open I did manage to push the Pi to ~70% RAM usage but never managed to get it to actually swap (beyond trivial levels).

However, I can’t claim I actually noticed any performance difference. It felt pretty much exactly the same as the 4Gb model - the only difference being that I could see the swap monitor sat resolutely at zero all day.

Verdict for the day: the 8Gb is certainly a technical improvement, and for some that extra memory may be crucial (especially if you’re heavily into containers). Personally, I could live happily with a 4Gb plus some swap. Incidentally, even on an 8Gb I’d still keep some swap (say, 1Gb) around. The myths surrounding things being faster without swap are generally just that: myths.

Thursday

Still on the Pi4 8Gb today. I did cheat on my Google Meet calls today, and used my little laptop instead, largely because I knew one of the calls was going to involve screen-sharing and at this point I was pretty confident I knew the Pi’s limits and that I couldn’t smoothly deal with that, and ~10 people on the call too (on Google Meet, if you opt for “Audio only” it really means that and doesn’t permit screen-sharing; unsurprising but slightly annoying today).

Come the evening I had to “cheat” on another meeting call but for different reasons. I was due to give a quick talk for the Python North West group, which was using Zoom and, about 30 minutes before I was due to connect, discovered there’s no Zoom client for ARM. I wouldn’t have minded going audio-only for this one - I didn’t have any slides, just a few notes for a quick 10 minute segment. Still, it was back to the laptop again.

Everything else was basically uneventful:

Needed to do some printing for my kid (colour laser printer on the network); worked without a hitch as it usually did on Ammy
Needed to scan some pages for my partner (USB connected scanner); also worked without a hitch
Did some quick image tweaks in Gimp (for this post!) with my partner’s Wacom tablet. No problems there either (just plug’n’play), though I’m pretty sure the extra RAM of the 8Gb model helped

Slight hassle plugging and unplugging all these USB things, but necessary. The Pi’s power supply isn’t a 750W beast like Ammy’s and doesn’t like having all my USB gadgets plugged in at once. Further, given I’m also booting off a USB attached device it’d be ill-advised to try and starve that of power! Might be time to invest in a powered USB hub…

Verdict for the day: if only Chromium could use the GPU for video encode and decode here. We’ll have to see what can be done about that (this is easier said than done; I have some inkling of what the Pi Foundation have gone through to get GPU decode into their version of Chromium, and it’s non-trivial).

Also, more stuff needs compiling for ARM. After the lack-of-Zoom-client discovery I went checking around for a few things. There’s no Chrome for ARM (well, outside of Chromebooks, so it’s obviously possible), no Zoom (as noted already), and no Skype. Signal and Telegram are available, but there’s definitely some prodding to be done here…

Side note: it was amusing to note the number of sites that happily offered me an amd64 deb!

Friday

Still on the Pi4 8Gb. Cheated on the Google Meet calls again with the laptop because, well, I might as well at this point. Everything else still on the Pi4, and uneventful. Watched a few YouTube videos; slightly jerky startup for the first second, and then played entirely smoothly while I was still working in another window.

Looks like it’s happy dealing with 1 video stream (even with it just doing CPU decoding), but pushing it to do more than that (e.g. during conferencing) isn’t viable without hardware acceleration.

Wound up doing some work in the evening on the piwheels website. The Firefox and Chromium developer tools are certainly a bit sluggish, but still useable. Once used to the slower refresh rate, it didn’t slow me down.

TL;DR / Conclusion

It’s so close to being a fully viable desktop replacement. The thing holding it back for me is video acceleration on Google Meet calls. If we had that, I could effectively switch to using this for the vast majority of my work, firing up the big PC (when it’s fixed :) only for seriously heavy duty things like video editing, or PC gaming, which are pretty rare activities for me, and which it really wouldn’t be realistic to expect a Pi to accomplish.

For others? Very much depends on typical uses and software. If you need Skype or Zoom, for instance, you’ll be stuck using the web-based versions (which’ll have the same issues as I did with Google Meet) because there’s no ARM compiled native clients (yet). But if you don’t need video conferencing (okay, that’s probably a fairly major ask in the midst of a pandemic where everyone in front of a computer is probably home-working), it’s a capable replacement.

Don’t go expecting Ryzen-like performance because you’re not going to get it … obviously. But I’d hardly call it sluggish either. Running off a USB-attached SSD everything starts up quickly and runs pretty smoothly. I was also impressed that, even on the 4Gb model, despite it eating into swap a bit, nothing ever felt like it was overwhelming the system. Though you might want to invest in a powered USB-hub!

Finally, for all the talk of the Pi 4 “running hot”, I only ever needed the fan-shim for video conferencing. My other workloads failed to push it into thermal throttling (though a few got close). The board certainly gets toasty, but it’s nothing like the small fan heater that Ammy was. If only it could heat my fingers up a bit!

Final note: any issues we haven’t sorted out by release, with any known workarounds, should be mentioned in the Groovy Release Notes.

Installing the new Pi Ubuntu Desktop

2020-10-12T00:00:00+01:00

Now that Martin’s given the game away on Twitter, it’s probably a good idea for me to post a quick guide on installing the new Ubuntu Desktop for the Pi on various media. So, here goes!

Stuff you’ll need

A Raspberry Pi Model 4B with 4Gb of RAM or higher. Theoretically, the desktop can run on less but we’re only officially supporting it on these models because with less than 4Gb of RAM it’s not a particularly “useful” experience!
The Raspberry Pi Imager tool for whatever platform you’re on.
The Ubuntu Desktop for Raspberry Pi image, for now anyway. Once Groovy’s released it’ll just appear in the menu in the imager tool, but for now you can grab the pre-release beta from:

http://cdimage.ubuntu.com/ubuntu/daily-preinstalled/current/

You need the groovy-preinstalled-desktop-arm64+raspi.img.xz image from there. You may note there’s no armhf image and the simple reason for that is we’re not providing one at this time; it’s arm64 only.

Now, continue with one of the sections below…

SD Card

To install on an SD card is pretty much simplicity itself:

Start the Imager Tool
Click “Choose OS”, select “Use custom”, and select the file you downloaded earlier; you don’t need to decompress it (after release you’ll be able to pick an entry under “Ubuntu”)
Click “Choose SD Card”, select the card you want to flash
Click “Write”

Then you can boot the card on your Pi 4.

USB HDD / SSD

In Groovy we’re (finally!) adding support for booting Ubuntu directly from a USB attached hard-drive or SSD with no SSD card involved. The first thing to ensure is that you’ve got an up to date EEPROM version on your Pi 4 (as USB booting was added after the Pi 4’s release). This is easy enough to accomplish under either Raspbian or Ubuntu:

sudo apt install rpi-eeprom - only strictly necessary on Ubuntu as it’s installed by default on Raspbian, but it won’t do any harm there
sudo vcgencmd bootloader_config > bootconf.txt - extract the current bootloader configuration to a text file
sed -i -e '/^BOOT_ORDER=/ s/=.*$/=0xf41/' bootconf.txt - set the BOOT_ORDER option to 0xf41 (meaning attempt SD card, then USB mass-storage device, then repeat; see pi4 bootloader configuration for more information). Alternatively vim bootconf.txt and make the edits yourself if you don’t trust my sed-hacking :)
rpi-eeprom-config --out pieeprom-new.bin --config bootconf.txt /lib/firmware/raspberrypi/bootloader/critical/pieeprom-2020-09-03.bin - generate a copy of the EEPROM with the updated configuration
sudo rpi-eeprom-update -d -f ./pieeprom-new.bin - set the system to flash the new EEPROM firmware on the next boot
sudo reboot - to apply any changes (the EEPROM is only updated during the early stages of boot)

Note

Don’t worry about having to do this on every EEPROM upgrade because you don’t have to. Each EEPROM upgrade extracts the current EEPROM configuration and merges it with the updated EEPROM binary before flashing it.

Now that the EEPROM’s dealt with, how do we get the image onto a hard drive?

Read the SD Card instructions above
Pretend your USB-attached hard-drive is a very fat SD card and just select it instead when you click on “Choose SD Card”
erm … that’s it

You should now be able to boot your flashed drive on your Pi 4 off your USB-attached hard-drive without any SD card inserted. Be aware that some drives have issues being used to boot the Pi. In particular:

Spinning hard-disks required a lot more power than SSDs and will very likely required a powered USB hub. If you have a stonkingly huge hard-drive, see the next section too…
Hubs themselves can cause compatibility issues, so frankly you’re much better off with an SSD to boot off (typically no need for a hub and no spin-up time issues).

There’s lots of good information on both the Pi forums and various GitHub issues for debugging boot issues; here’s a selection of links in a rough “look at this first” order:

Stonkingly huge HDD / SSD

What if you have a hard-drive larger than 2Tb (or SSD if you’re richer than Croesus) and want to actually use all that space? The default MBR partitioning on the image won’t be much use as it can’t address anything beyond 2Tb. Thankfully, the Pi’s bootloader now supports GPT partitioning too so this isn’t a big deal and just requires a minor tweak to the partition table.

The following instructions are Linux-specific. I’m sure it’s possible to do this on Windows and Mac OS too, but you’ll have to Google that if you need it. I would advise doing this with the drive attached to another machine (i.e. when you’re not booted from it). It might be possible to do this on a “live” system, but I haven’t tried that so YMMV!

sudo apt install gdisk - just to make sure you’ve got gdisk (the GPT partitioning tool) installed
sudo gdisk /dev/sdX - where sdX is whatever your hard-drive is called (consult the end of dmesg for a hint)
If gdisk complains about a valid MBR and a corrupt GPT, don’t worry (this can happen when re-imaging drives that’ve had a GPT partition table before). Just select 1 to use the valid MBR and continue
gdisk should’ve converted your MBR partition table to GPT in RAM (not written to the disk yet). Enter p to print the table and make sure it looks relatively sane:
- two partitions,
- a small one with type 0700 (“Microsoft basic data”) which is the boot partition, and
- a larger one with type 8300 (“Linux filesystem”) which is the root partition
Enter w to write the new partition table to the disk and exit.
Safely eject the drive and boot from it!

Note that you can do this before the first boot, or you can do it later on. However, if you do this after the first boot you may need to manually resize the root partition to take up the full disk size (if that’s what you want). If you do this before the first boot, the image should expand the root partition to fill the disk (I’ve checked resizing works under GPT, but I haven’t got a spare HD >2Tb to actually try that with! :).

Have you actually tested this?

For the last week I’ve been using the desktop image on and off on a Pi 4 with 4Gb of RAM (I’ve got an 8Gb model here, but obviously the point is to try and validate that 4Gb is “sufficient”), with my little laptop as a backup. However, this week I’m embarking on something a little more adventurous: using it full time as my only desktop (laptop strictly for emergency backup only, and no desktop either because … well, it sort of blew up).

The specific configuration I’m using is: booting off a small 120Gb SSD I had spare. Anyway, if I come up with any tweaks along the way I’ll post a write-up later this week. Wish me luck!

Ubuntu Desktops on the Pi

2020-04-17T00:00:00+01:00

See Also: A look at the Hirsute desktop image, Jammy desktop image, and Making desktops with cloud-init.

One of the things I’ve been looking at recently is the relative performance of our desktop packages on the Pi (a lot of people seem to be trying to use the Pi 4 as a desktop work-horse - and it is reasonably capable of this with the right selections).

Anyway, here’s some quick notes from my experiments. Firstly, the various desktop packages and some comments (just in the order I tried them):

ubuntu-desktop: A bit sticky here and there (particularly during login and application selection - basically anywhere Gnome tries to do any animation), but tolerable.
xubuntu-desktop: A bit slicker than the default on the Pi; could actually be a reasonable choice for a default desktop.
lubuntu-desktop: It’s not pretty, but damn this flies! No real surprise there given it’s LXQt which is the basis of Raspbian too. With a bit of layout tweaking this would definitely be my go to choice of desktop under Ubuntu on the Pi.
kubuntu-desktop: Ouch, this is painful. Don’t misunderstand me - I love Kubuntu generally (I’m one of those old stick-in-the-muds that bemoans all the options that’ve disappeared from Gnome over the years and loves that they’re all still there on KDE). However, it’s undeniably heavier and on the Pi, even on a 4Gb Pi 4, it’s just unusable.
ubuntu-budgie-desktop: Oooh, pretty! Sadly, not that functional. The start menu is better than the default but (shock! horror!) the terminal won’t start, so that’s this experiment prematurely ended (yes, I could switch tty and fiddle around but the Mac-like prettiness hasn’t tempted me that much :).

Compatibility notes

I was only testing the arm64 images; I’d expect similar results under armhf though. Several things are worth adding to improve compatibility or enable hardware (these apply to pretty much all the desktops above):

Graphics

Add dtoverlay=vc4-fkms-v3d to /boot/firmware/syscfg.txt to enable the “fake” KMS frame-buffer. Lots of GUI instability without this. For all those that want this to be the default: so do I, but there’s issues with doing this that need solving first (breaks u-boot’s video console, and breaks boot on the 3A+).

Audio

From Focal (20.04) onwards, dtparam=audio=on should be the default in the boot configuration so audio should at least be present. However, if you’re installing a desktop environment you also want to tweak a pulse-audio parameter to improve its scheduling and avoid choppy audio. Edit /etc/pulse/default.pa and find the line:

load-module module-udev-detect

And append tsched=0 so that the line reads:

load-module module-udev-detect tsched=0

(thanks to BranLoux896 on the Raspberry Pi forums for mentioning this!)

Login managers

Most of the desktop packages provide a selection of display managers (gdm3, lightdm, ssdm, etc.) on installation. I attempted most selections and found that lightdm doesn’t work for some reason I haven’t tried to figure out yet, but gdm3 and ssdm work fine.

However, one issue with gdm3: it always defaults to the “ubuntu” desktop even if you’ve installed, say, lubuntu-desktop. After selecting the user to login as, click the gears at the bottom right and select the actual desktop you installed before completing the login :)

Wireless & Bluetooth

Install the “iw” package (which’ll pull in “crda”) and then run sudo iw reg set GB (if you’re in the UK like me; substitute GB for your region code otherwise) to set the WiFi region correctly (usually enables a few channels that aren’t enabled in the default “world region”). I’ll endeavour to get this in the image by default in Focal, but you’ll still need to set the region code for now.

Install the “pi-bluetooth” package to enable Bluetooth. This “just works” once the package is installed.

Remote desktops

I ran a quick experiment in Terminal Services-style remote desktops with xrdp under lubuntu-desktop (the only one I figured would be light enough to be worthwhile for such endeavours). To try this:

install the xrdp package
add the xrdp user to the ssl-cert group
reboot

Then you should be able to connect remotely using an RDP client (like remmina) providing you specify the username and password in the connection settings.

Another quick experiment in screen-scraping style remote desktop was semi-successful with the tigervnc-scraping-server package but a further experiment to do TS-style remote desktop with VNC wouldn’t work (some issue with GDM I didn’t have time to dig into).

Power saving

Power saving is … rather pointless. I forget which desktop I saw this on, but one of them decided to put the Pi to sleep on idle timeout and … that just effectively switched the Pi off! So, turn off all those suspend timers :)