It's Pi all the way down...

by

Noble’s release is coming up (as I edit this, it’s now many months in the past and even noble’s .1 release is long gone), so it’s definitely time I write something about this most difficult of births…

Welcome back, pointers

What’s new? For Pi users the big news is that this is an LTS, and it’ll be the first Ubuntu LTS that supports the new hardware of the day: the Pi 5. For those not familiar, that means 5 years of free support (and another 5 paid, or free at various tiers of usage).

All well and good, but what’s new in the software itself? Not quite as much as I’d hoped. The final part of my series on NBD (part 3 following on from part 1, part 2) won’t operate until the first second point release (24.04.2) scheduled for February next year, as I didn’t manage to shepherd the nbd-client package through into main in time. Sadly, there wasn’t time to address libcamera either and so, at present, it’s completely broken and won’t even list the Pi’s camera modules. This will be a priority to fix in the next (oracular) next +1 (plucky) cycle (for eventual SRU to noble). In fact, those are just the tip of a rather long list of stuff I need to fix.

But enough doom and gloom. One piece of very good news is that the Firefox snap (or more precisely the GNOME content snap) finally got updated to mesa version 23.2. This brings full hardware acceleration on the Pi 5 to the browser, so instead of the hilariously pathetic 2fps we got with the classic Aquarium benchmark in mantic, it’s more like 40-60fps 30-40fps in noble (I managed 60fps at 1080p full-screen when first testing this pre-release, but by the time we got to release it seems something else in mesa changed and we were back down to 30-40 — still, better than 2!).

GNOME’s also been updated to version 46. Amongst many changes, this fixed a long standing and, as we’ll see, quite pertinent issue with its notifications. As you may have noticed in prior versions, desktop notifications could have “actions” associated with them which would appear as buttons beneath the notification. For example, when plugging in a storage device the action on the associated notification would be “Open with Files”.

A desktop notification showing a new "69GB volume" has been mounted, with an action button at the bottom labelled "Open with Files"

But if you happened to miss the notification, and it wound up in the tray, the actions were gone…

The "69GB volume" desktop notification from earlier, but now in the backlog tray, missing the action button

As the linked ticket showed, this has been a pretty long standing bug in the desktop, but GNOME 46 finally squashes it, and just in time for one of the new Pi-specific bits in noble. The Pi 5’s new PMIC includes the ability to inform users when the previous reboot was caused by an undervolt (a “brownout” because the power-supply isn’t up to snuff). The RaspiOS desktop helpfully informs users via a desktop notification when this happens, and now so does Ubuntu.

A desktop notification from the "Raspberry Pi PMIC Monitor" indicating "This power supply is not capable of supplying 5A; power to peripherals will be restricted". Beneath the warning are two action buttons: "More information", and "Don't show again"

As you can see, the Ubuntu version also includes actions to silence the warning, and to get more information. But when the warning disappears into the system tray, with previous versions of GNOME there were no actions and, worse still, the notification was truncated. Now you can expand it in the tray to see everything you need.

The desktop notification from earlier about the power supply, but now in the backlog tray with a truncated message and no action buttons. However, an expansion button is now present in the top right The desktop notification from earlier about the power supply, but expanded to show the full message, and the two action buttons, "More information" and "Don't show again"

What a piece of work…

Another major piece of news is a new official flavour of Ubuntu for the Raspberry Pi: Edubuntu have spun up a pre-installed Raspberry Pi desktop image. There are a wealth of applications pre-installed here, including several I was previously unaware of like Marble, a fantastic globe explorer including some wonderful ancient map modes (ah, the island of California!), Kig, a constructive geometry application, and laby, a very nice gamified coding tutor supporting several languages.

How infinite in faculty

The other big news is that, while armhf (32-bit ARM) as an architecture is still supported for 24.04’s lifetime, we will not be producing Raspberry Pi images for it. This means there is no upgrade path for users currently on (for example) 22.04 armhf images. Those users will need to re-image with the arm64 image.

Why?

There are multiple reasons here, but the primary reason is a combination of the Pi 5’s limited support for 32-bit architecture, and a difference in the way RaspiOS and Ubuntu support multiple models of Raspberry Pi with a small number of images.

Traditionally (and even today in some parts of the SBC space), OS vendors produced an OS image per board. Imagine this in the context of Raspberry Pi. There would be an image for the Raspberry Pi 2B, another for the Raspberry Pi 3B, possibly another for the 3B+, certainly another for the 4B, and so on, each tailored to their specific board. There might be further variations on top of this, like armhf and arm64 images for the 3B, two more for the 4B, maybe different desktop variants, the list goes on.

The result would be an immensely confusing proliferation of images, with the user having to know exactly which board they had, and typically not being able to move eminently moveable storage (like an SD card) between boards.

RaspiOS (or Raspbian formerly) was quite the pioneer here. Early on, as the Pi family grew from the B+ to the 2B and beyond, it was recognized that asking teachers (particularly, given the educational focus) to know exactly which board(s) they had, and figure out which image they needed to download, was simply not practical. This is why there’s still a single RaspiOS image that works on absolutely all Raspberry Pi boards: keep it simple for the user.

Obviously there’s been some proliferation (a lite variant without a desktop, and architecture variations), and the situation of having a single image that works universally probably can’t last forever, but ultimately the goal of minimizing the number of images is still sought.

What about Ubuntu?

When I first joined the Foundations team several (nearly six?) years ago, the “image per board” model was being followed. There was an Ubuntu image for the 2B, another for the 3B, and they were thinking about another for the (at the time, forthcoming) 4B [1], assuming they could drop the 2B image in due course as it ran out of support.

The first thing I had to correct was the notion that the 2B was going to drop out of support anytime soon. Raspberry Pi have always promised 10 years of production on all models, and so far they’ve not disappointed: as of writing this the 2B is still available. This also meant that following the “image per board” model was completely unsustainable. The resulting number of images would be insane (not to mention the support burden). So I switched Ubuntu to a “minimal images” model, specifically one image for armhf and another arm64 [2].

How?

While RaspiOS and Ubuntu both follow the “minimal images” model, we both do so in quite different ways. What matters here is the separation between “kernel” and “userland”, and what architectures are supported by each SoC. In userland, things are relatively simple:

SoC ARMv6 ARMv7 ARMv8
2835 (Pi 1, Pi Zero)    
2836 (Pi 2)  
2837 (Pi 3, 3+, Zero 2)
2711 (Pi 4, 400)
2712 (Pi 5, 500)

As you can see, ARMv6 is a “universal” architecture for userland, so compile everything except the kernel for ARMv6 and your image will work on all boards. Great! But what about the kernel…

SoC ARMv6 ARMv7 ARMv8
2835 (Pi 1, Pi Zero)    
2836 (Pi 2)    
2837 (Pi 3, 3+, Zero 2)  
2711 (Pi 4, 400)  
2712 (Pi 5, 500)    

Now things are more complicated. For a truly universal image, you require separate kernels. This is the route RaspiOS has taken on their 32-bit images. The userland is ARMv6, but there are four [3] separate kernels on the image, with the bootloader selecting the appropriate one for the board.

At Ubuntu, we don’t have the resources to maintain four separate kernel builds for the Raspberry Pi [4]. However, we also never (officially) supported the Pi 1 models. With no need for ARMv6 support, the route we took in Ubuntu was a simpler “pure architecture” model: the armhf images use an ARMv7 kernel and userland; the arm64 images use an ARMv8 kernel and userland.

To be clear, I don’t think either model is perfect. There are advantages, disadvantages, and compromises all round here. This is just how things are as a result of history and the limitations and resources surrounding each distro. Still, the goal of minimizing the number of images is a worthwhile one however it is achieved.

Good for you, it’s a growth industry…

There’s a problem. Note that the 2712 on the Pi 5 cannot boot an ARMv7 kernel. So with noble we were in the position of having an armhf image that could boot on some modern Pi models, but not all. The options available were:

  1. Retro-fit an arm64 kernel onto the armhf userland images and have the bootloader select the right one, a bit like RaspiOS. Ultimately this was too complex to support (several systems assume there is “one kernel” on the system, not multiple that need to be kept in lock-step)
  2. Make two armhf images, one with an armhf kernel and one with an arm64 kernel. Given this would only be of use to Pi 5 users who wanted an armhf userland, this seemed far too niche to expend time and effort on (you’ll see just how niche in a bit!).
  3. Make the armhf image, but explicitly state it’s not supported on the Pi 5, and likely any future Raspberry Pi boards.
  4. Drop the armhf image, leaving it as a legacy architecture only for old releases.

Personally, I would have preferred the third option. However, armhf has always had minimal (and waning) interest from Ubuntu users. A few years ago, for every one armhf download there were 5 arm64 downloads. Today, that ratio is 1:30 and rising.

Given that arm64 support covers all boards with the exception of the Raspberry Pi 1 (which we never supported) and the Raspberry Pi 2 (which is easily classified as a “legacy” board at this point, i.e. people should not be planning new products based on this board), the decision was made to simplify things in this release, and drop armhf for the Raspberry Pi. The resulting arm64 images available for noble (and onwards) work on every Raspberry Pi board we’ve ever supported except the Pi 2.

The support matrix for all models of Raspberry Pi, plotted against all currently supported versions of Ubuntu can be found in the Raspberry Pi section of the new Ubuntu Boards documentation project. This aims to replace the out of date (and rather variable) Ubuntu wiki pages with a single source of information for installing Ubuntu on all manner of single board computers (including Raspberry Pi, naturally).

Let’s have a drink and forget the whole damned thing

To say it’s been a challenging year would be an understatement of frankly comic proportions.

Between the extensive work on the solution to the Year 2038 problem (i.e. the 64-bit time_t transition on armhf), the challenges involved in the xz-utils hack, some considerable churn in personnel, and the mess we found ourselves in over the D0 stepping (more on that in the next post!), 2024 has been … tough. Still, I’m vaguely hoping 2025, and 25.04 (plucky) might be less “eventful”.

I’ve got a new helper on the Pi side of Foundations, with the result that we might actually get the camera working this cycle (at least, that’s the plan). Various long-standing architectural issues might resolve this cycle (he says, looking at the desktop installer). And there’s a fair chance I won’t crash into any more hardware incompatibilities until … oh, at least the middle of the year.

I hope.

Merry Christmas, everyone!


[1]There was no image for the B+ (or prior) boards because Ubuntu never (officially) supported the ARMv6 architecture on those boards
[2]Both were server images only at the time — we didn’t have a desktop offering as the Ubuntu desktop was far too heavy to run on the 3B+, which was the latest model at the time
[3]There are two ARMv7 kernels: one for 2837 boards (Pi 3 era), one for the 2711 (Pi 4). By default, the 2711 boots the ARMv8 64-bit kernel, but can be forced back to 32-bit for various reasons, in which case it boots the kernel7l.img rather than kernel7.img
[4]More precisely: we maintain dozens of different kernel variations but rarely more than one or two configurations for a specific board or vendor