Playing with Blocks - It’s All Connected

Three (?) Rowdy Layers

Rather than go through all the individual commands, which would result in this post looking like a foreword followed by the dump of a shell script, I’ll go through the high level procedure because the detail is almost exactly the same as the original articles but for a few minor differences and some deliberate omissions.

Firstly, we’ll deal with the content of the first article and set up LVM:

1. Follow the first article instructions until it’s time to create the PV (with pvcreate):

   • Plug in the SD card
   • Unmount any auto-mounted partitions
   • Repartition with gdisk

2. At this point we need to create the PV with 16MB less space than is available in the second partition, to leave a little room for the LUKS header around the PV:

   • From the gdisk session, you should have a printout of the new partition table; if you don’t, run sudo gdisk -l /dev/mmcblk0 (assuming /dev/mmcblk0 is your SD card)

   • For the second partition, calculate the number of sectors it occupies (which is simply end sector minus start sector) and then subtract 32768 (each sector is 512 bytes, so 32768 is 16MB). For example, on a 32GB SD card:

     $ sudo gdisk -l /dev/mmcblk0
     GPT fdisk (gdisk) version 1.0.8
     
     Partition table scan:
       MBR: protective
       BSD: not present
       APM: not present
       GPT: present
     
     Found valid GPT with protective MBR; using GPT.
     Disk /dev/mmcblk0: 62333952 sectors, 29.7 GiB
     Model: STORAGE DEVICE
     Sector size (logical/physical): 512/512 bytes
     Disk identifier (GUID): A9BDFD97-F2A5-4419-9A83-B0160820FDBB
     Partition table holds up to 128 entries
     Main partition table begins at sector 2 and ends at sector 33
     First usable sector is 34, last usable sector is 62333918
     Partitions will be aligned on 2048-sector boundaries
     Total free space is 4061 sectors (2.0 MiB)
     
     Number  Start (sector)    End (sector)  Size       Code  Name
        1            2048         1048576   511.0 MiB   0700  system-boot
        2         1050624        62333918   29.2 GiB    8E00  lvm
     $ PV_SECTORS=$((62333918 - 1050624 - 32768))
     $ echo $PV_SECTORS
     61250526
     

   • Proceed to creating the logical volume, but explicitly specify the number of sectors to use when creating the PV with --setphysicalvolumesize and the number of sectors with the suffix “s”:

     $ sudo pvcreate --setphysicalvolumesize ${PV_SECTORS}s /dev/mmcblk0p2
     Physical volume "/dev/mmcblk0p2" successfully created.
     $ sudo vgcreate pivg /dev/mmcblk0p2
     Volume group "pivg" successfully created
     $ sudo lvcreate --size 8G --name root pivg
     Logical volume "root" created.
     

3. Finish the first article’s instructions:

   • Unpacking the image
   • Creating a loop device for the image
   • Transferring the content of the image partitions to the card
   • Updating the boot command line and fstab to point to the “new” root device
   • Unmount the root and boot partitions, and disable the VG
   • Don’t eject the SD card, we still need to do some surgery!

LUKS Dujour

Now it’s time to move on to the second article and set up the encrypted LUKS container around the physical volume (PV):

1. Skip everything up to the encryption step:

   • No need to download another image
   • No need to boot the image yet
   • No need to unmount stuff (we already did that)
   • No need to shrink partitions (the root partition hasn’t been auto-expanded because we haven’t booted anything, and we’ve pre-shrunk the PV containing the root)

2. Now it’s time to encrypt the second partition containing the PV:

   • First the encryption; this is exactly as in the second article:

     $ sudo cryptsetup reencrypt --encrypt --reduce-device-size 16M --cipher xchacha12,aes-adiantum-plain64 /dev/mmcblk0p2
     
     WARNING!
     ========
     This will overwrite data on LUKS2-temp-a823bd67-a632-41cb-955e-8776b8b3fb5b.new irrevocably.
     
     Are you sure? (Type 'yes' in capital letters): YES
     Enter passphrase for LUKS2-temp-a823bd67-a632-41cb-955e-8776b8b3fb5b.new:
     Verify passphrase:
     Finished, time 36:53.687, 29915 MiB written, speed  13.5 MiB/s
     

   • Next we open the encrypted container:

     $ sudo cryptsetup open /dev/mmcblk0p2 rootfs
     Enter passphrase for /dev/mmcblk0p2:
     

   • Then we re-scan for physical volumes (PVs) and re-activate the pivg volume group (VG) we created earlier; this isn’t in the original article but is necessary as we’ve altered the mapper configuration:

     $ sudo pvscan
       PV /dev/mapper/rootfs   VG pivg            lvm2 [29.20 GiB / 21.20 GiB free]
       Total: 1 [29.20 GiB] / in use: 1 [29.20 GiB] / in no VG: 0 [0   ]
     $ sudo vgchange -ay pivg
       1 logical volume(s) in volume group "pivg" now active
     

   • We skip over all the resizing bits because we’ve already established the size of the root volume, and just mount the root file-system (we don’t need to mount the boot file-system). This is, again, slightly different to the original article because the root file-system is a mapper device, pivg-root as in the first article:

     $ sudo mkdir -p /mnt/root
     $ sudo mount /dev/mapper/pivg-root /mnt/root
     

3. Then a bit (but only a bit) of configuration manipulation:

   • Add our entry to /etc/crypttab. Please heed the advice in the original article regarding the source device here; this is the source device as seen by the Pi:

     $ cat /mnt/root/etc/crypttab
     # <target name> <source device>         <key file>      <options>
     $ echo "rootfs  /dev/mmcblk0p2  none  luks,discard" >> /mnt/root/etc/crypttab
     $ cat /mnt/root/etc/crypttab
     # <target name> <source device>         <key file>      <options>
     rootfs  /dev/mmcblk0p2  none  luks,discard
     

   • Skip the bits about editing fstab and the boot command line; we’ve already done everything we need to for those. Just unmount root file-system, and close the encrypted container:

     $ sudo umount /mnt/root
     $ sudo cryptsetup close rootfs
     

4. Now just follow the second article to the end:

   • Eject the card and boot it on your Pi
   • There’s a long delay while the initial root mount fails
   • In the emergency prompt that appears, open the encrypted device and exit the prompt
   • Once booted, update the initramfs
   • Reboot to make sure everything works!

Future Cases

Congratulations! At this point you should have a setup eerily similar to the one I use on my main development Pi, albeit that one boots off a large SSD instead of an SD card. Future things worth exploring with this setup:

• How to get plymouth prompting nicely for the password; plymouth’s always been on the Pi server images but for some reason it doesn’t work properly (unlike the Pi desktop images where it’s happy). I had a cursory look into this a couple of cycles ago, but didn’t get anywhere at that time.
• How about ditching password encryption and using a key on removable storage. For instance, if booting off SD card, how about using a USB key with the encryption key. Or for my favoured setup where I boot off a USB3 attached SSD drive, how about using an SD card with the key?
• With this configuration I usually set up LXD with an LVM thin-pool for its default storage. This usually requires taking a crow-bar to LXD and forcing it to use the thin-pool (with lvm.vg.force_reuse; LXD doesn’t like working with a non-empty VG for reasons I’ve never full understood, but at least it’s possible). Let me know in the comments if it would be useful to have another article covering this!