Wherefore UART Thou?

All things are ready, if our packages be so

Pretty much all communication with MicroPython is done via serial link. In more advanced cases this involves hooking up wires to the serial pins on the Pico itself, but that’s not the focus of this post. We’re just going to take the easy route of talking to a serial port emulated over the Micro-USB connection. The things you’ll need to follow along:

• A computer running Ubuntu 22.10 (Kinetic). This can be a PC, or a Raspberry Pi. Basically anything with a USB port.
• A Raspberry Pi Pico. This can be the Pico or Pico W; the examples assume the latter but will work equally well on the former (I’ll touch on the W’s WiFi facilities in a future post). For that matter, these examples should be easily adaptable to any other RP2040 or ESP32-based boards.
• A micro-USB cable to connect your Ubuntu computer to your Pico.

We start by installing a MicroPython firmware on your Pico. The Raspberry Pi MicroPython documentation covers this perfectly but a quick overview would be:

1. Download the firmware (a file with a “.uf2” extension) for your specific board; the Pico and Pico W have separate builds.
2. Hold down the BOOTSEL button (the only button) on your Pico, while connecting it to your computer with the micro-USB cable.
3. Release the BOOTSEL button once it’s connected.
4. After a few seconds you should see a USB storage drive called “RPI-RP2” appear on your computer. Open this device in your file explorer.
5. Copy the “.uf2” file you downloaded over to the RPI-RP2 drive.
6. After a few seconds the “RPI-RP2” drive should disappear; this indicates the Pico has flashed the firmware and rebooted into the MicroPython prompt.

Now that the Pico’s set, let’s install some new toys on your Ubuntu machine. The rshell tool was new in jammy (22.04), but mpremote is new to kinetic (22.10):

$ sudo apt install pyboard-rshell micropython-mpremote

You may be wondering why I’ve excluded “thonny” here. Partly that’s because the Raspberry Pi documentation covers it perfectly already. However, it’s also because these two command line tools have some interesting facilities unique to them, and they’re how I tend to work with MicroPython.

The fault … is not in our cable, but in our file modes

Both these tools communicate over a serial connection with MicroPython running on the microcontroller, so the first thing we need to establish is … which serial device? The vast majority of the time, it will be /dev/ttyACM0 but you can double check this by starting a terminal window and running:

$ sudo dmesg -w

This will dump the kernel ring buffer (basically the kernel log) and “follow” it, printing out new messages as they occur. Now plug the Pico back in again, and you should see something like the following appear:

[24893.432219] usb 1-2.4: new full-speed USB device number 13 using xhci_hcd
[24893.759502] usb 1-2.4: New USB device found, idVendor=2e8a, idProduct=0005, bcdDevice= 1.00
[24893.759506] usb 1-2.4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[24893.759508] usb 1-2.4: Product: Board in FS mode
[24893.759509] usb 1-2.4: Manufacturer: MicroPython
[24893.759510] usb 1-2.4: SerialNumber: 1234567890abcdef
[24893.778722] cdc_acm 1-2.4:1.0: ttyACM0: USB ACM device

That last line is the important one, indicating that the new serial device is called ttyACM0, so the device path is simply /dev/ttyACM0. Next we should check that your user has access to this serial device:

$ ls -l /dev/ttyACM0
crw-rw---- 1 root dialout 166, 0 Oct 19 16:01 /dev/ttyACM0
$ groups
ubuntu adm dialout cdrom floppy sudo audio dip video plugdev netdev lxd

Note that the serial device belongs to user root and group dialout, and that our user also belongs to the dialout group. If you have any issues talking to MicroPython, this is the first thing to check. On server Ubuntu images, the default user belongs to the dialout group but sadly this isn’t the case on Ubuntu desktop images (LP: #1923363). If your user does not belong to the dialout group, run the following:

$ sudo adduser $USER dialout

Afterwards, log out and log back in again (group membership changes are only evaluated at login time).

… and make a heaven of rshell

The first thing to try is just to get to the MicroPython REPL (the interactive prompt). We’ll export RSHELL_PORT so we don’t have to continually specify the port on the command line (if your port is /dev/ttyACM0, that’s the default anyway but it never hurts to be certain):

$ export RSHELL_PORT=/dev/ttyACM0
$ rshell repl
Using buffer-size of 32
Connecting to /dev/ttyACM0 (buffer-size 32)...
Trying to connect to REPL  connected
Retrieving sysname ... rp2
Testing if ubinascii.unhexlify exists ... Y
Retrieving root directories ...
Setting time ... Oct 19, 2022 17:42:43
Evaluating board_name ... pyboard
Retrieving time epoch ... Jan 01, 1970
Entering REPL. Use Control-X to exit.
>
MicroPython v1.19.1 on 2022-10-14; Raspberry Pi Pico W with RP2040
Type "help()" for more information.
>>>
>>> import sys
>>> sys.version_info
(3, 4, 0)

Several things to note here. Firstly, the message “Entering REPL. Use Control-X to exit”. Now you know how to get back to your command line! Secondly, that the current version of MicroPython is (mostly) compatible with the Python 3.4 language specification. Actually, it includes some features from later versions too (like f-strings from version 3.6), but these aren’t feature-complete yet:

>>> f"Compatible with Python {'.'.join(str(i) for i in sys.version_info)}"
'Compatible with Python 3.4.0'

Now, hit Ctrl-X to quit the MicroPython REPL, and let’s explore the Pico’s file-system. rshell implements a number of common file operations including ls, cp, rm, and cat but the most important detail is that it considers the Pico’s file-system to exist under the path /pyboard/. If we list that path currently we’ll find it’s empty:

$ rshell --quiet ls /pyboard/

Let’s create a basic script for MicroPython to run on startup, copy it to the board and run it on boot. The script we’ll create is the hardware equivalent of the classic “Hello, world!” script: blinking an LED. Fire up your favourite editor and enter the following script:

1 from machine import Pin
2 from time import sleep
3 
4 led = Pin('LED', Pin.OUT)
5 while True:
6     led.toggle()
7     sleep(1)

This script should work on a Pico W, but on a Pico the built-in LED is wired slightly differently. If you’ve got a Pico, change 'LED' to 25 (which is the GPIO pin the internal LED is connected to).

Save this script as main.py and then copy it to your connected Pico like so:

$ rshell --quiet cp main.py /pyboard/
Copying '/home/dave/main.py' to '/pyboard/main.py' ...
$ rshell --quiet ls /pyboard/
main.py

So far, so good, but nothing seems to be happening yet. That’s because we need to reset the Pico to get the script running. There’s a few ways to do this: the simplest is to unplug and then re-connect the Pico. That works but is rather unsatisfying as it also reset the serial connection which doesn’t help with debugging. The other method is to start the REPL and reset MicroPython with the common Ctrl+D shortcut:

$ rshell --quiet repl
Entering REPL. Use Control-X to exit.
>
MicroPython v1.19.1 on 2022-10-14; Raspberry Pi Pico W with RP2040
Type "help()" for more information.
>>>
>>> ^D
MPY: soft reboot

The ^D above indicates where I pressed Ctrl+D. At this point your LED should be happily blinking away happily but … what happened to our REPL? It’s still there, but it’s sat in our while loop. In other words, we won’t get our prompt back until we break out of the loop. Press Ctrl+C to do so and you should get back to the regular MicroPython prompt:

^C
Traceback (most recent call last):
  File "main.py", line 7, in <module>
KeyboardInterrupt:
MicroPython v1.19.1 on 2022-10-14; Raspberry Pi Pico W with RP2040
Type "help()" for more information.
>>>

Of course, having done that our program has also stopped (the LED won’t be blinking any more), but usefully the REPL is still running in the context of our (terminated) main.py script. In other words, we can poke around its globals and even manipulate things:

>>> led
Pin(WL_GPIO0, mode=OUT)
>>> led.off()
>>> led.on()

Generally speaking, this is how I handle the vast majority of my debugging needs in MicroPython. Either a few carefully inserted print statements in my program (which can then be watched from the REPL), or hitting Ctrl+C at some appropriate point and poking around the program’s state are generally sufficient.

How about debugging a more complex example? The following is a simple script that blinks the LED with a morse-code message:

 1 from machine import Pin
 2 from time import sleep
 3 
 4 def morse(msg, dit=0.1):
 5     dah = dit * 3
 6     intra_char = dit
 7     inter_char = dah
 8     inter_word = dit * 7
 9     codes = {'S': '...', 'O': '---'}
10 
11     for word in msg.split():
12         sleep(inter_word)
13         for char in word.upper():
14             sleep(inter_char)
15             for bit in codes[char]:
16                 sleep(intra_char)
17                 led.on()
18                 if bit == '.':
19                     sleep(dit)
20                 elif bit == '-':
21                     sleep(dah)
22                 else:
23                     assert False, 'invalid morse char!'
24                 led.off()
25 
26 led = Pin('LED', Pin.OUT)
27 morse('SOS')
28 morse('PICO')

If we upload this as main.py and run it, we find the second call to morse will fail because we haven’t got enough characters in our codes dictionary:

$ rshell --quiet cp main.py /pyboard/
Copying '/home/dave/main.py' to '/pyboard/main.py' ...
$ rshell --quiet repl
Entering REPL. Use Control-X to exit.
>
MicroPython v1.19.1 on 2022-10-14; Raspberry Pi Pico W with RP2040
Type "help()" for more information.
>>>
>>> ^D
MPY: soft reboot
Traceback (most recent call last):
  File "main.py", line 28, in <module>
  File "main.py", line 15, in morse
KeyError: P
MicroPython v1.19.1 on 2022-10-14; Raspberry Pi Pico W with RP2040
Type "help()" for more information.
>>>

How could we determine this if we didn’t understand the exception that’s thrown? The experienced Python user may be tempted to throw something like print(repr(locals())) in the morse function to dump the local variables after they’re set up. However, MicroPython has some subtle differences to CPython and one of them is that locals doesn’t really work.

This is one of the reasons a lot of MicroPython scripts use more globals than might typically be considered “good practice” in regular Python. Another is that MicroPython scripts (necessarily) tend to be a lot simpler, often fitting in a single module so spilling state into globals isn’t quite such an egregious habit there.

Hence, in this case if I were confused about the KeyError I might move most of the locals in morse into the global namespace and dump the state from within the function. In this case the top of the updated script looks something like this:

 1 from machine import Pin
 2 from time import sleep
 3 from logging import fatal, info
 4 
 5 dit = 0.1
 6 dah = dit * 3
 7 intra_char = dit
 8 inter_char = dah
 9 inter_word = dit * 7
10 codes = {'S': '...', 'O': '---'}
11 
12 def morse(msg):
13     for word in msg.split():
14         sleep(inter_word)

At this point, I could query codes from the REPL after the failure and probably figure things out. This approach can even be advantageous to your script anyway as it guarantees that these variables aren’t being recalculated on each run of morse (programming for a micro-controller sometimes requires a somewhat different mindset to programming for a “full-blown” computer).

Under baud’s heavy burden do I rsync

What about investigation of something “after the fact”, when the Pico’s not plugged into a serial console? In this case there’s two methods I tend to turn to:

1. Take a leaf out of Linux’s book: log stuff to text files!
2. Take a hint from the Pi’s bootloader (and the script above): blink the LED to indicate different error conditions

We have to be bit careful with the first point: we don’t have the practically infinite logging space that we’d have on a PC. For that reason I tend to have my error logging routines start their file from scratch on each run. I also combine a fatal routine (which should log an exception) with an infinite blinking routine.

The following is a slightly cut-down version of a logging.py module I often use:

 1 import os
 2 import io
 3 import sys
 4 import time
 5 from machine import Pin
 6 from time import sleep
 7 
 8 log_filename = 'log.txt'
 9 log_file = None
10 
11 def format_msg(level, msg, ts=None):
12     if ts is None:
13         ts = time.localtime()
14     return f'{ts[0]:04d}-{ts[1]:02d}-{ts[2]:02d}T{ts[3]:02d}:{ts[4]:02d}:{ts[5]:02d} {level:-8s} {msg}'
15 
16 def log(level, msg):
17     global log_file
18     if log_file is None:
19         log_file = open(log_filename, 'w')
20     s = format_msg(level, msg)
21     print(s)
22     log_file.write(s)
23     log_file.write('\n')
24 
25 def debug(msg):
26     log('debug', msg)
27 
28 def info(msg):
29     log('info', msg)
30 
31 def warn(msg):
32     log('warning', msg)
33 
34 def error(msg):
35     log('error', msg)
36 
37 def exception(exc):
38     with io.StringIO() as buf:
39         sys.print_exception(exc, buf)
40         buf.seek(0)
41         for line in buf:
42             log('error', line.rstrip())
43 
44 def fatal(exc, blink=3, pin=None):
45     exception(exc)
46     if pin is None:
47         machine = os.uname().machine
48         if 'Pico W' in machine:
49             pin = 'LED'
50         elif 'Pico' in machine:
51             pin = 25
52         else:
53             error(f'Cannot blink: unknown board {machine}')
54     if pin is not None:
55         led = Pin(pin, Pin.OUT)
56         while True:
57             for i in range(count):
58                 led.on()
59                 sleep(0.2)
60                 led.off()
61                 sleep(0.2)
62             sleep(1)

And here’s our updated morse script (still broken!) with some sensible logging calls, and a catch-all at the end which will dump the (otherwise unhandled) exception to the log file and enter an infinite blinking loop to warn us that something’s gone horribly wrong:

 1 from machine import Pin
 2 from time import sleep
 3 
 4 import logging
 5 
 6 dit = 0.1
 7 dah = dit * 3
 8 intra_char = dit
 9 inter_char = dah
10 inter_word = dit * 7
11 codes = {'S': '...', 'O': '---'}
12 
13 def morse(msg):
14     logging.info(f'Blinking {repr(msg)}')
15     for word in msg.split():
16         sleep(inter_word)
17         for char in word.upper():
18             sleep(inter_char)
19             for bit in codes[char]:
20                 sleep(intra_char)
21                 led.on()
22                 if bit == '.':
23                     sleep(dit)
24                 elif bit == '-':
25                     sleep(dah)
26                 else:
27                     assert False, 'invalid morse char!'
28                 led.off()
29 
30 try:
31     led = Pin('LED', Pin.OUT)
32     morse('SOS')
33     morse('PICO')
34 except Exception as exc:
35     logging.fatal(exc, blink=3)

We’re now getting into multiple files that need to be kept up to date between our computer and the Pico. We can’t exactly run git on the Pico, but rshell still has some tricks up its sleeve to make maintaining this a bit easier in the form of a (very basic) rsync implementation:

$ rshell --quiet rsync . /pyboard/
Checking /pyboard/main.py
Checking /pyboard/logging.py
/home/dave/projects/home/pico/morse/logging.py is newer than /pyboard/logging.py - copying

This makes development cycles of Pico setups with multiple files much faster. Unfortunately there’s currently an issue with this when the host’s clock isn’t set to UTC. Personally I just set my Pi’s clock to UTC to work around this, but I should take a proper look when I get a second!

Now, once we reset the board we get the initial “SOS” blink, then a steady triple-blink on the internal LED when things have failed, and we can query the log file like so:

$ rshell --quiet cat /pyboard/log.txt
2022-10-19T22:58:24 info     Blinking 'SOS'
2022-10-19T22:58:28 info     Blinking 'PICO'
2022-10-19T22:58:29 error    Traceback (most recent call last):
2022-10-19T22:58:29 error      File "main.py", line 33, in <module>
2022-10-19T22:58:29 error      File "main.py", line 19, in morse
2022-10-19T22:58:29 error    KeyError: P

We could change the final except clause to blink differently for differing exceptions, e.g. 4 blinks for a socket issue, 3 blinks for other IO errors, 2 blinks for all other exceptions.

That’s enough on rshell for now. The package includes a full man-page, rshell(1), which I would strongly recommend reading for all the other features it includes.

Mount, mount, my soul!

This brings us to the new stuff in Ubuntu Kinetic (22.10): the mpremote utility. Unlike rshell which is a third-party tool (though still one of my favourites for interfacing with MicroPython), mpremote is straight from the MicroPython developers themselves.

The first thing to deal with is how we specify the serial port for the MicroPython board in mpremote. The full syntax is connect <device>, for example:

$ mpremote connect /dev/ttyACM0 repl
Connected to MicroPython at /dev/ttyACM0
Use Ctrl-] to exit this shell

>>>

However, this is a bit of a mouthful (fingerful?) to type so mpremote ships with a whole load of built-in aliases for common serial devices:

$ mpremote --help
mpremote -- MicroPython remote control
See https://docs.micropython.org/en/latest/reference/mpremote.html

List of commands:
  connect     connect to given device
  disconnect  disconnect current device
  edit        edit files on the device
  eval        evaluate and print the string
  exec        execute the string
  fs          execute filesystem commands on the device
  help        print help and exit
  mip         install packages from micropython-lib or third-party sources
  mount       mount local directory on device
  repl        connect to given device
  resume      resume a previous mpremote session (will not auto soft-reset)
  run         run the given local script
  soft-reset  perform a soft-reset of the device
  umount      unmount the local directory
  version     print version and exit

List of shortcuts:
  --help
  --version
  a0          connect to serial port "/dev/ttyACM0"
  a1          connect to serial port "/dev/ttyACM1"
  a2          connect to serial port "/dev/ttyACM2"
  a3          connect to serial port "/dev/ttyACM3"
  bootloader  make the device enter its bootloader
  c0          connect to serial port "COM0"
  c1          connect to serial port "COM1"
  c2          connect to serial port "COM2"
  c3          connect to serial port "COM3"
  cat
  cp
  devs        list available serial ports
  df
  ls
  mkdir
  reset       reset the device after delay
  rm
  rmdir
  setrtc
  touch
  u0          connect to serial port "/dev/ttyUSB0"
  u1          connect to serial port "/dev/ttyUSB1"
  u2          connect to serial port "/dev/ttyUSB2"
  u3          connect to serial port "/dev/ttyUSB3"

Here we can see that a0 is an alias for connect /dev/ttyACM0 so we can use this instead:

$ mpremote a0 repl
Connected to MicroPython at /dev/ttyACM0
Use Ctrl-] to exit this shell

>>>

The mpremote utility includes many of the same facilities as rshell like ls, cp, rm, and cat but with a different idea of the location of the remote file-system. Unlike rshell which pretends the Pico is mounted under /pyboard/, mpremote prefixes MicroPython paths with a colon:

$ mpremote a0 ls :
ls :
     327 log.txt
    1387 logging.py
     807 main.py
$ mpremote a0 cp *.py :
cp logging.py :
cp main.py :
$ mpremote a0 cat :log.txt
2022-10-19T22:58:24 info     Blinking 'SOS'
2022-10-19T22:58:28 info     Blinking 'PICO'
2022-10-19T22:58:29 error    Traceback (most recent call last):
2022-10-19T22:58:29 error      File "main.py", line 33, in <module>
2022-10-19T22:58:29 error      File "main.py", line 19, in morse
2022-10-19T22:58:29 error    KeyError: P

However, there’s one particular facility that’s unique to mpremote: the mount sub-command. This allows you to mount a directory on your computer on the MicroPython board under the /remote directory:

$ mpremote a0 mount .
Local directory . is mounted at /remote
Connected to MicroPython at /dev/ttyACM0
Use Ctrl-] to exit this shell
>
MicroPython v1.19.1 on 2022-10-14; Raspberry Pi Pico W with RP2040
Type "help()" for more information.
>>> import os
>>> os.listdir('.')
['main.py', 'logging.py']
>>> os.listdir('/')
['remote', 'log.txt', 'logging.py', 'main.py']

How does this help us? You can edit your MicroPython code on your computer, and have your MicroPython board run it straight from there without constantly copying stuff back and forth! When you run mount . the MicroPython REPL that’s launched is already in the /remote path (as you can see in the snippet above where listing “.” shows only main.py and logging.py.

In this case we almost certainly don’t want the MicroPython interpreter running a main.py script when it starts; we just want it to drop to the REPL and let us run our script manually. So the first thing to do is remove :main.py which is the copy of main.py on the board. Then we’ll mount our directory on the board, and manually run our script by importing main. As expected, it’ll blink SOS, then fail but afterwards (once we kill the script with Ctrl+C, then exit the REPL with Ctrl+]) we should have a “log.txt” on the computer:

$ mpremote a0 rm main.py
rm :main.py
$ mpremote a0 ls :
ls :
     327 log.txt
    1394 logging.py
$ ls
logging.py  main.py
$ mpremote a0 mount .
Local directory . is mounted at /remote
Connected to MicroPython at /dev/ttyACM0
Use Ctrl-] to exit this shell
>
MicroPython v1.19.1 on 2022-10-14; Raspberry Pi Pico W with RP2040
Type "help()" for more information.
>>> import main
2022-10-21T10:10:45 info     Blinking 'SOS'
2022-10-21T10:10:50 info     Blinking 'PICO'
2022-10-21T10:10:51 error    Traceback (most recent call last):
2022-10-21T10:10:51 error      File "main.py", line 33, in <module>
2022-10-21T10:10:51 error      File "main.py", line 19, in morse
2022-10-21T10:10:51 error    KeyError: P
^C
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "main.py", line 35, in <module>
  File "logging.py", line 59, in fatal
KeyboardInterrupt:
>>> ^]
$ ls
logging.py  log.txt  main.py
$ cat log.txt
2022-10-21T10:10:45 info     Blinking 'SOS'
2022-10-21T10:10:50 info     Blinking 'PICO'
2022-10-21T10:10:51 error    Traceback (most recent call last):
2022-10-21T10:10:51 error      File "main.py", line 33, in <module>
2022-10-21T10:10:51 error      File "main.py", line 19, in morse
2022-10-21T10:10:51 error    KeyError: P

We could now edit our scripts on the computer (in another terminal) and then immediately retry them on the board without any copying!

Exit, pursued by Baudelaire

That’s quite enough waffling from me! I blame the editor (who he? —Ed).

There’s plenty more detail in the man-pages of both rshell and mpremote, and an absolute ton of MicroPython capable boards out there to explore, not to mention the myriad things you can attach to them.

Have fun!