Posts

Posted on

Upgrading ESP32 bootloader wirelessly using esphome

I have a few ESP32 devices around the house that are hard to access for USB upgrades. When I got them, I did the setup using a pogo plug jig I cobbled together, expecting to do all further upgrades wirelessly. I knew I’d have to take them apart and find the jig to do any future USB update but didn’t expect to have to do that.

All worked to plan until I started getting these warnings from esphome:

... Chip rev >= 3.0 detected. Set minimum_chip_revision: "3.0" under esp32 > framework > advanced to reduce binary size
... Bootloader too old for OTA rollback and SRAM1 as IRAM (+40KB). Flash via USB once to update the bootloader

Ok. The first one is easy enough but that second one says I need to flash via USB. What’s that about?

It seems that there’s no safe way to update the bootloader on an ESP32. Anything that interrupts the flash is likely leave it in a soft brick state and I’d have to flash it via USB. Hmm… so basically if the flash succeeds I’m done and if it fails I’m no worse off than I was in the first place. Sounds like it’s worth a try.

So how do I flash the bootloader wirelessly? There’s no option in the ESPHome Device Builder UI. The esphome documentation does show how to do it from the command line, https://esphome.io/components/ota/esphome/#updating-the-bootloader-on-esp32.1 Click through to read the documentation but I’ll paste the most important part here

Take that warning seriously. I was able to upgrade all four of my devices without incident but I can’t promise you’ll be as lucky. Don’t even think about doing this upgrade during a thunderstorm or if your power is flaky today. Maybe avoid running big downloads or torrents while the flash is in progress. The good news is that the bootloader flash is really fast so the window for things to fail is short.

So we’re looking at a 3 flash process:

  1. Flash once to enable allow_partition_access and minimum_chip_revision.
  2. Flash the bootloader via the command line.
  3. Flash again to disable allow_partition_access and enable sram1_as_iram.

In theory, you can do steps 1 and 3 using the UI but since you need to log into the esphome container anyway it’s just easier to do it all from the command line.

So off you go:

  1. Add minimum_chip_revision and allow_partition_access your yaml file(s) in the following places:
    esp32:
    ...
    framework:
    ...
    advanced:
    minimum_chip_revision: "3.0"
    ota:
    - platform: esphome
    ...
    allow_partition_access: true
    Do I need to remind you to use only spaces for indentation, no tabs, and that indentation actually means something in YAML files? Make sure you line things up as shown.
  2. Flash the device(s)
    a. Log into your homeassistant instance:
    ssh homeassistant
    b. From there, log into your esphome docker container:
    docker exec -it addon_5c53de3b_esphome /bin/bash
    c. Flash this configuration:
    cd /config/esphome
    esphome run your-device-file.yaml
    Check the log file that displays when the flash completes. The message about setting minimum_chip_revision should be gone but the message about the bootloader will still be there.
    d. Flash the bootloader (still from the /config/esphome directory). Remember. This is the dangerous step!!!
    esphome upload --bootloader your-device-file.yaml
    This should finish pretty quickly.
    e. Go check your logs again:
    esphome logs your-device-file.yaml
    The only warning you should be seeing now is:
    Bootloader supports SRAM1 as IRAM (+40KB). Set sram1_as_iram: true under esp32 > framework > advanced
    Note that it must say Bootloader supports. That means you’ve successfully updated the bootloader and are ready to enable sram1_as_iram. If you don’t see that message and especially if you’re still seeing the Bootloader too old message STOP RIGHT HERE and retrace your steps. If you enable sram1_as_iram with an old bootloader you may brick your device.
    f. Remove the “allow_partition_access: true” that you previously added to the ota section of your yaml file(s) and add the following right after “minimum_chip_revision: "3.0"“:
    minimum_chip_revision: "3.0"
    sram1_as_iram: true
    g. Flash this (final) configuration:
    esphome run your-device-file.yaml
    Check the log file. Both warnings should now be gone.

Enjoy your fully updated ESP32 device, your up to date bootloader, the extra memory, and the safe OTA rollbacks that come with it.

  1. Interestingly, the --bootloader option is not documented along with the esphome upload command: https://esphome.io/guides/cli/#upload-command. I think the message is pretty clear that you’re on your own recovering in case of failure. ↩︎

Posted on

Preserve multi-tab history in ptyaxis

I got tired of losing the shell history for most of my tabs in ptyxis1 when it exits so I threw this together: https://gist.github.com/dbrand666/0b54c5edc659b69bf8dd7134277d9445. It should be pretty much self-documenting. There’s nothing to configure. Just drop it in ~/.bashrc.d/ptyxis-history and it’ll handle any new shell tab.

What it does

The idea starts with creating a separate history file for each tab. They’re stored in ~/.history, which is created if it doesn’t already exist. The history files are numeric – the process ID of the shell.

When a new level 0 bash is launched, it looks for an orphaned history file in there and adopts it. If there are none left, it creates a new one.

I’ve only been using it a couple of days but it seems to work.

  1. the default Gnome Terminal these day, at least in Fedora ↩︎
Posted on

Passing through a raw drive in Proxmox

The migration from ESXi to proxmox continues…

I have a large drive I’ve been using for backups. On ESXi, I had it installed in an external USB dock which was passed through to a VM. On my new server, I have hot swap SATA bays so I decided to eliminate the USB overhead and install it directly. I found the instructions to pass through a SATA drive, basically:

qm set <vmid> -scsi<n> /dev/disk/by-id/<drive string>

I typed that command, a drive was added to my VM. I saw it in drive manager. BUT no drive letter. I rebooted. Still no drive letter.

I looked close in drive manager. Instead of the partitions I knew were on the disk, I saw only a protected gpt partition.

Searching around I found the most common cause was some quirk of USB drives. Apparently a lot of them like to fake a 4k sector size for historical reasons. No problem says I. I make a backup of my backup drive, repartition the drive in Windows and it looks good.

But then I switch to Linux and take a look at the formatting. It’s trash. testdisk can make sense of it but it’s clear that something is still off.

I discovered that Proxmox (QEMU in general) doesn’t really pass drives through like ESXi does. It basically adds it’s own virtualization layer over the drive and that’s what the guest seems. It likes to use 512 byte sectors. It’s not too hard to request 4k sectors (like my drive uses natively) to QEMU but Proxmox doesn’t have such an option.

I found this page which described how somebody got creative in ramming the required options through to Proxmox. I just want to write down the process I used to find what needs adding since the commands I needed were very different than the ones he used.

First, I took a snapshot of the qm commands I had before adding the drive:

qm showcmd <vmid> >without-drive.txt

Then I added my drive as I mentioned previously.

qm set <vmid> -scsi<n> /dev/disk/by-id/<drive string>
qm showcmd <vmid> >with-drive.txt

I then used ccdiff to find the additions and pasted them into a text file, scsi<n>.txt. Be careful selecting the commands – ccdiff can be a bit naive in picking the character differences. These are the commands Proxmox adds to QEMU to get the drive added.

I then unlinked the drive from my VM:

qm unlink <vmid> --idlist scsi<n>

To add the drive back, I used:

qm set <vmid> -args "$(cat scsi<n>.txt)"

I then booted up the VM to check whether I was on track. As expected, the drive was passed through but still had the wrong sector size. Time to add the the options to set the sector size: logical_block_size=4096,physical_block_size=4096. When I looked at the commands in scsi<n>.txt, I found 2 devices. One was a virtio and the other a ide-ht. These options go on that second device.

After editing the scsi<n>.txt file and running the set args command again, I booted up the VM. The drive had the correct sector size.

BTW, you can clear out the args by calling the same command with an empty argument if you’re done with this hack:

qm set <vmid> -args ""

One annoying thing is that the drive doesn’t show up on the Hardware tab in Proxmox. It doesn’t know what these commands do. I added a note on the Summary tab pointing me back to this blog post so I remember what I did. Hopefully Proxmox will support this at some point in the future.

P.S. The mount option -o big_writes really speeds up writing to NTFS drives from Linux.

P.P.S. I added one more option as a safety measure to ensure that even if I add the same drive to more than one VM I don’t risk filesystem corruption by having the same raw drive mounted on more than one running VM. Right before the "filename":"/dev/disk/by-id/, add "locking":"on",.

Posted on

KeePassXC as Secret Service Integration

KeePassXC has an option to serve as the Secret Service Integration on Linux. Unfortunately when you try to enable it you’ll find that Gnome has already provided its own. You’d think it would be simple to disable that service and provide your own but after trying the obvious and then reading through years of discussion I couldn’t find a simple answer.

I gave up. Instead I removed execute permission from /usr/bin/gnome-keyring-daemon:

sudo chmod -x /usr/bin/gnome-keyring-daemon

I suspect it’ll get reverted next time an update comes through. If it does, I’ll probably just add the chmod to /etc/rc.local.

Update May 5, 2026: Sure enough, the file permissions were restored with the Fedora 44 upgrade. /etc/rc.local has been gone for a while. Instead, I added this to root’s crontab:

@reboot         chmod -x /usr/bin/gnome-keyring-daemon

It seems to be working so far. The only issue I hit is that KeePassXC kept popping up asking be for a password without a file or key file prefilled. I fixed that by going into Tools>Settings>Secret Service Integration>General and unchecking Prompt to unlock database before searching.

Edit: Next issue. After every login, something tells KeePassXC to create a new password file called Passwords. I let it create one the first time but it’s still not satisfied. Even if I open the old one, it still insists. I can hold down Esc until it gives up but I’m sure something isn’t right.

Posted on

Back to Fedora

A few years ago I bought a new laptop that came with Windows 11. I hadn’t used Windows in a long time, thought WSL was really cool and decided to give it an extended trial.

WSL worked surprisingly well most of the time, in some ways I liked it even better than MacOS – it was real Linux, no more fighting with Brew. Still, it had it’s own issues, most notably a layer of abstraction between WSL and the network and devices. VSCode made a valiant effort to make it all transparent but somehow Windows kept poking through.

More recently, with all the AI integration into the OS and the persistent WSL and Docker Desktop crashes, I realized I didn’t really want the Windows anyway.

So, I installed Fedora (dual boot for now).

Some pleasant surprises

(Encrypted) Windows drive

I was able to mount my encrypted Windows drive directly from Nautilus! Just click on it and supply the recovery key when prompted. The Gnome keyring will even remember the password, if desired.

WSL drive

Accessing my WSL drive was a bit more of a challenge. First off, most pages I found were about importing Linux drives into WSL. The few pages I found about mounting the WSL virtual drive referenced libguestfs-tools. I tried to follow those instructions with no success. What finally worked was this sequence:

sudo modprobe nbd max_part=8
sudo qemu-nbd --connect=/dev/nbd0 '/run/media/<linux user name>/Windows/Users/<windows user name>/AppData/Local/Packages/CanonicalGroupLimited.UbuntuonWindows_<random instance id>/LocalState/ext4.vhdx'
sudo mkdir /media/wsl2
sudo mount -r /dev/nbd0 /media/wsl2

It took some rummaging around on the Windows drive to find the right vhdx file – find is your friend.

Todo: Mount this automatically. See https://unix.stackexchange.com/questions/791753/how-to-automatically-mount-external-bitlocker-encrypted-drives-at-boot-on-linux for the Windows part. Maybe this for the nbd part: https://mattgadient.com/how-to-using-systemd-to-mount-nbd-devices-on-boot-ubuntu/.

Less pleasant surprises

Nextcloud

I use Nextcloud for file sharing. On Windows, Nexcloud supports virtual files. These files are retrieved on demand. That means I don’t have to download and store my whole Nextcloud share locally. There is supposedly “experimental” support for virtual files on Linux but even when it works, it isn’t at all transparent.

What I ended up doing for now is to set up two different Nexcloud shares. One with the Nextcloud client that selectively syncs only the files I need to have local copies of at all time and another using Gnome’s “Online Accounts” that acts kind of like an NFS or SMB share – online only..

KeepassXC

The first thing I noticed when I installed KeepassXC is that auto-type didn’t work – the Auto-Type tab was missing from the Settings entirely. My first instinct was to blame Flatpak so I uninstalled that and reinstalled as an RPM.

That didn’t change anything. Turns out this one was Wayland’s fault.

I needed to add QT_QPA_PLATFORM=xcb to my environment. That did fix the issue but I couldn’t find a pretty way to add it just for keepassxc that would work when I launched from Gnome. I ended up writing this wrapper script and putting it in ~/bin:

#!/bin/bash
real="$(PATH="${PATH##*"$HOME"/bin:}" which "${0##*/}")"
QT_QPA_PLATFORM=xcb exec "$real" "$@"

What this does is find the “real” keepassxc by stripping everything up to and including the ~/bin directory from the path and then searching for keepassxc again. Then it invokes that keepassxc with any arguments that might have been passed in but adding QT_QPA_PLATFORM=xcb. Note that keepassxc isn’t mentioned anywhere in this script – it can be used to wrap any program that might need this variable set – just make links to the same file.

Codecs

Fedora doesn’t come ready for media. I had to enable rpmfusion repos and install a few packages:

sudo dnf install https://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-$(rpm -E %fedora).noarch.rpm
sudo dnf install   https://download1.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm
sudo dnf upgrade
sudo dnf swap ffmpeg-free ffmpeg --allowerasing
sudo dnf group install multimedia

I also installed this but it might have been included in the multimedia group.

sudo dnf install libavcodec-freeworld

Waydroid (Android emulation)

There’s a useful guide here: https://pimylifeup.com/ubuntu-waydroid/

Posted on

Squeezelite on an access point? Why not?

I’ve replaced all my old access points with the seriously overspec’ed Linksys MX4300. They’re all running OpenWRT so I’ve been thinking what else I can give them to do. Why not let them replace my Raspberry Pi based media players?

So I moved the USB audio adapters from the RPIs to the USB port on one of the access points. Log into the router and run:

opkg update
opkg install squeezelite-full kmod-usb-audio
/etc/init.d/squeezelite start
/etc/init.d/squeezelite enable

That’s pretty much it. Music Assistant picked it right up.

One last thing, since I have multiple players I went into /etc/config/squeezelite and set option name.

Fewer wires, less equipment, less power (probably).

Posted on

Changhong Service Menu

I have an old dumb Changhong TV and wanted to set it up to turn on and off automatically when my Roku is turned on. I always have a few smart sockets lying around but the TV defaults to powered off when power is applied.

I found a note somewhere that the service menu can be accessed by pressing the source button on the remote and entering the code 3138. That worked so I went digging through the menus.

Under System Setting>Power ON Mode there were options for On, Off, and Memory. I went with Memory for now but might change it to On later.

While I was in the System Setting page I also noticed an option called Logo Select. I tried setting to off hoping it would get rid of the Creating Easy Life logo on power up.

Both options worked as expected. If I plug it in now, it turns on (assuming it was on when I unplugged it) and it boots up without a splash screen.

BTW, there’s a bunch of interesting stuff in the service menu – even the ability to flip the screen. They seem to have anticipated folks using these as kiosk displays.

Posted on

Setting up ZFS RAIDZ(N) with missing drives

I’m planning to migrate from an old RAIDZ1 pool of three drives to a new RAIDZ2 pool with five. I was thinking that I’d start by setting up four drives in a degraded configuration and add the fifth when I get around to buying it. I figured I could create a loopback drive (or the FreeBSD equivalent, set up the pool, and remove that drive before adding any actual data.

The problem is that I’m using TrueNAS and it, probably wisely, doesn’t let me select that loopback drive when it’s time to set up the pool. I did quite a bit of searching and everyone says that there’s definitely no way to do it via the UI.

Since I’m not afraid of the command line, I tried looking around to find out exactly what parameters TrueNAS uses when it creates the pool. Even though I’m cutting corners, I still wanted to stay as close as possible to a standard configuration. I found these old instructions but I wasn’t convinced the zpool create command would give me exactly the same options that TrueNAS would use, especially since I wanted things like compression and encryption like my old drive had.

So I decided to take another approach.

I knew that it’s possible to expand a RAIDZ array by replacing drives with larger drives. The available space stays unchanged as drives are upgraded but once the last drive is upgraded the array takes on the size of the new drives.

I happened to have a really old drive laying around unused. Really old. 640GB! But it didn’t really much matter. I wasn’t going to be trusting it with any data. I installed that drive, went in to create my RAIDZ2 pool, and selected it along with my 4 “real” drives. All good. I now had a RAIDZ2 pool that was ready to hold about 1.5TB of data (3x640GB).

Next step was to replace that drive with a large loopback drive:

# Create a sparse file to hold the loopback device
truncate -s <disk size, eg 8t> disk.img
# Create the loopback device
mdconfig disk.img
(it output a device name, eg. md0)
zpool replace <pool name> <little drive> <loopback drive>

The pool now showed the full capacity expected of the larger drives. Then I removed that loopback drive from the pool, deleted the loopback device, and the backing file:

zpool detach <pool name> <loopback drive, eg. md0>
mdconfig -du <loopback drive, eg. md0>
rm disk.img

The capacity was unchanged (still the right size) but the pool state showed DEGRADED, as expected.

When the new drive arrives, I’ll add it. Probably best to do it from the TrueNAS GUI. From “Dashboard” scroll down to the pool and select the pool status gear icon. Click on the offline drive and choose Replace.

Posted on

Controlling an Onkyo receiver with a Pi

With my media player/voice assistant running on a Pi Zero, I noticed that my stereo cabinet was a bit warm. The Pi doesn’t use much power but my receiver does. I used to control power to the receiver with an X10 appliance module but I’m phasing that out.

I found a few pages describing how to control Onkyo equipment using their Onkyo RI “Remote Interactive” protocol. I built the interface cable and use the Python code from here and codes from here. I also had to install pigpiod. Use sudo systemctl enable pigpiod and sudo systemctl start pigpiod to start the service (for now) and enable it (after reboots.

Test it from the command line to ensure that the receiver turns on/off as expected.

Next step was to integrate with squeezelite.

I found this man page which documents a -C option which will close the output device after a specified timeout if idle. I set it to half an hour. This works together with the -S option which runs a script when the output device turns on/off. Unfortunately, at this time at least, the version of squeezelite in the repo is too old. My quick fix was to replace the binary /usr/bin/squeezelite with the latest from here which was squeezelite-2.0.0.1486-armhf.tar.gz.

Here’s the script I’m passing to squeezelite’s -S option (my receiver is a TX-8020):

#!/bin/bash
cd .../onkyo-rpi || exit 1
if [ $1 = 0 ]; then
        python main.py --gpio 26 0x420
else
        python main.py --gpio 26 0x02f
fi

I caller the script onkyo-power.sh. Test that it turns the receiver on when called with a 1 argument and off when called with a 0.

A few other notes…

I first assumed that a native C code implementation would be more reliable. I used onkyoricli which I found here. I had no luck with it and after hooking up an oscilloscope to check the output, I saw that it was distorted. The code is correct but the waveforms seem to be getting interrupted by the Linux scheduler and end up with gaps in them. The python package doesn’t try to generate the waveforms itself but instead depends on the pigpiod daemon which seems to handle the realtime issues correctly.

Another issue I ran into is that as soon as I hooked up the RI cable, my audio output was garbled. I tried using a better power supply for the Pi but it didn’t help. I noticed that the noise started as soon as I attached the ground pin on the Pi to the RI cable – attaching only the signal pin was fine. When I checked the signal between the Pi and the RI cable ground I could see a 60Hz signal between them. I believe the audio connection to the receiver is grounded and that I was looking at a ground loop between that connection and the RI port. I ended up leaving off the redundant ground connection between the RI port and the Pi, leaving only the signal pin connected.

Posted on

Home assistant media player with voice input

I’ve been running piCorePlayer for a long time, first with LMS and more recently with Music Assistant. It’s really simple to set up and has many interesting features (like an optional built-in LMS) that I wasn’t using anyway.

I’ve been watching Home Assistant’s Year of Voice project with interest and was about to order an Atom Echo from M5Stack when I realized that the music players I already have scattered around the house should be able to process voice too without much trouble.

First step was to replace piCorePlayer with a Raspberry Pi OS so I’d be working with a more conventional Linux environment. I started with Raspberry Pi Imager to create an SD card. I installed the Lite version since I wasn’t planning on using the UI and configured SSH. Once it booted up I saw that it had already expanded storage so I was ready to start installing the player.

A couple of things I had to get out of the way first. I’m still using that cheap Ethernet adapter so I had to follow this guide first and reboot. Next up, I noticed that the squeezelite package wants to install all kinds of UI things. I thought it would be a good idea to disable all of that before I got started. Create a file named /etc/apt/apt.conf.d/99_norecommend and enter the following settings:

APT::Install-Recommends "false";
APT::AutoRemove::RecommendsImportant "false";
APT::AutoRemove::SuggestsImportant "false";

That will keep apt from installing recommended or suggested packages (like an X11 server).

Next, we can install squeezelite. You can type this all on one line or just cut/paste the whole block:

sudo apt-get update &&
sudo apt-get upgrade &&
sudo apt-get install squeezelite

You may want to go into /etc/default/squeezelite and see if you want to change anything. I changed the player name and increased the ALSA buffer size as recommended here.

Reboot and we should see the player.

The next step is to install Wyoming Satellite. Follow the instructions on that page. I’m using an old Raspberry Pi Zero (not even W). If you’re starting from scratch, the Zero 2W looks like a better choice since it can do local wake word detection.

When I tried adding the Audio Enhancements I hit the following error:

Traceback (most recent call last):
File ".../wyoming-satellite-master/script/run", line 12, in
subprocess.check_call([context.env_exe, "-m", "wyoming_satellite"] + sys.argv[1:])
File "/usr/lib/python3.11/subprocess.py", line 413, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['.../wyoming-satellite-master/.venv/bin/python3', '-m', 'wyoming_satellite', '--name', 'Wyoming Satellite', '--uri', 'tcp://0.0.0.0:10700', '--mic-command', 'arecord -r 16000 -c 1 -f S16_LE -t raw -D default:CARD=Device_1', '--snd-command', 'aplay -r 22050 -c 1 -f S16_LE -t raw', '--wake-uri', 'tcp://...:10400', '--wake-word-name', 'ok_nabu', '--mic-auto-gain', '10', '--mic-noise-suppression', '2']' died with ...

Seems that the binaries in the webrtc-noise-gain package aren’t compatible (at least with the Pi Zero). The solution was to build them which takes a rather long time…

sudo apt-get install python3-dev
. .venv/bin/activate
pip3 install –no-binary ‘:all:’ webrtc-noise-gain==1.2.3

It took a few hours but it works!

Play around with the audio enhancements settings to see what works for you.

I suggest adding an automation to pause the player as soon as the wake word is detected.