Why USB boot on a locked device?

Secure field devices deliberately lock down access in production: USB serial download and fastboot are disabled, the UART console is closed, the bootloader accepts only signed images. That is correct - each of these paths would otherwise be an attack vector. But manufacturing and service need a recovery path: what if a device no longer boots, or a new image must be installed without opening the enclosure?

The answer is a USB recovery boot that does not weaken the security chain: a signed system is loaded from a FAT USB stick and passes through the same HAB signature verification as a normal boot. If verification fails, nothing boots. This article describes such a spike on an i.MX6-based energy field device running U-Boot 2024.01 - including the problems that only appeared on real hardware.

The spike guiding question: can the controller boot from USB at all, and can HAB authentication be integrated into the autoboot path without bypassing it? Both were demonstrated on hardware; some items remain open and are named honestly below.

The scope was deliberately narrow: extend the existing U-Boot environment, reuse the existing authentication, and document what happens on real hardware. No new bootloader, no new security mechanism, no changes to the fused trust root. That restraint is what makes the result trustworthy - a recovery path that rebuilds the security architecture from scratch would need its own verification effort.

The security boundary: HAB stays untouched

A recovery path must not be a backdoor. The rule: every boot path - internal, USB, TFTP - authenticates the fitImage through the same HAB mechanism before the kernel starts. The USB path therefore only extends the loading logic (where does the image come from?), not the verification logic (is it allowed to start?). Concretely, the existing authentication routine of the normal boot is reused, not reimplemented.

This includes error handling: three cases must be explicitly distinguished - no USB device found (skip, continue normal boot), image not loadable (abort with a message), HAB authentication failed (abort, no boot). A silent fallback to an unsigned boot would be a security bug.

The comparison with the locked paths is instructive: fastboot and USB serial download (SDP) allow loading and executing a bootloader in RAM - on an open device that is a legitimate manufacturing path, on a closed device an attack path, because a signed bootloader can be extracted from an update image. USB recovery boot is different: it loads a signed system from the stick and verifies it via HAB before booting. The security boundary therefore runs not between "USB yes/no" but between "verified or unverified".

Preparing the boot media

The stick needs a FAT partition with four artifacts from the Yocto build: the dm-verity-protected rootfs (SquashFS with appended signature), the device tree, the signed fitImage and the dm-verity initramfs. Important: the initramfs .verity.env file must also go on the stick - it contains metadata the initramfs expects at boot (more on that below).

Filenames should be generic and stable so the U-Boot environment finds them without guessing. If you keep the build names, parameterize the environment accordingly - both work, but the mapping must be documented.

# FAT partition on the stick (example: /dev/sdb1)
mount /dev/sdb1 usb
cp rootfs.squashfs.verity usb/
cp board.dtb usb/
cp fitImage usb/
cp dm-verity-initramfs.squashfs.verity usb/
cp dm-verity-initramfs.squashfs.verity.env usb/
umount usb

Manual boot first

Before extending autoboot, the manual path must work. The sequence: start the USB subsystem, load device tree and fitImage from the FAT partition into RAM, stop the USB subsystem, then bootm. Stopping before boot matters - the kernel should re-enumerate the stick itself rather than inherit a half-initialized bus.

Starting the fitImage via bootm selects the matching configuration based on the device tree. The simple form suffices for the manual test; the autoboot path later uses the explicit FIT configuration conf-<dtb> so kernel and device tree are guaranteed to match.

The load addresses are platform-specific and must fit the memory layout: on the test device, fitImage and device tree sit in a RAM region that does not collide with the kernel target or the FDT target. Copying addresses from an old boot log without checking the current memory layout risks images overwriting each other while loading - a fault that only surfaces as strange behavior after the kernel starts. The board default environment supplies the right values; custom adjustments should rarely be needed.

usb start
fatload usb 0:1 $fdt_addr board.dtb
fatload usb 0:1 $loadaddr fitImage
usb stop
bootm $loadaddr - $fdt_addr

FIT image and HAB: two checks, one goal

A FIT image (Flattened Image Tree) bundles kernel, device tree and optionally an initramfs into a single file, described by a device-tree-like structure. Each subimage section carries a hash (e.g. sha256); bootm verifies these hashes while loading ("Verifying Hash Integrity"). This protects against transfer errors and random corruption - but not against an attacker who replaces the whole image including its hashes. That is what the HAB signature is for: it authenticates the FIT through the RSA chain of trust before the kernel starts at all.

The two checks complement each other: the FIT hash ensures subimage integrity, the HAB signature ensures authenticity of the whole package. Implementing only one of them leaves a gap. The FIT configuration also matters: a FIT can contain several kernel/DTB combinations; the configuration conf-<dtb> selects the one matching the device tree. The USB boot path must use the same configuration as the internal boot - otherwise a kernel may start with the wrong device tree, producing failures on real hardware that are hard to diagnose.

Extending autoboot

The core is two environment variables: one for the USB boot kernel command line and one for the boot sequence itself. The sequence first checks whether a stick is present at all (fatls as a presence probe), then loads device tree and fitImage, sets the bootargs and calls the existing HAB authentication. Only on successful verification does bootm run with the matching FIT configuration.

Each failure branch prints its own message and does not boot. The variable is hooked into the existing autoboot environment so the USB path is tried before the internal boot - a plugged, signed stick takes precedence, without a stick the device continues normally. The variable names are freely choosable; what matters is reusing the authentication routine.

env set usb_bootargs 'env set bootargs console=ttymxc3,115200 rootfstype=ramfs rootwait'

env set bootcmd_usb 'usb start; if fatls usb 0:1; then \
  echo "USB-Boot: device found, attempting boot"; \
  fatload usb 0:1 $fdt_addr $fdt_file; \
  fatload usb 0:1 $loadaddr $image; \
  usb stop; \
  run usb_bootargs; \
  if run authenticate_image; then \
    if bootm ${loadaddr} - ${loadaddr}#conf-${fdt_file}; then \
      echo "USB-Boot: booting now..."; \
    else echo "USB-Boot: failed to load image!"; fi; \
  else echo "USB-Boot: HAB authentication failed"; fi; \
else echo "USB-Boot: no USB device found, skipping..."; usb stop; fi'

run bootcmd_usb

Reading the boot log

The boot log proves the chain step by step: bootm loads the FIT, verifies the sha256 hashes of the subimages (kernel, device tree), relocates the FDT and starts the kernel. On the test device: kernel 6.1, 256 MB RAM, SPI NAND as the rootfs medium, CAAM Era 8. The decisive line is "Verifying Hash Integrity ... sha256+ OK" - it shows the FIT was loaded tamper-free.

Then the dm-verity initramfs takes over: it locates the signature behind the rootfs, verifies it and sets up dm-verity before the actual rootfs is mounted. In the log you see the kernel USB enumeration, the stick attaching as a SCSI removable disk, and the initramfs starting. Knowing this order lets you assign failures unambiguously to a stage.

## Loading kernel from FIT Image at 82000000 ...
   Using 'conf-<board>.dtb' configuration
   Trying 'kernel-1' kernel subimage
     Description:  Linux kernel
     Type:         Kernel Image
     Data Size:    <size> Bytes
     Hash algo:    sha256
   Verifying Hash Integrity ... sha256+ OK
## Flattened Device Tree blob at 83000000
   Loading Kernel Image
   Loading Device Tree ... OK
Starting kernel ...

What the initramfs does next

After the kernel starts, the initramfs compiled into the kernel runs as the first userspace process. Its job in the dm-verity setup is a fixed sequence: first it checks a mark (MARK) at the expected offset behind the rootfs, then reads signature size and env size (SIGN_SIZE, ENV_SIZE), verifies the ECDSA signature of the root hash with the public key integrated at build time, and finally sets up dm-verity. Only then is the verified rootfs mounted and handed over to the actual userspace via switch_root.

Each of these stages can fail - and the error message tells you where: a missing or wrong mark usually means the rootfs package was copied incompletely (the .verity.env file is missing). A failed signature check means rootfs and signature do not match - for instance after a later modification of the rootfs. A dm-verity error while mounting means a block does not match the hash tree. Knowing the sequence lets you read the boot log as a diagnostic protocol rather than a wall of errors.

For USB boot this sequence is identical to the internal boot - the initramfs does not distinguish where the rootfs came from. That is exactly the desired property: the recovery path inherits verification instead of bypassing it. The only difference lies in the origin of the data (stick instead of NAND) and in whether the medium is fast and complete enough.

Practical pitfalls

First finding: slow USB sticks. The EHCI controller hits timeouts when the stick reads below roughly 20 MiB/s on average. The symptom: repeated "EHCI timed out on TD" messages, then "Error reading cluster" and "Failed to load fitImage". The fix is trivial but important: use a fast stick and measure read speed beforehand. A service kit should contain only a tested model.

Second finding: the initramfs expects a .verity.env file on the boot medium. Without it, dm-verity setup aborts with a mark check ("Wrong DM-VERITY-ENV-MARK"). The file therefore must go on the stick - it is part of the signed rootfs package and easily overlooked when copying.

Third finding: no shell access in the production initramfs. The dm-verity initramfs is minimal and spawns no shell - debugging needs a dedicated USB-boot initramfs with shell capability. As a workaround, a getty on the serial console was documented (inittab lines for sulogin or start_getty). Important: such a debug path may only exist in development images, never in the production initramfs of a closed device.

Stick speed is easy to check beforehand: on Linux, read a large file from the stick (e.g. dd if=/dev/sdb of=/dev/null bs=1M) and measure throughput. Well below 20 MiB/s, the stick is unsuitable for recovery boot - not because of a specification, but because the U-Boot EHCI driver times out on slow devices. This measurement belongs in the service documentation so not every technician walks through the same failure chain.

# EHCI timeout signature of a too-slow stick:
EHCI timed out on TD - token=0x1f8c80
Error reading cluster
Failed to load 'fitImage'

# Getty workaround (development images only), /etc/inittab:
z6:6:respawn:/sbin/sulogin
mxc3:12345:respawn:/bin/start_getty 115200 ttymxc3 vt102

From spike to service path

A demonstrated spike is not yet a service process. For USB boot to work reliably in the field, several things must be fixed: a tested stick model with proven read speed (the 20 MiB/s threshold is not a theoretical value but derived from EHCI timeouts), a documented copy procedure for all five artifacts including .verity.env, and a version mapping between stick content and device software. A service kit with unverified sticks is a source of errors, not a safeguard.

The U-Boot environment itself also needs attention: it is typically persisted via env save and is only CRC-protected, not signed. Whoever adds the USB boot path to the environment should restrict the writable variables and ensure that an attacker with console access cannot simply remove the authentication step. On closed devices the console is locked anyway - so the USB path must work without console interaction, purely by plugging in the stick.

Finally, documentation: the service technician needs a short instruction (plug in stick, wait, check log), not a HAB treatise. And the development path needs the opposite - the full failure analysis with the three failure branches, the EHCI symptoms and the initramfs sequence. Maintaining both from the same source keeps the process consistent when images or the environment change.

Open items and lessons

Honest status at the end of the spike: HAB-authenticated USB boot was demonstrated on hardware (FIT hash verified, kernel started, dm-verity initramfs took over). Still open: a dedicated USB initramfs with shell, testing on a device with burnt SRK fuses (closed state), a working network stack with SSH in the recovery system, and tools to edit the U-Boot environment from Linux. The closed-device test is the security-relevant one - only there does the path prove itself with enforced HAB verification.

The lesson: a recovery path is not a separate security project but an extension of the existing chain. Reuse the authentication routine, make failure branches explicit, and prepare the medium completely (including .verity.env), and you get a service access that does not move the security boundary. And: the practical problems - stick speed, missing files, missing shell - only surface on real hardware, not in code review.