Reading Boot Logs

Boot Logs Are a Goldmine

Every embedded Linux device narrates its own configuration as it starts up. The kernel, bootloader, init system, and every service daemon announce what they are doing, what files they are reading, what credentials they are using, and what errors they encounter. Most developers leave debug verbosity enabled in production because the UART console is not intended to be accessible — they never thought you would be sitting here reading it.

A full boot log from an IoT device can contain: - Credentials used by services at startup - Kernel version (for CVE research) - Full filesystem layout - Network configuration - The complete software stack and version numbers - Hidden services not exposed through normal interfaces

You do not need to break in. The device tells you everything. You just have to listen.

Connect Before You Power On

This is the most common beginner mistake: plugging in UART after the device has already booted. You will connect to a running shell, which is useful, but you will miss everything the device said during the first 5–30 seconds — which is where the most sensitive information lives.

Rule: connect your UART adapter, open your terminal, have your logging ready, then apply power.

The boot sequence is not repeatable without another power cycle. U-Boot and the kernel do not re-print their initialization messages once the system is running. If you miss them, you power cycle and try again.

# Have picocom running and logging BEFORE you power the device
picocom -b 115200 --logfile boot_$(date +%Y%m%d_%H%M%S).log /dev/ttyUSB0

# Or with espilon-monitor — structured logging with timestamps
./espilon-monitor --logdir ./target_logs /dev/ttyUSB0

What to Look For

Credentials and Secrets

This is the category that gets devices compromised. Init scripts, service startup sequences, and connection managers regularly print credentials to the console. Examples of what appears in real boot logs:

• Default root passwords printed in clear text during first-boot setup scripts
• WiFi SSIDs and passphrases in wpa_supplicant or udhcpc startup lines
• API keys and authentication tokens in service startup arguments
• SSH host key fingerprints (useful for device identification and MITM detection)
• Database connection strings with embedded passwords
• MQTT broker credentials in IoT agent startup

Any line containing password, passwd, key, token, secret, or credential is worth reading twice.

System Information

• Kernel version: Cross-reference against CVE databases. An unpatched 4.9 kernel on a device with known local privilege escalation CVEs is an immediate finding.
• Filesystem: ext4, squashfs, jffs2, ubifs — tells you about persistence, writable areas, and flash layout.
• Mount points: Where the OS is mounted, where /tmp is (usually RAM), where config is stored.
• Running services: Every service that starts is an attack surface. Note versions.
• Hardware model and board revision: Useful for finding matching firmware dumps, datasheets, and known vulnerability research.
• MAC addresses: Useful for device fingerprinting and network reconnaissance correlation.

File Paths

The device prints paths to config files, scripts, and data directories throughout boot. Many of these are writable, world-readable, or contain hardcoded values. Note every path that appears during service startup. Particular interest: backup directories, update scripts, factory reset handlers.

Error Messages

Failed service starts are intelligence. A service that fails to start because it cannot connect to a backend reveals the backend URL and protocol. A missing file error reveals an expected file location you can potentially create. A permission error reveals the privilege model — if a service is running as root and failing on a world-readable config, that tells you something about the security posture.

Realistic Boot Log Walkthrough

Below is a representative boot log excerpt. Read it the way an attacker reads it — annotating what matters.

U-Boot 2019.10 (Mar 14 2023 - 09:22:41 +0000)

CPU:   MT7621AT ver:1.19 eco:3
DRAM:  256 MiB
NAND:  128 MiB
Loading Environment from NAND... OK

Hit any key to stop autoboot:  0          ← [bootloader interrupt window — 3 seconds]

Starting kernel ...

[    0.000000] Linux version 4.14.221 (build@buildserver) (gcc version 8.4.0)
               ← [kernel 4.14.x — check for Dirty COW, eBPF escapes, namespace bugs]

[    1.247831] mtd: device 3 (rootfs) set to be root filesystem
[    2.103894] VFS: Mounted root (squashfs filesystem) readonly
               ← [root FS is squashfs — read-only, but /tmp /var are likely RAM]

[    3.441207] Freeing unused kernel memory: 1024K

Starting syslogd: OK
Starting klogd: OK
Running /etc/init.d/rcS

[  rcS  ] Mounting /proc and /sys... done
[  rcS  ] Mounting tmpfs on /tmp... done
[  rcS  ] Hostname: ESP-GW-A3F2
[  rcS  ] MAC address: 00:1A:2B:3C:4D:5E     ← [device fingerprint]

[  rcS  ] Configuring network...
[  udhcpc] Sending discover on eth0
[  udhcpc] Lease obtained: 192.168.1.104 gw 192.168.1.1

[  rcS  ] Starting dropbear SSH server...
[  dropbear] Generating RSA host key.
[  dropbear] Host key fingerprint: SHA256:xK3mP9qVwZrT1nYhJcLdFsOuAgBiEvNpQjRyCkXmHsU=
             ← [SSH fingerprint — record for MITM detection / device correlation]

[  rcS  ] Loading wifi credentials from /etc/config/wireless
[  wifi ] Connecting to SSID: HomeNetwork_2G  passphrase: S3cr3tWifi2024!
          ← [PLAINTEXT WIFI PASSWORD in boot log]

[  rcS  ] Starting mosquitto MQTT broker...
[  mosquitto] config: /etc/mosquitto/mosquitto.conf
[  mosquitto] Starting in background

[  rcS  ] Starting telemetry agent...
[  agent ] Connecting to api.vendor-cloud.com:8883
[  agent ] Client ID: ESP-GW-A3F2
[  agent ] Auth: user=device_fleet  pass=Fl33tS3cr3t!2023
           ← [MQTT CREDENTIALS for fleet management backend]

[  rcS  ] Starting web interface on port 80...
[  httpd ] Document root: /www
[  httpd ] CGI scripts: /www/cgi-bin/

[  rcS  ] Running factory defaults check...
[  init ] Admin password not changed from default
[  init ] Default credentials: admin / admin1234
          ← [DEFAULT ADMIN PASSWORD — likely valid right now]

[  rcS  ] Init complete. System ready.

esp-gw login:

From this single boot log, you now have: - WiFi passphrase for the local network - MQTT fleet credentials for the cloud backend - Default web admin credentials - SSH host key fingerprint - Kernel version for CVE research (Linux 4.14.221) - Full filesystem layout (squashfs root, tmpfs /tmp) - Device hostname and MAC address

None of this required any exploitation. The device printed it unprompted.

grep Patterns for Fast Extraction

When working with captured log files, use grep to immediately surface the most interesting lines.

# Credential hunting — most important
grep -iE 'password|passwd|secret|key|token|credential|passphrase' boot.log

# Error and failure analysis
grep -iE 'error|fail|warn|denied|refused|cannot|unable' boot.log

# User and privilege context
grep -iE 'root|admin|user|uid|gid|sudo|privilege' boot.log

# Network and endpoint discovery
grep -iE 'connect|http|mqtt|ssh|ftp|telnet|api\.|://|port' boot.log

# File path extraction
grep -oE '(/[a-zA-Z0-9_./-]+){2,}' boot.log | sort -u

# Version strings for CVE research
grep -iE 'version|ver |v[0-9]+\.[0-9]' boot.log

Live Pattern Detection with espilon-monitor

The grep workflow works on captured files, but espilon-monitor can flag sensitive content in real time as the device boots — before you have even finished your coffee.

Create a pattern file:

mkdir -p patterns

cat > patterns/secrets.pat << EOF
CRITICAL  password=
CRITICAL  passwd=
CRITICAL  passphrase=
CRITICAL  default credentials
HIGH      token=
HIGH      api_key=
HIGH      auth:
HIGH      secret=
WARN      admin
WARN      root
INFO      version
INFO      kernel
EOF

Run espilon-monitor with the pattern file:

./espilon-monitor --patterns patterns/secrets.pat /dev/ttyUSB0

Lines matching CRITICAL patterns will be highlighted in red and logged with a [CRITICAL] prefix in the output file. HIGH severity lines get flagged in orange. This lets you catch credentials the instant they appear on the console, even during a fast boot sequence where you might miss a line scrolling by.

Combined with --logdir, you get a structured log with timestamps, severity flags, and a separate findings.log that contains only the matched lines:

./espilon-monitor --patterns patterns/secrets.pat --logdir ./target_logs /dev/ttyUSB0
# Generates:
#   target_logs/boot_20260514_134512.log    — full raw log
#   target_logs/findings_20260514_134512.log — only flagged lines

Full Boot Capture Workflow

Step 1: Prepare your environment before touching the device.

# picocom with timestamped log
picocom -b 115200 --logfile boot_$(date +%Y%m%d_%H%M%S).log /dev/ttyUSB0

# espilon-monitor with full structured logging
./espilon-monitor --logdir ./target_logs /dev/ttyUSB0

Step 2: Apply power to the device. Do not type anything. Let it boot completely.

Step 3: Once you see a login prompt or the device goes idle, you have the full log.

Step 4: Post-capture analysis.

# Quick triage
grep -iE 'password|passwd|passphrase|secret|token' boot_*.log

# Count unique errors
grep -iE 'error|fail' boot_*.log | sort | uniq -c | sort -rn | head -20

# Extract all file paths mentioned
grep -oE '(/[a-zA-Z0-9_./-]+){2,}' boot_*.log | sort -u > paths.txt

# Extract version strings for CVE lookup
grep -iE 'version [0-9]|v[0-9]+\.[0-9]+\.[0-9]+' boot_*.log | sort -u > versions.txt

Step 5: Research kernel version against NVD and vendor security advisories. Check every service version. Cross-reference credentials against other interfaces (web UI, Telnet, SSH).

What to Do When the Log Is Clean

Some devices suppress boot output intentionally, redirect console to null, or use a custom init that minimizes verbosity. Options:

• Interrupt the bootloader: If U-Boot is present and the countdown is visible, pressing a key drops you to U-Boot shell where you can read environment variables, flash contents, and boot arguments directly.
• Modify boot arguments: In U-Boot, change the kernel command line to add console=ttyS0,115200 loglevel=7 earlyprintk to force verbose kernel output.
• Look for a second UART: Some devices have a debug UART separate from the console UART, often at a different baudrate, sometimes printing more detailed diagnostics.

The boot log is your first and best source of intelligence on any embedded Linux device. Always capture it. Always analyze it before doing anything else.