Lesson 2 — Filesystem Hunting

The Mindset: Systematic Over Lucky

Random browsing through a filesystem produces random results. Systematic grep does not. The goal of this lesson is to build a repeatable search workflow that covers all the obvious locations before you move to harder targets.

Assume you have already extracted the firmware and you are inside the squashfs-root/ directory. Everything here is readable. Start wide, then narrow.

Step 0: Orientation

Before running any searches, understand what you are looking at.

# Size of the filesystem
du -sh squashfs-root/

# What kind of device is this?
cat squashfs-root/etc/openwrt_release 2>/dev/null
cat squashfs-root/etc/os-release 2>/dev/null
cat squashfs-root/etc/version 2>/dev/null

# What binaries exist?
ls squashfs-root/usr/bin/ squashfs-root/usr/sbin/ squashfs-root/bin/

# Any web server?
ls squashfs-root/www/ squashfs-root/usr/www/ squashfs-root/srv/ 2>/dev/null

Knowing the OS and device type tells you which NVRAM format to expect, which web stack is present, and which directories are likely to contain interesting material.

Step 1: Credential Grep in Config Files

cd squashfs-root/

# Passwords in structured config files
grep -rEi "(password|passwd|pwd|passw)\s*[:=]\s*\S+" . \
  --include="*.conf" \
  --include="*.cfg" \
  --include="*.ini" \
  --include="*.json" \
  -l 2>/dev/null

# Then read the matching files
grep -rEi "(password|passwd|pwd|passw)\s*[:=]\s*\S+" etc/ \
  --include="*.conf" --include="*.cfg" --include="*.ini" 2>/dev/null

The -l flag first gives you the list of files. Then you run without -l to see the actual matching lines. This two-step approach avoids walls of output when the pattern matches hundreds of times in binary-adjacent files.

Step 2: API Key Patterns

API keys have characteristic shapes. Search for them by pattern, not by keyword.

# Generic: long alphanumeric tokens (32+ chars in quotes or at line start)
grep -rE "['\"][A-Za-z0-9_\-]{32,}['\"]" etc/ www/ usr/ 2>/dev/null

# AWS Access Key ID — always starts with AKIA
grep -rE "AKIA[0-9A-Z]{16}" . 2>/dev/null

# AWS Secret Access Key — 40-char base64-ish after specific keywords
grep -rEi "aws_secret_access_key\s*[:=]\s*[A-Za-z0-9/+=]{40}" . 2>/dev/null

# Google API key format
grep -rE "AIza[0-9A-Za-z\-_]{35}" . 2>/dev/null

# GitHub personal access token (classic)
grep -rE "ghp_[A-Za-z0-9]{36}" . 2>/dev/null

# Generic Bearer token pattern
grep -rEi "bearer\s+[A-Za-z0-9\-_\.]{20,}" . 2>/dev/null

Step 3: Private Keys and Certificates

Private keys are the most critical finding. A device certificate private key allows you to impersonate the device or decrypt its TLS traffic.

# Find PEM-encoded private keys by header string
grep -rl "BEGIN RSA PRIVATE KEY" . 2>/dev/null
grep -rl "BEGIN PRIVATE KEY" . 2>/dev/null
grep -rl "BEGIN EC PRIVATE KEY" . 2>/dev/null
grep -rl "BEGIN OPENSSH PRIVATE KEY" . 2>/dev/null

# Find certificate files by extension
find . -name "*.pem" -o -name "*.crt" -o -name "*.cer" \
       -o -name "*.p12" -o -name "*.pfx" -o -name "*.der" 2>/dev/null

# Check all found PEM files — is this a key or just a cert?
for f in $(find . -name "*.pem" 2>/dev/null); do
    echo "--- $f ---"
    head -1 "$f"
done

A file containing BEGIN CERTIFICATE is a public certificate — interesting but less critical. A file containing any PRIVATE KEY variant is a private key — this is a critical finding.

Step 4: Init Script Mining

Init scripts run as root at boot. They are the place where services are started with credentials. These scripts are shell code — all arguments are visible.

# Password-like arguments in init scripts
grep -r "password\|passwd\|-P \|-p \|--password" \
  etc/init.d/ etc/rc.d/ etc/rc.local 2>/dev/null

# Exported environment variables that look like secrets
grep -r "export.*KEY\|export.*TOKEN\|export.*SECRET\|export.*PASSWORD" \
  . 2>/dev/null

# MQTT broker credentials (common in IoT)
grep -r "mosquitto\|mqtt" etc/init.d/ etc/rc.d/ 2>/dev/null | grep -i "pass\|user"

# Database passwords passed to mysqld or sqlite
grep -r "mysql\|sqlite\|database" etc/init.d/ 2>/dev/null | grep -i "pass"

Pay attention to scripts that source another file (source /etc/config/secrets or . /etc/profile). Follow the include chain — the actual credentials are in the sourced file.

Step 5: Web Application Secrets

The web interface is where credentials are most frequently exposed, because web developers coming from a server background often do not realize that "server-side" code in firmware is fully readable.

# PHP config files
find www/ -name "config*.php" -o -name "settings*.php" \
          -o -name "database.php" -o -name "connect.php" 2>/dev/null

# JavaScript files with embedded keys (common in React/Vue frontends)
grep -rE "(api_key|apiKey|apiSecret|clientSecret)\s*[:=]\s*['\"][^'\"]{10,}['\"]" \
  www/ 2>/dev/null

# CGI scripts — often shell scripts comparing a hardcoded password
grep -r "admin\|password\|passwd\|root" www/cgi-bin/ 2>/dev/null | \
  grep -v "^Binary"

# Look for HTTP Basic Auth credentials embedded in CGI
grep -rE "Authorization:\s*Basic\s+[A-Za-z0-9+/=]{10,}" . 2>/dev/null

CGI scripts written as shell scripts are extremely common on older embedded devices. They look like this:

# Example of what you might find inside a CGI script
if [ "$USER" = "admin" ] && [ "$PASS" = "telnet1234" ]; then
    echo "authenticated"
fi

That pattern is exactly what you are searching for.

Step 6: The /etc/shadow Trick

Even when shadow password hashing is enabled, the hashes in /etc/shadow are offline-crackable. More importantly, many embedded devices use weak hashing algorithms.

# Show all accounts that have an active password (not locked with ! or *)
grep -v ":\!:\|:\*:" squashfs-root/etc/shadow 2>/dev/null

# Show the hash algorithm used (first field after username:)
# $1$ = MD5 (fast to crack)
# $5$ = SHA-256
# $6$ = SHA-512
# No $ prefix = DES (extremely fast to crack)

If you see root:$1$ — that is MD5-hashed. Run it through hashcat with rockyou.txt in mode 500. You will crack a surprising percentage.

If you see root:0gjqgzT9YBHUY — that is DES with no $ prefix. Mode 1500 in hashcat. Crackable in seconds on a GPU.

If /etc/shadow is absent and credentials are in /etc/passwd, the password field is in the second column (root:HASH:...). DES hashes in /etc/passwd are extremely common on older embedded Linux.

Step 7: Database Files

# Find all SQLite databases
find . -name "*.db" -o -name "*.sqlite" -o -name "*.sqlite3" 2>/dev/null

# Quick dump without sqlite3 installed
strings squashfs-root/var/db/users.db | grep -iE "admin|pass|user|token"

# Full dump if sqlite3 is available on your machine
sqlite3 squashfs-root/var/db/config.db ".dump" 2>/dev/null | \
  grep -iE "password|token|key|secret"

Putting It Together: A First-Pass Script

Run this from your working directory (the directory containing squashfs-root/):

#!/bin/bash
TARGET="squashfs-root"

echo "[*] Config file credentials"
grep -rEi "(password|passwd|pwd)\s*[:=]\s*\S+" $TARGET/etc/ \
  --include="*.conf" --include="*.cfg" --include="*.ini" 2>/dev/null

echo "[*] Private keys"
grep -rl "PRIVATE KEY" $TARGET 2>/dev/null

echo "[*] AWS keys"
grep -rE "AKIA[0-9A-Z]{16}" $TARGET 2>/dev/null

echo "[*] Init script exports"
grep -r "export.*\(KEY\|TOKEN\|SECRET\|PASSWORD\)" $TARGET/etc/ 2>/dev/null

echo "[*] Shadow accounts"
grep -v ":\!:\|:\*:" $TARGET/etc/shadow 2>/dev/null

echo "[*] SQLite databases"
find $TARGET -name "*.db" -o -name "*.sqlite" 2>/dev/null | \
  xargs -I{} strings {} | grep -iE "pass|key|token" 2>/dev/null

This script is intentionally noisy. It will produce false positives. Your job is to triage the output and confirm which findings are real secrets versus example values, documentation, or variable names.

What Counts as a Real Finding

A real finding requires: 1. The secret is not a placeholder — your_password_here, CHANGEME, example.com are not secrets. 2. The secret has the right format for its type — An AWS key that starts with AKIA and is followed by exactly 16 uppercase alphanumeric characters is valid format. 3. The secret is reachable — A WiFi PSK in an init script that runs on the production device is a real credential that someone on the street within radio range can use.

Document every real finding with: the full file path, the line number, the surrounding context (two lines before and after), and the type of secret.