ESP32 Security/ ESP32 NVS Forensics / NVS Decoding

NVS Decoding

practical
🔧
Required hardware
PC Linux esp-idf ou nvs_partition_tool.py

From Raw Bytes to Readable Data

You now have an nvs.bin file — 24576 bytes of structured binary data. The goal of this lesson is to convert those bytes into a human-readable list of namespace → key → value triplets. We cover two approaches: automated parsing with nvs-tool (fast, recommended for well-formed partitions), and manual decoding (necessary when the parser fails, or when you want to understand exactly what you are looking at).

Installing nvs-tool

nvs-tool is a Python utility for parsing ESP32 NVS binary images. It is not part of the standard esptool package but can be installed separately:

pip install nvs-tool

Verify the installation:

nvs_tool --version
# or
python3 -m nvs_tool --help

If pip install nvs-tool fails, an alternative is the NVS partition utility from the Espressif IDF component manager or building from source:

# From the ESP-IDF tools directory (if you have IDF installed)
python3 $IDF_PATH/tools/mass_mfg/nvs_tool.py --help

# Or use the nvs_partition_tool from esp-idf-nvs-partition-gen
pip install esp-idf-nvs-partition-gen

Basic Decoding

nvs_tool read nvs.bin

The output groups entries by namespace. For a typical WiFi provisioning device:

Namespace: wifi
  ssid       [string]  HomeNetwork_5G
  password   [string]  hunter2correct

Namespace: device
  serial     [string]  ESP-A1B2C3D4E5
  firmware   [string]  v2.1.4-release
  hw_rev     [uint8]   3

Namespace: mqtt
  broker     [string]  192.168.1.50
  port       [uint16]  1883
  user       [string]  esp_client_001
  pass       [string]  Br0k3rP@ss!

Namespace: http
  api_key    [string]  sk-a8f3c1d9e2b67045...
  endpoint   [string]  https://api.example.com/v2

Each line shows the namespace, key name, data type in brackets, and the decoded value. Integer types are shown as decimal. Strings are shown as ASCII text. Blobs are shown as hex.

Understanding What nvs-tool Does Internally

Understanding the parsing logic helps you recover data when the tool fails.

Step 1: Page Discovery

nvs-tool scans the binary for valid page headers by looking for the magic value 0xA5A5A5A5 at 4096-byte aligned offsets. For a 24KB NVS partition, it checks offsets 0, 4096, 8192, 12288, 16384, and 20480.

MAGIC = 0xA5A5A5A5
PAGE_SIZE = 4096

for i in range(num_pages):
    offset = i * PAGE_SIZE
    magic = struct.unpack_from('<I', data, offset)[0]
    if magic == MAGIC:
        parse_page(data, offset)

Step 2: Entry State Bitmap

After the 32-byte page header, the 32-byte entry state bitmap is read. Each pair of bits represents one of the 126 possible entries:

  • Bits 11: empty slot
  • Bits 10: written (active)
  • Bits 00: erased (deleted — forensically interesting)
  • Bits 01: invalid
bitmap_offset = page_offset + 32
bitmap = data[bitmap_offset:bitmap_offset + 32]

for entry_idx in range(126):
    byte_idx = entry_idx // 4
    bit_pos = (entry_idx % 4) * 2
    state_bits = (bitmap[byte_idx] >> bit_pos) & 0b11
    # 0b11 = empty, 0b10 = written, 0b00 = erased

Step 3: Namespace Index Resolution

The first namespace entry in a page (those with namespace_index == 0) map namespace name strings to numeric indices. Parse these first:

namespace_map = {}  # index → name

for entry in page_entries:
    if entry.namespace_index == 0 and entry.type == TYPE_STRING:
        # The key IS the namespace name, value is the index number
        ns_index = entry.value[0]  # single byte stored in value field
        ns_name = entry.key.rstrip(b'\x00').decode('ascii')
        namespace_map[ns_index] = ns_name

Step 4: Data Entry Parsing

For each non-namespace entry, combine the namespace index, key, type, and value:

for entry in page_entries:
    if entry.namespace_index == 0:
        continue  # already handled as namespace definition

    ns_name = namespace_map.get(entry.namespace_index, f"<ns_{entry.namespace_index}>")
    key = entry.key.rstrip(b'\x00').decode('ascii')

    if entry.type in INTEGER_TYPES:
        value = parse_integer(entry.value, entry.type)
    elif entry.type == TYPE_STRING:
        value = parse_string(data, page_offset, entry)
    elif entry.type == TYPE_BLOB_IDX:
        value = parse_blob(data, page_offset, entry)

Manual Decoding with xxd

When nvs-tool is unavailable or the partition has corruption, raw hexdump analysis works. Here is what to look for:

xxd nvs.bin | head -80

Page 0 header (first 32 bytes):

00000000: a5a5 a5a5 0000 0001 fe00 0000 00ff ffff  ................
          ─────────── ─────────── ── ────────── ──
          magic        seq_num     ver  state      reserved...
  • a5 a5 a5 a5 — valid NVS page magic
  • 00 00 00 01 — sequence number 1 (first page written)
  • fe — state = 0xFE = active
  • 00 — version

Entry bitmap (bytes 32–63):

00000020: aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa  ................
          ─────────────────────────────────────────
          aa = 1010 1010 in binary
          Each pair: 10 = written entry

First entry (bytes 64–95):

00000040: 0021 0100 a3f1 8c2d 7769 6669 0000 0000  .!.....-wifi....
          ── ── ── ── ─────────── ───────────────────
          ns  ty  sp  ck  CRC32    key "wifi\0\0\0\0\0"
00000050: 0000 0000 0000 0000 0100 0000 0000 0000  ................
          ─────────────────────── ──
          key cont (padding)       value: index=1

Breaking down: - 00 — namespace_index = 0 (this is a namespace definition entry) - 21 — type = 0x21 = string - 01 — span = 1 (fits in one entry) - 00 — chunk_index = 0 - a3f1 8c2d — CRC32 - 7769 6669 00... — key = "wifi\0" (null-padded to 16 bytes) - 01 00 ... — value byte = 0x01, meaning namespace "wifi" is assigned index 1

String and Blob Entry Layout

String Entries

A string entry with span > 1 means the string data spills into subsequent entries:

Entry 0 (span=2): namespace_idx=1, type=0x21, key="password"
    value field (8 bytes): first 8 bytes of the string data
Entry 1 (span marker): contains continuation of string data (32 bytes total)

To reconstruct: take the 8 bytes from the value field of entry 0, then concatenate the full 32 bytes of entry 1, and trim at the null terminator.

For a 16-character password like "MyWifiPass123456": - Bytes 0–7: first 8 chars in the value field - Bytes 8–23: remaining 8 chars + null in the next 32-byte entry's first 16 bytes

Blob Entries

Blobs use three entry types working together:

  1. blob_idx (type 0x48): an index entry storing total blob size and chunk count
  2. blob_data (type 0x42): one or more data chunk entries, each holding up to 32 bytes
  3. The chunk_index field sequences the data entries in order

For a 96-byte TLS certificate stored as a blob: - blob_idx entry: total_size=96, chunk_count=3 - blob_data entry 0: chunk_index=0, 32 bytes of cert data - blob_data entry 1: chunk_index=1, 32 bytes of cert data - blob_data entry 2: chunk_index=2, 32 bytes of cert data

Recovering Deleted Entries

This is where NVS forensics goes beyond what the running application can see. Entries marked as erased in the bitmap (00 bits) may still have their 32-byte data blocks intact in flash.

To find deleted entries:

# Show all erased entries including their data (nvs-tool flag)
nvs_tool read --include-erased nvs.bin

# Or manually: look for entries where bitmap says 00 but data is not 0xFF
python3 << 'EOF'
import struct

with open('nvs.bin', 'rb') as f:
    data = f.read()

PAGE_SIZE = 4096
for page_num in range(len(data) // PAGE_SIZE):
    page_off = page_num * PAGE_SIZE
    magic = struct.unpack_from('<I', data, page_off)[0]
    if magic != 0xA5A5A5A5:
        continue

    bitmap_off = page_off + 32
    entries_off = page_off + 64

    for entry_idx in range(126):
        byte_idx = entry_idx // 4
        bit_pos = (entry_idx % 4) * 2
        state = (data[bitmap_off + byte_idx] >> bit_pos) & 0b11

        if state == 0b00:  # erased
            entry_off = entries_off + entry_idx * 32
            entry_data = data[entry_off:entry_off + 32]

            # Check if data is not all 0xFF (not truly blank)
            if entry_data != b'\xff' * 32:
                key_bytes = entry_data[8:24]
                key_str = key_bytes.split(b'\x00')[0].decode('ascii', errors='replace')
                print(f"Page {page_num}, Entry {entry_idx}: ERASED but has data, key='{key_str}'")
                print(f"  Raw: {entry_data.hex()}")
EOF

Deleted entries appear when: - A developer overwrote a credential (the old value is still in flash under an erased bitmap entry) - The application deleted an NVS key with nvs_erase_key() - A factory reset function called nvs_erase_all() — this marks entries erased but does not erase the flash sector immediately

What Encrypted NVS Looks Like

When NVS encryption is active, the page headers are partially visible (magic and sequence number are not encrypted) but all entry data is AES-XTS ciphertext.

xxd nvs.bin | head -20

With encryption:

00000000: a5a5 a5a5 0000 0001 fe00 0000 3a8f 21c4  ............:.!.
00000020: 7f93 c2e1 4ab5 d8f2 01ac 3e9d c77b 5f2a  ....J.....>..{_*
00000040: 8e2f a391 0b74 e6c3 d815 9a4f 2c7e b1d9  ./...t.....O,~..
00000050: 5f3a c801 e9b7 4f28 a632 5d1c 8f0e 7b42  _:....O(.2]...{B

Recognizing encrypted vs. unencrypted NVS: - Unencrypted: Entry starts at offset 64, first byte is a namespace index (0–254), second byte is a type (0x01, 0x02, 0x04, 0x08, 0x11, 0x12, 0x14, 0x18, 0x21, 0x41, 0x42, 0x48). Bytes 8–23 contain a readable ASCII key. - Encrypted: Bytes at entry offsets look like pseudo-random noise. No recognizable type bytes. No ASCII keys.

Decrypting NVS (if you have the nvs_keys partition)

# Extract key partition from full dump
dd if=full_dump.bin bs=1 skip=$((0x310000)) count=$((0x1000)) of=nvs_keys.bin

# Decrypt NVS using nvs-tool
nvs_tool read --key-partition nvs_keys.bin nvs.bin

# Or if using the IDF tool
python3 $IDF_PATH/tools/mass_mfg/nvs_tool.py \
    --key nvs_keys.bin \
    read nvs.bin

nvs-tool reads the first 32 bytes of nvs_keys.bin as the AES-XTS key, decrypts the NVS partition in memory, and then parses and displays the entries.

Complete Decoding Workflow

# 1. Quick check: is NVS present and valid?
xxd -l 4 nvs.bin
# Expect: a5 a5 a5 a5

# 2. Try automated parsing
nvs_tool read nvs.bin

# 3. If encrypted, check for nvs_keys
gen_esp32part.py ptable.bin | grep nvs_keys
# If found, extract and decrypt:
dd if=full_dump.bin bs=1 skip=$((0x310000)) count=$((0x1000)) of=nvs_keys.bin
nvs_tool read --key-partition nvs_keys.bin nvs.bin

# 4. Include erased/deleted entries in output
nvs_tool read --include-erased nvs.bin

# 5. Save output for analysis
nvs_tool read --include-erased nvs.bin > nvs_decoded.txt
grep -i "pass\|key\|secret\|token\|ssid\|cert" nvs_decoded.txt

The Layers of Recovery

NVS decoding is not binary — it is not just "got data" or "got nothing." A well-formed unencrypted NVS gives you all active entries immediately. Analyzing erased bitmap entries gives you previously deleted entries. Having the nvs_keys partition gives you decrypted entries even when encryption was intended to protect them. Each layer of technique expands the scope of what you can recover. In Lesson 5, we look at what that recovered data actually contains across real device categories.