Wiki / Concepts

Reverse Engineering

Recovering the design and behaviour of a program from the compiled artifact alone, when no source and no documentation exist.

Reverse engineering is the practice of recovering what a program does, and eventually how it was designed, starting from the compiled artifact alone. No source, no comments, no documentation: only bytes and whatever the toolchain happened to leave behind.

What it is

A compiler is a lossy transformation. It throws away variable names, type names, comments and most of the structure the author wrote, and emits machine code plus a thin layer of metadata. Reverse engineering runs that transformation backwards as far as it will go. Two families of technique do the work:

  • Static analysis reads the artifact without running it. A disassembler turns bytes into instructions, a decompiler lifts those instructions into readable pseudo-C, and cross-references reveal who calls what. Nothing executes, so nothing can hide behind a runtime check.
  • Dynamic analysis runs the artifact and watches. A debugger stops at chosen instructions and shows real register and memory values; an emulator runs foreign code on your own machine; tracing records the syscalls, library calls or MMIO accesses actually made.

Neither is sufficient alone. Static analysis shows every path but cannot tell you which one runs; dynamic analysis shows the truth about one execution and nothing about the rest.

Why it matters

Almost everything you attack is closed. Vendor firmware, a stripped binary in a rootfs, a proprietary protocol client, a licence check, a key derivation routine: none of it ships with source. Reverse engineering is what converts an opaque blob into an understood system, and understanding is the prerequisite for everything else, whether that is finding a memory-safety bug, recovering a hardcoded key, or reimplementing a protocol well enough to speak it.

The working loop

1. Identify   file, strings, architecture, entry point, whether it is stripped
2. Orient     find the parts you already understand: libc calls, error strings, format strings
3. Anchor     pick one known behaviour, find where it is implemented, work outwards
4. Name       rename every function and variable the moment you understand it
5. Verify     confirm with a debugger, an emulator, or by reimplementing the routine

Step 4 is the one beginners skip and the one that decides whether a large binary stays tractable. Renaming turns a wall of FUN_00401a30 into a map you can navigate a week later.

Anchors that pay off first

  • Strings. An error message, a format string, a filesystem path or a URL is a direct index into the code that produces it. Cross-reference it and you land in the middle of the relevant routine.
  • Library calls. strcpy, memcpy, malloc, open, recv are labelled even in a stripped binary if it links dynamically, and their arguments tell you the shape of the data.
  • Constants. Cryptographic S-boxes, magic numbers, protocol identifiers and CRC polynomials are recognisable on sight and identify a whole algorithm at once.
  • The entry point and the main loop. Reading a binary forwards from main rarely works; reading backwards from something specific almost always does.

Pitfalls

  • Guessing instead of verifying. A plausible reading of decompiled code is not a fact. Confirm it dynamically, or by reimplementing the routine and comparing outputs, before building on it.
  • Trusting the decompiler. Decompilation is a best-effort reconstruction. Wrong calling conventions, wrong types, misidentified structure fields and optimised-away branches all produce readable pseudo-C that is simply not what the CPU does. When it stops making sense, drop to the disassembly.
  • Reversing more than you need. The goal is an answer, not a complete understanding. If the question is "where does the key come from", stop when you know.
  • Ignoring the cheap path. Emulating a binary and letting it compute the answer for you is often faster than reading it, and a public tool may already implement whatever algorithm it uses.

What it pairs with

ghidra and radare2 are the usual static tools; qemu provides emulation when the target architecture is not yours. On embedded targets the artifact rarely arrives as a tidy executable, and firmware-reverse-engineering covers what changes when there is no operating system, no symbols and no file format to lean on.

Further reading