AwesomeREA.org

2020 · Conference paper

How far we have come: testing decompilation correctness of C decompilers

Zhibo Liu; Shuai Wang

Tests whether C decompilers preserve program behavior. The authors' DecFuzzer compiles a Csmith-generated program, decompiles the executable, substitutes the decompiled function for the original in the source, recompiles, and compares the two programs' outputs; equivalence-modulo-inputs (EMI) mutation then derives further test programs from the seeds. Across four popular decompilers the authors report 1,423 inputs that trigger decompilation errors and, after manual analysis, 13 bugs in two open-source decompilers. They argue that modern decompilers are making real progress on functional correctness, while type recovery and optimization still hinder accurate output.

Scope and limitations

The tests use small generated programs, not real-world binaries, and a failed recompilation is recorded separately from a behavioral mismatch. In the authors' artifact, test programs are compiled as 32-bit x86 with GCC at -O0, and Csmith is restricted to a single function without arrays, structs, unions, pointers or long long types, so optimized code and richer data types are outside that configuration. The four decompilers (IDA Pro, JEB3, RetDec and Radare2's r2ghidra plugin) were tested as of 2020; the error counts and bugs describe those releases, not current ones. IDA Pro and JEB3 are commercial and not included in the artifact, which sets up only RetDec and r2ghidra.

Why it fits this collection

Adds differential testing of decompiler correctness through recompilation, complementing the collection's decompiler benchmarks, such as DecompileBench, that evaluate output on real functions.

Publication and review details

Venue
ACM ISSTA 2020
Status
Conference paper
Publication / posting date
July 18, 2020
Date source
ACM’s Crossref deposit records print and online publication on 2020-07-18, and the ACM article page gives the same publication date. This entry uses that date, not the conference schedule.
DOI
10.1145/3395363.3397370

Crossref and the ACM page give the two-author order, pages 475–487 and the title in the lowercase form used here; the conference programme and the artifact README capitalize it “How Far We Have Come: Testing Decompilation Correctness of C Decompilers”. dl.acm.org refused automated requests during this review and no other copy of the paper text was found, so the method and results above follow the abstract on the ISSTA 2020 programme page, the ACM page captured by the Wayback Machine (abstract, reference list and an Artifacts Evaluated – Functional badge) and the artifact itself; paper-only details, such as per-decompiler results, are not recorded. Reference [9] of the paper links monkbai/DecFuzzer. The reviewed 2020-04-23 commit is the ISSTA’20 artifact-evaluation package: it targets Ubuntu 18.04, needs MySQL for EMI mutation, a prebuilt RetDec v4.0 and Radare2 with r2ghidra, and links seed and result data on Dropbox. The repository has no license file and GitHub reports none, so no license is stated here. Nothing was installed or run.

Sources reviewed

Source review only; experiments were not rerun for this collection. Cite the original authors using the linked paper or BibTeX.

Report an outdated entry Suggest a translation fix

Primary sources

  1. https://dl.acm.org/doi/10.1145/3395363.3397370
  2. https://api.crossref.org/works/10.1145/3395363.3397370
  3. https://web.archive.org/web/20251215175843/https://dl.acm.org/doi/abs/10.1145/3395363.3397370
  4. https://conf.researchr.org/details/issta-2020/issta-2020-papers/26/How-Far-We-Have-Come-Testing-Decompilation-Correctness-of-C-Decompilers
  5. README.md GitHub · monkbai/DecFuzzer · Revision 8e109c1
  6. src/Config.py GitHub · monkbai/DecFuzzer · Revision 8e109c1