2026 · Conference paper
Decompiling the Synergy: An Empirical Study of Human–LLM Teaming in Software Reverse Engineering
Studies how human analysts work with LLM assistance during static software reverse engineering. A survey of 153 practitioners informed six LLM features built into an instrumented IDA Pro plugin. Then 48 participants (24 self-reported novices and 24 experts) analyzed two CTF-style binaries, with the features enabled on one randomly assigned challenge. From graded writeups, interaction logs and surveys, the authors report 18 findings: novice understanding rose by about 98% to expert levels while experts gained little, known-algorithm functions were triaged up to 2.4× faster, and artifact recovery rose by at least 66%, alongside harmful hallucinations.
Scope and limitations
A human study of LLM-assisted analysis, not an evaluation of autonomous agents or of LLM decompilation accuracy. It uses two unobfuscated C challenges of 26 functions each, static analysis in IDA Pro only (no debuggers or symbolic execution), researcher-written prompts, and a choice of GPT-4o, GPT-4o-Mini, GPT-4-Turbo, Claude-3-5-Sonnet or Gemini-Pro. The 2024–2025 sessions retain 48 of 51 participants; expertise is self-reported, and understanding is a rubric score of writeups graded by two researchers. LLM-created artifacts were not correlated with better understanding, and three LLM users reported a nonexistent buffer overflow that an LLM had suggested; the vulnerability-finding feature was negatively correlated with understanding. Individual observations, writeups and survey responses are not released, even on request; aggregated data is available only on request.
Why it fits this collection
Measures how LLM assistance inside a decompiler changes human reverse-engineering outcomes, complementing agent-only evaluations. The paper points to DAILA, listed in the curated catalog, as the actively maintained version of its study plugin.
Publication and review details
- Venue
- NDSS 2026
- Status
- Conference paper
- Publication / posting date
- 2026 (exact date not verified)
- Date source
- The Internet Society’s Crossref deposit and the official NDSS paper identify publication year 2026. The PDF footer names the 23–27 February 2026 symposium; no finer publication date is asserted from the event dates.
- DOI
- 10.14722/ndss.2026.240380
The nine-author order follows the PDF, the NDSS paper page and Crossref. The PDF spells ‘Adam Doupé’, while Crossref and the paper page give ‘Doupe’; the page also lists Ruoyu “Fish” Wang. The PDF footer gives ISBN 979-8-9919276-8-0 and the DOI; Crossref lists no pages, and no page range was verified. No arXiv version was found. Footnote 1 and the Open Science section link mahaloz/dec-synergy-study. Its reviewed 2026-07-22 snapshot declares BSD-2-Clause and contains both challenges (sources, binaries, IDA databases and a solver) and the study infrastructure: a pwn.college Dojo module and REaLLM, the study version of DAILA. REaLLM requires Python 3.10+, libbs 2.9.1 and dailalib 3.15.6; study functionality needs a separately licensed IDA Pro 9 installation and the user’s own OpenAI, Anthropic or Gemini API keys. The infrastructure README documents a Docker build and a dojo.patch that this snapshot does not contain. The paper reports the known-algorithm speedup as up to 2.4× in the abstract, 238% in the introduction and at least 248% for experts on four functions in Finding 7. Finding 6 calls a p = 0.4 result statistically significant, despite the paper’s α = 0.05. These figures are author-reported. No upstream code was run and no experiments were reproduced.
Sources reviewed
Source review only; experiments were not rerun for this collection. Cite the original authors using the linked paper or BibTeX.
Primary sources
- https://www.ndss-symposium.org/ndss-paper/decompiling-the-synergy-an-empirical-study-of-human-llm-teaming-in-software-reverse-engineering/
- https://www.ndss-symposium.org/wp-content/uploads/2026-f380-paper.pdf
- https://doi.org/10.14722/ndss.2026.240380
- https://api.crossref.org/works/10.14722/ndss.2026.240380
- README.md GitHub · mahaloz/dec-synergy-study · Revision aef1c5b
- LICENSE GitHub · mahaloz/dec-synergy-study · Revision aef1c5b
- infrastructure/README.md GitHub · mahaloz/dec-synergy-study · Revision aef1c5b
- infrastructure/reallm-plugin/setup.cfg GitHub · mahaloz/dec-synergy-study · Revision aef1c5b
- infrastructure/reallm-plugin/reallm/__init__.py GitHub · mahaloz/dec-synergy-study · Revision aef1c5b
- challenges GitHub · mahaloz/dec-synergy-study · Revision aef1c5b