RESEARCH & EVIDENCE

Papers & benchmarks

10 source-reviewed papers on binary reverse engineering, decompilation, agent evaluation, and website reconstruction, with citations and research limitations.

Browse repositoriesDownload bibliography · BibTeX ↓

Conference papers, accepted papers, and preprints are labeled separately. We check primary sources; inclusion does not mean experiments were rerun or results independently reproduced.

Binary reverse engineering

9 papers
2024EMNLP 2024Conference paper

LLM4Decompile: Decompiling Binary Code with Large Language Models

Hanzhuo Tan; Qi Luo; Jing Li; Yuqun Zhang

Trains language models for assembly-to-C decompilation and for refining Ghidra pseudocode, evaluating whether recovered functions can be recompiled and executed correctly.

Scope & limitations

The paper limits its experiments to C on x86 and function-level benchmarks. Re-executability on sampled tests does not establish whole-program equivalence or recovery of the original source. Later repository releases are separate from the 2024 paper.

Bibliographic details & sources

Why it fits this collection

Direct evidence for LLM decompilation and a concrete baseline for evaluating agent-generated or refined pseudocode.

Publication / posting date
November 2024
Date source
ACL proceedings publication month
First arXiv submission
March 8, 2024
arXiv
2403.05286
Reviewed arXiv version
v3
DOI
10.18653/v1/2024.emnlp-main.203
Sources checked
October 16, 2026

Publication date is the conference month; first_arxiv_date is recorded separately.

Primary sources

Source review only; experiments were not rerun for this collection.

2025Findings of ACL 2025Conference paper

DecompileBench: A Comprehensive Benchmark for Evaluating Decompilers in Real-World Scenarios

Zeyu Gao; Yuxin Cui; Hao Wang; Siliang Qin; Yuanda Wang; Zhang Bolun; Chao Zhang

Evaluates traditional and LLM-based decompilers using functions from real programs, runtime behavioral checks, and an LLM-assisted assessment of analyst-facing code understandability.

Scope & limitations

The evaluated corpus has 23,400 functions from 130 programs. Fuzzing-based checks have finite coverage, and understandability scores depend partly on an LLM judge. The artifact warns that rebuilt OSS-Fuzz projects can differ from the paper's versions.

Bibliographic details & sources

Why it fits this collection

Helps readers separate readable decompiler output from behaviorally faithful output when choosing or evaluating reverse-engineering tools.

Publication / posting date
July 2025
Date source
ACL proceedings publication month
First arXiv submission
May 16, 2025
arXiv
2505.11340
Reviewed arXiv version
v1
DOI
10.18653/v1/2025.findings-acl.1194
Sources checked
October 16, 2026

The published ACL record and PDF use 'Zhang Bolun'; the arXiv record uses 'Bolun Zhang'. The paper's Jennieett/DecompileBench link now redirects to vul337/DecompileBench. This is distinct from Tan et al.'s Decompile-Bench (arXiv:2505.12668).

Primary sources

Source review only; experiments were not rerun for this collection.

2019ASE 2019Conference paper

DIRE: A Neural Approach to Decompiled Identifier Naming

Jeremy Lacomis; Pengcheng Yin; Edward J. Schwartz; Miltiadis Allamanis; Claire Le Goues; Graham Neubig; Bogdan Vasilescu

Combines lexical and structural information from decompiler output to predict meaningful variable names, with a training and evaluation corpus derived from compiled C projects.

Scope & limitations

This is learned identifier naming, not complete source reconstruction or an autonomous agent. Its reported results concern an x86-64 C corpus and depend on decompiler output. The original author repository is deprecated and points to a successor.

Bibliographic details & sources

Why it fits this collection

A foundational reference for the renaming step frequently delegated to reverse-engineering assistants.

Publication / posting date
November 2019
Date source
Author artifact citation and ASE 2019 conference program
First arXiv submission
September 19, 2019
arXiv
1909.09029
Reviewed arXiv version
v2
Sources checked
October 16, 2026

The arXiv manuscript title uses 'Naming'; the ASE program and original code repository also use the variant 'Renaming'. Both refer to this paper. Code URL is the verified original author artifact, explicitly deprecated, rather than a claim of current package support.

Primary sources

Source review only; experiments were not rerun for this collection.

2022USENIX Security 2022Conference paper

Augmenting Decompiler Output with Learned Variable Names and Types

Qibin Chen; Jeremy Lacomis; Edward J. Schwartz; Claire Le Goues; Graham Neubig; Bogdan Vasilescu

Introduces DIRTY, a Transformer model that recommends variable names and types from decompiled code and layout information, together with the DIRT corpus for training and evaluation.

Scope & limitations

The original pipeline is built around C programs and Hex-Rays output. Predicted names and types are learned hypotheses, not guaranteed original symbols. The published paper is from 2022, although its arXiv preprint first appeared in 2021.

Bibliographic details & sources

Why it fits this collection

Extends name recovery to types, making it directly relevant to assessing semantic annotations added by binary-analysis assistants.

Publication / posting date
August 2022
Date source
USENIX proceedings BibTeX publication month
First arXiv submission
August 13, 2021
arXiv
2108.06363
Reviewed arXiv version
v1
Sources checked
October 16, 2026

Year and publication month come from the final USENIX conference record; the arXiv date is not substituted for publication.

Primary sources

Source review only; experiments were not rerun for this collection.

2024ACM CCS 2024Conference paper

ReSym: Harnessing LLMs to Recover Variable and Data Structure Symbols from Stripped Binaries

Danning Xie; Zhuo Zhang; Nan Jiang; Xiangzhe Xu; Lin Tan; Xiangyu Zhang

Combines two fine-tuned language models with Prolog-based cross-function reasoning to infer local-variable symbols and user-defined data structures from stripped-binary decompilation.

Scope & limitations

The method depends on underlying decompilation and learned predictions over C/C++ data. Cross-checking reduces conflicting predictions but does not prove that recovered names or layouts match the unavailable source. The public artifact includes models, processed data, and evaluation code.

Bibliographic details & sources

Why it fits this collection

A concrete hybrid approach for grounding LLM-generated reverse-engineering annotations in program analysis.

Publication / posting date
October 2024
Date source
Author-hosted published PDF identifies CCS 2024, October 14–18
DOI
10.1145/3658644.3670340
Sources checked
October 16, 2026

Publisher-page retrieval was unavailable, but the DOI, title, six authors, venue, and October 2024 date are confirmed by the publicly accessible published PDF and official author artifact.

Primary sources

Source review only; experiments were not rerun for this collection.

2026arXivPreprint

CrackMeBench: Binary Reverse Engineering for Agents

Isaac David; Arthur Gervais

Evaluates agents on executable-checked CrackMe tasks: infer validation logic from binaries and submit an accepted input or artifact, while recording tool use, time, and scoring attempts.

Scope & limitations

The v0 study uses 12 generated tasks and eight public calibration tasks, Linux x86-64 ELF binaries, a shell interface, and a five-minute budget. Its compact educational tasks do not establish performance on large applications. No paper-linked public artifact URL was verified.

Bibliographic details & sources

Why it fits this collection

Directly measures whether an autonomous reverse-engineering workflow produces a verifiably accepted result.

Publication / posting date
May 11, 2026
Date source
arXiv v1 submission
First arXiv submission
May 11, 2026
arXiv
2605.10597
Reviewed arXiv version
v1
Sources checked
October 16, 2026

No conference venue is stated in the reviewed arXiv record. Tool repositories cited in its bibliography are not treated as the paper's own code.

Primary sources

Source review only; experiments were not rerun for this collection.

2026COLM 2026Conference accepted

CREBench: Evaluating Large Language Models in Cryptographic Binary Reverse Engineering

Baicheng Chen; Yu Wang; Ziheng Zhou; Xiangru Liu; Juanru Li; Yilei Chen; Tianxing He

Benchmarks cryptographic binary reverse engineering across algorithm recognition, key or IV extraction, wrapper reconstruction, and flag recovery using 432 controlled challenges.

Scope & limitations

The corpus combines 48 algorithms, three insecure key-use scenarios, and three difficulty levels. It excludes professional-grade obfuscators such as Tigress and O-LLVM; results should not be generalized to heavily protected arbitrary binaries.

Bibliographic details & sources

Why it fits this collection

Provides a focused, decomposed test of cryptographic behavior recovery, with a verified official implementation.

Publication / posting date
2026 (exact date not verified)
Date source
COLM 2026 venue confirmed by arXiv comments and official author project; exact proceedings publication date not verified
First arXiv submission
April 4, 2026
arXiv
2604.03750
Reviewed arXiv version
v2
Sources checked
October 16, 2026

arXiv v2 is dated 2026-08-06. The official project announces COLM 2026 acceptance; its BibTeX still uses an arXiv citation. This is unrelated to the creativity-evaluation paper also named CreBench.

Primary sources

Source review only; experiments were not rerun for this collection.

2026arXivPreprint

The Next Challenge for Agentic Cybersecurity: A Realistic, Contamination-Free Reverse Engineering Benchmark

Jeremy Spence; Nicholas Assaderaghi; Feng Xiao; Jinhao Zhu; Nikil Ravi; Xiangyu Qi; Matthew Jagielski; Raluca Ada Popa; Eric Wallace; Guannan Wei; Yangruibo Ding; Zhuo Zhang

Introduces SRE-Bench, with privately authored programs and protected binary variants, to study agent recovery of program semantics while reducing training-data contamination.

Scope & limitations

The revised study reports 19 programs, 262 binary instances, and 1,572 graded tasks. Binaries and task instances are not publicly released; access is through an evaluation service. Its contamination-control claim is the authors' methodology, not an independent guarantee.

Bibliographic details & sources

Why it fits this collection

Adds realistic-scale and contamination-aware evaluation context for agent-assisted reverse engineering without mislabeling a private corpus as open data.

Publication / posting date
August 11, 2026
Date source
arXiv v1 submission; summary and author list use revised v2
First arXiv submission
August 11, 2026
arXiv
2608.11469
Reviewed arXiv version
v2
Sources checked
October 16, 2026

Use the 12-author v2 list, revised 2026-10-01, rather than the older eight-author v1 search snippets. No conference venue or own public code repository was verified. AgentRE-Bench GitHub links in the bibliography refer to another benchmark.

Primary sources

Source review only; experiments were not rerun for this collection.

2026arXivPreprint

Humans vs. Machines: A Comparison With AI Agents in Software Reverse Engineering

Samuele Doria; Stefano De Rosa; Ruoyu Wang; Zion Leonahenahe Basque; Eleonora Losiouk; Simone Aonzo

Compares human and LLM-agent understanding and navigation on the same static reverse-engineering challenges, then studies how binary representations consume context windows.

Scope & limitations

Understanding is graded on two compact challenges; the separate 4,000-binary study measures context capacity, not task-solving accuracy. The interface is static IDA/MCP without dynamic execution, and prior exposure to challenge source remains an acknowledged validity threat.

Bibliographic details & sources

Why it fits this collection

Informs tool-interface design, selective retrieval, and persistent analysis state for reverse-engineering agents.

Publication / posting date
October 10, 2026
Date source
arXiv v1 submission
First arXiv submission
October 10, 2026
arXiv
2610.14960
Reviewed arXiv version
v1
Sources checked
October 16, 2026

Recent preprint, with no confirmed conference venue or paper-specific public code link. arXiv marks its generated DOI as pending registration, so the canonical link is the arXiv record.

Primary sources

Source review only; experiments were not rerun for this collection.

Website reconstruction

1 paper
2025NAACL 2025Conference paper

Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineering

Chenglei Si; Yanzhe Zhang; Ryan Li; Zhengyuan Yang; Ruibo Liu; Diyi Yang

Evaluates reconstruction of frontend code from webpage screenshots using real-world reference pages, visual-element and layout metrics, and human assessment.

Scope & limitations

This is visual website reconstruction, not binary decompilation or recovery of backend behavior and interactive workflows. The main benchmark contains 484 webpages with placeholder images; code and data have separate licensing terms.

Bibliographic details & sources

Why it fits this collection

Directly supports the collection's screenshot-to-website reconstruction scope while keeping visual fidelity separate from behavioral equivalence.

Publication / posting date
April 2025
Date source
ACL proceedings publication month
arXiv
2403.03163
DOI
10.18653/v1/2025.naacl-long.199
Sources checked
October 16, 2026

Use the final NAACL 2025 title and six-author list; the older project page/arXiv-era citation has a different title and omits Ryan Li. SALT-NLP/Design2Code hosts the project website, whereas NoviScl/Design2Code is the verified research-code repository.

Primary sources

Source review only; experiments were not rerun for this collection.