2026 · Preprint
SoK: Are LLMs Reliable at Source Code Recovery? A Taxonomy and Empirical Evaluation
This SoK organizes LLM-assisted source-recovery designs and separates compilation, readability, structural and behavioral metrics. Its controlled study varies architecture, compiler optimization and symbol availability, with smaller ablations for input representation, model roles and source language.
Scope and limitations
The 45,000 cases are variants of 500 functions, not independent programs. Input/language ablations use 50 functions; translated language variants were produced with an LLM and checked using finite I/O tests. The study assumes a derivable call interface, is prompt-sensitive and cannot exclude ExeBench pretraining exposure. Test agreement and relocation-masked byte matching have different assumptions.
Why it fits this collection
Adds a recent evaluation taxonomy and controlled evidence for comparing source-recovery methods without collapsing distinct correctness signals.
Publication and review details
- Venue
- arXiv
- Status
- Preprint
- Publication / posting date
- October 8, 2026
- Date source
- First arXiv submission date; no separate proceedings or journal publication was verified.
- First arXiv submission
- October 8, 2026
- arXiv
- 2610.11556
- Reviewed arXiv version
- v1
The arXiv v1 article is CC BY 4.0. Its anonymous artifact was readable in a browser; the page shows an October 1 snapshot available until 2027-03-29, without a full Git commit. The inspected README says a fresh database cannot directly re-score shipped generation files; redrawing supplied CSV results is a separate step. Artifact licensing was not established from the inspected README/root list. No archive, model or experiment was run.
Sources reviewed
Source review only; experiments were not rerun for this collection. Cite the original authors using the linked paper or BibTeX.