2026 · Preprint
SCDBench: A Benchmark for LLM-Based Smart Contract Decompilers
Benchmarks LLM-based EVM decompilation on 600 source-verified Ethereum Solidity contracts, sampled in easy, medium and hard thirds from 772,736 unique bytecode–source pairs after near-duplicate filtering. Each task supplies EVM assembly and optional four-byte selector hints and asks for self-contained Solidity with compiler settings. Outputs are scored in four cumulative stages: format completeness, compilation, ABI recovery, and semantic consistency, where 227,383 recorded test cases for 14,553 public functions are replayed on the recompiled output. With one compiler-feedback repair call, GPT-5.3-Codex compiled 90.3% of contracts, while the best semantic results were 29.4% of functions and 42 of 600 contracts recovered perfectly, both by Claude Opus 4.7.
Scope and limitations
Replay compares return data, reverts, logs and touched storage on generated inputs that reach 77.8% average code coverage; passing it is not proof of equivalence, and replay needs an Ethereum archive node. The contracts are source-verified Ethereum Solidity from compiler v0.4 to v0.8, so unverified, obfuscated, Vyper or other-chain bytecode is not covered. Three models are evaluated zero-shot through OpenRouter: Claude Opus 4.7 and GPT-5.3-Codex at high reasoning effort, and GLM-5 with and without extended thinking. Results and quoted costs describe those 2026 snapshots. Gigahorse and Heimdall-rs are scored only on selector and prototype recovery because they do not emit compilable Solidity; DiSCo and the fine-tuned decompiler of David et al. are discussed but not run.
Why it fits this collection
Adds a shared benchmark with execution-based semantic checks for smart-contract decompilers, complementing EVM decompilation papers that evaluate on separate datasets and metrics.
Publication and review details
- Venue
- arXiv
- Status
- Preprint
- Publication / posting date
- May 27, 2026
- Date source
- arXiv v1 submission date; no later version, journal reference or proceedings publication was found.
- First arXiv submission
- May 27, 2026
- arXiv
- 2605.29059
- Reviewed arXiv version
- v1
Reviewed arXiv v1, the only version, which is labelled a preprint and has no comment, journal reference or DOI. The paper says the benchmark dataset and evaluation artifacts, including prompts, scripts and model outputs, are provided, and its broader-impact appendix describes a released dataset, but v1 gives no URL. GitHub and Hugging Face searches found no release, so no code link is recorded. Compilation and replay use solc and Foundry. No dataset, code or model calls were run.
Sources reviewed
Source review only; experiments were not rerun for this collection. Cite the original authors using the linked paper or BibTeX.