AwesomeREA.org

2026 · Conference paper

CREBench: Evaluating Large Language Models in Cryptographic Binary Reverse Engineering

Baicheng Chen; Yu Wang; Ziheng Zhou; Xiangru Liu; Juanru Li; Yilei Chen; Tianxing He

Benchmarks cryptographic binary reverse engineering across algorithm recognition, key or IV extraction, wrapper reconstruction, and flag recovery using 432 controlled challenges.

Scope and limitations

The corpus combines 48 algorithms, three insecure key-use scenarios, and three difficulty levels. It excludes professional-grade obfuscators such as Tigress and O-LLVM; results should not be generalized to heavily protected arbitrary binaries.

Why it fits this collection

Provides a focused, decomposed test of cryptographic behavior recovery, with a verified official implementation.

Publication and review details

Venue
COLM 2026
Status
Conference paper
Publication / posting date
2026 (exact date not verified)
Date source
Publication year from the official COLM 2026 OpenReview record and its BibTeX; no finer publication date is asserted.
First arXiv submission
April 4, 2026
arXiv
2604.03750
Reviewed arXiv version
v2

The COLM 2026 OpenReview record has the same title, seven authors and abstract as arXiv v2 (2026-08-06), whose header reads 'Published as a conference paper at COLM 2026' and which the official project calls its camera-ready version. OpenReview’s 2026-07-08 publication timestamp coincides with the project’s acceptance announcement and is not used as a publication date; the October 6–9, 2026 conference and its Poster Session 5 listing are event details. The repository BibTeX still cites arXiv. This is unrelated to the creativity-evaluation paper also named CreBench.

Sources reviewed

Source review only; experiments were not rerun for this collection. Cite the original authors using the linked paper or BibTeX.

Primary sources

  1. https://openreview.net/forum?id=tLYdf8hWJB
  2. https://colm.cc/Conferences/2026/AcceptedPapers
  3. https://colm.cc/Conferences/2026
  4. https://arxiv.org/abs/2604.03750
  5. https://arxiv.org/html/2604.03750v2
  6. https://arxiv.org/pdf/2604.03750v2
  7. https://github.com/wangyu-ovo/CREBench
  8. https://jams-zhou-james.github.io/CREBench/