AwesomeREA.org

2024 · Conference paper

Assemblage: Automatic Binary Dataset Construction for Machine Learning

Chang Liu; Rebecca Saul; Yihao Sun; Edward Raff; Maya Fuchs; Townsend Southard Pantano; James Holt; Kristopher Micinski

Assemblage is a distributed system that crawls GitHub and vcpkg, builds C/C++ projects under many compiler configurations, and records source-to-binary mappings such as function addresses, source code, comments and PDB or DWARF debug information. Run on AWS for about a year, it produced 890k Windows PE binaries built with Visual Studio 2015–2022 across optimization levels, architectures and Debug/Release modes, plus 428k Linux ELF binaries from GCC and Clang, in 29 configurations. Case studies use the data for compiler provenance with PassTell, function similarity with a graph neural network and jTrans, and function-boundary identification comparing XDA with IDA.

Scope and limitations

The binaries are builds of open-source GitHub and vcpkg projects, many of them small student projects or utilities, so they do not represent commercial, packed or obfuscated software. Assemblage does not independently scan for malware; the authors rely on GitHub’s policy against malware and plan scanning for a later version. The reported build success rate was about 30% on Windows and 7% on Linux. Public datasets include only repositories with permissive licenses, so the downloadable copy is a subset of the paper’s corpus, and repository links in the metadata tie binaries to authors’ GitHub profiles. The case studies retrain existing models under their original protocols with minimal changes; they show Windows/Linux generalization gaps, such as jTrans MRR of 0.52 on Windows against 0.83 on Linux, rather than new analysis methods.

Why it fits this collection

Adds a dataset paper for Windows PE binaries with source and debug mappings, complementing the Linux ELF function corpora used by decompilation research in this section.

Publication and review details

Venue
NeurIPS 2024 Datasets and Benchmarks
Status
Conference paper
Publication / posting date
2024 (exact date not verified)
Date source
The official NeurIPS proceedings and publisher-deposited Crossref metadata identify publication year 2024; Crossref provides no finer publication date. The first arXiv submission date is recorded separately.
First arXiv submission
May 7, 2024
arXiv
2405.03991
Reviewed arXiv version
v2
DOI
10.52202/079017-1871

Crossref and the NeurIPS proceedings page give the eight-author order, volume 37 and pages 58698–58715; Crossref records only the year. The proceedings page’s 2024-12-16 citation date also appears on another NeurIPS 2024 paper page, so year precision is kept. Reviewed arXiv v2 (2024-11-02), which carries the NeurIPS comment; v1 dates from 2024-05-07. At the reviewed f8ac584 commit (2026-08-08), the repository README says the code is MIT-licensed and the published dataset is a permissively licensed subset; pyproject.toml declares MIT, but there is no LICENSE file and GitHub detects none. The README also says the codebase was refactored in July 2026 and extended with a Rust worker, that Windows/MSVC builds sit under backend/assemblage/legacy/ and need a Windows host, and that a pre-refactor branch keeps the earlier code. The Hugging Face cards for changliu8541/Assemblage_PE, whose public copy has 91k Windows PE binaries, and Assemblage_LinuxELF declare license “other” and ask users to follow each source repository’s license. The repository’s separate DeepHistory corpus is not covered by this record. No binaries or datasets were downloaded, and no code or experiments were run.

Sources reviewed

Source review only; experiments were not rerun for this collection. Cite the original authors using the linked paper or BibTeX.

Report an outdated entry Suggest a translation fix

Primary sources

  1. https://proceedings.neurips.cc/paper_files/paper/2024/hash/6bbefc73a187dd42e0dc065b4e7a0615-Abstract-Datasets_and_Benchmarks_Track.html
  2. https://proceedings.neurips.cc/paper_files/paper/2024/file/6bbefc73a187dd42e0dc065b4e7a0615-Paper-Datasets_and_Benchmarks_Track.pdf
  3. https://api.crossref.org/works/10.52202/079017-1871
  4. https://arxiv.org/abs/2405.03991v2
  5. https://arxiv.org/pdf/2405.03991v2
  6. README.md GitHub · Assemblage-Dataset/Assemblage · Revision f8ac584
  7. pyproject.toml GitHub · Assemblage-Dataset/Assemblage · Revision f8ac584
  8. https://huggingface.co/datasets/changliu8541/Assemblage_PE/blob/19a8f947f5a04d03edea78e61790226b55f92ac2/README.md
  9. https://huggingface.co/datasets/changliu8541/Assemblage_LinuxELF/blob/5d58b08b500ea279a9880f80fd4b217e1897035a/README.md