2022 · Conference paper
How Machine Learning Is Solving the Binary Function Similarity Problem
Measures binary function similarity research by re-implementing ten representative approaches from the security, programming-language and machine-learning communities in one IDA Pro-based framework, from fuzzy hashing (Catalog1, FunctionSimSearch) to graph neural network, NLP-encoder and transformer models. Two new datasets support six tasks that vary optimization, compiler, bitness and architecture, scored by AUC and ranking metrics, plus a firmware vulnerability-search case. The authors report that fuzzy hashing fails in harder cross-architecture settings, GNN-based models perform best in almost all tasks, and many recent models score similarly on common data.
Scope and limitations
A 2022 comparison of the authors’ re-implementations, not of the original papers’ code or reported numbers, which the authors did not try to reproduce; CodeCMR was run by its own authors, and Trex was tested only on Dataset-2 with its authors’ model. Dynamic-analysis, symbolic-execution and other slow direct-comparison approaches were excluded by design, and LLM-based similarity models are not covered. Dataset-1 compiles seven open-source projects (ClamAV, Curl, Nmap, OpenSSL, Unrar, Z3, Zlib) with GCC and Clang for x86-64, ARM and MIPS in 32 and 64 bit at O0–O3 and Os with function inlining disabled, and drops functions under five basic blocks. Dataset-2 reuses ten GCC 7.5 projects released by the Trex authors. Feature extraction depends on IDA Pro 7.3; BinaryAI covered only part of the data because a recent Hex-Rays MIPS decompiler license was unavailable. The firmware case searches ten OpenSSL functions in two images. Obfuscated code is not evaluated, and the results are not a current ranking.
Why it fits this collection
Adds a common-framework measurement of binary function matching, which reverse engineers use to transfer labels from known functions to stripped code, complementing single-model records such as jTrans and BinaryAI.
Publication and review details
- Venue
- USENIX Security 2022
- Status
- Conference paper
- Publication / posting date
- August 2022
- Date source
- Publication month from the official USENIX BibTeX (month = aug) and the final PDF cover for the 31st USENIX Security Symposium, August 10–12, 2022; no exact publication day is asserted.
No paper DOI or arXiv identifier was verified; the record uses the reviewed USENIX Security presentation-page and sec22-marcelli.pdf profile. The USENIX BibTeX gives the 31st USENIX Security Symposium, pages 2099–2116, USENIX Association and month August; the presentation page also links a prepublication PDF and slides. The repository README lists errata for a Section 3.2 statement on BinDiff and Diaphora cross-architecture support and for the Figure 2 captions. At reviewed commit f0da093 (2024-05-15), repository code is MIT-licensed, but the README lists GPL-3.0 (Catalog1, SAFE, IDA CodeCMR plugin), LGPL-2.1 (Gensim-based Asm2vec), Apache-2.0 (FunctionSimSearch, GGSNN/GMN) and CC BY-NC-SA 4.0 (GNN-s2v) components; compiled binaries keep their own licenses. Binaries (about 15 GB) and extracted features (about 38 GB) download from Google Drive, and generating new IDBs or features requires IDA Pro. Only documentation files were read; no datasets were downloaded and no code or experiments were run.
Sources reviewed
Source review only; experiments were not rerun for this collection. Cite the original authors using the linked paper or BibTeX.
Primary sources
- https://www.usenix.org/conference/usenixsecurity22/presentation/marcelli
- https://www.usenix.org/system/files/sec22-marcelli.pdf
- README.md GitHub · Cisco-Talos/binary_function_similarity · Revision f0da093
- LICENSE GitHub · Cisco-Talos/binary_function_similarity · Revision f0da093
- Binaries/README.md GitHub · Cisco-Talos/binary_function_similarity · Revision f0da093
- DBs/README.md GitHub · Cisco-Talos/binary_function_similarity · Revision f0da093
- IDA_scripts/README.md GitHub · Cisco-Talos/binary_function_similarity · Revision f0da093