AwesomeREA.org

2025 · Conference paper

Nova: Generative Language Models for Assembly Code with Hierarchical Attention and Contrastive Learning

Nan Jiang; Chengxiao Wang; Kevin Liu; Xiangzhe Xu; Lin Tan; Xiangyu Zhang; Petr Babkin

Nova is a generative LLM for x86-64 assembly, trained from DeepSeek-Coder checkpoints on stripped, normalized gcc -O0 to -O3 assembly built from AnghaBench and The Stack. It adds hierarchical attention that builds per-instruction summaries, and contrastive objectives that keep a function's source and O0–O3 assembly close while ordering them by optimization level. Fine-tuned for decompilation, Nova-1.3B and Nova-6.7B reach 25.17% and 34.36% average Pass@1 on HumanEval-Decompile, 14.84 and 12.41 points above same-size LLM4Decompile; separately fine-tuned models report the highest average Recall@1 for O0-to-O3 binary code similarity on six real-world projects.

Scope and limitations

Decompilation is evaluated function by function on the 164 HumanEval-Decompile C problems, each compiled for x86-64 at O0–O3, by running 20 samples per function against the problems' test cases; Pass@k on these tests is neither whole-program nor equivalence evidence. Baselines other than LLM4Decompile and Nova were prompted with three-shot examples rather than fine-tuned. Similarity results cover O0 queries against O3 candidates in pools of 50–500 functions from Binutils, Curl, ImageMagick, SQLite, OpenSSL and PuTTY, after fine-tuning on BinaryCorp-3M. Decompilation fine-tuning used a 2.16M-pair sample, the README says only 50K files came from The Stack, and the paper limits Nova to x86-64 assembly and 1.3B and 6.7B models. Inputs require the paper's assembly normalization.

Why it fits this collection

Adds an assembly-specialized foundation model evaluated on both decompilation and binary code similarity, complementing the decompilation-specific LLM papers in this section.

Publication and review details

Venue
ICLR 2025
Status
Conference paper
Publication / posting date
2025 (exact date not verified)
Date source
Publication year stated by the official ICLR proceedings; no finer publication date is asserted. The proceedings page's citation date of 2025-05-01 is not adopted, and the arXiv dates are not substituted.
First arXiv submission
November 22, 2023
arXiv
2311.13721
Reviewed arXiv version
v7

Authors follow the proceedings, which list pages 95905–95926. arXiv has seven versions: v1 of 2023-11-22 was titled 'Nova+: Generative Language Models for Binaries' (written Nova$^+$ in the arXiv metadata) and listed six authors, without Petr Babkin; the checked later versions (v3, v5, v7) use the final title, and v7 of 2025-11-11 states ICLR 2025 publication. The repository README's abstract gives a relative decompilation gain of up to 146.54%, while the paper reports absolute percentage points. Licence signals are listed separately: the reviewed 2026-01-04 repository commit carries a BSD 3-Clause LICENSE, and the Hugging Face cards for nova-1.3b, nova-6.7b and their -bcr decompilation variants are tagged bsd-3-clause-clear; the models start from DeepSeek-Coder checkpoints, whose terms are not assessed here. requirements.txt pins torch 2.3.0, transformers 4.40.2, vllm 0.5.0 and deepspeed 0.14.4 for a Python 3.10 environment, and training examples use four GPUs. No models were downloaded and no code or experiments were run.

Sources reviewed

Source review only; experiments were not rerun for this collection. Cite the original authors using the linked paper or BibTeX.

Primary sources

  1. https://proceedings.iclr.cc/paper_files/paper/2025/hash/ef283d62b4bce30854a8d4827f331229-Abstract-Conference.html
  2. https://proceedings.iclr.cc/paper_files/paper/2025/file/ef283d62b4bce30854a8d4827f331229-Paper-Conference.pdf
  3. https://arxiv.org/abs/2311.13721
  4. https://arxiv.org/abs/2311.13721v1
  5. README.md GitHub · lt-asset/nova · Revision 6dfaf43
  6. LICENSE GitHub · lt-asset/nova · Revision 6dfaf43
  7. requirements.txt GitHub · lt-asset/nova · Revision 6dfaf43
  8. https://huggingface.co/lt-asset/nova-1.3b/blob/4b4805bac4f13ef8bec678072ef60609ea3b0e77/README.md
  9. https://huggingface.co/lt-asset/nova-6.7b/blob/dd808302b3f5929c29e8940f4e0c0e4cc38a5a63/README.md
  10. https://huggingface.co/lt-asset/nova-1.3b-bcr/blob/9101c192eb22bbb982ebe94830385cedee5b78e0/README.md
  11. https://huggingface.co/lt-asset/nova-6.7b-bcr/blob/69a8ccd089abc902b7ce9b4e17298a887fbe3626/README.md