2024 · Conference paper
StackSight: Unveiling WebAssembly through Large Language Models and Neurosymbolic Chain-of-Thought Decompilation
StackSight decompiles WebAssembly text-format functions into C++ with LLM prompting. A static analysis annotates each WAT instruction with the virtual-stack state, using symbolic notation where branches diverge; a one-shot chain-of-thought prompt then asks the model to recover variable meanings, summarize the function and write C++. With GPT-4 the share of outputs passing all benchmark tests rises from 19.7% to 36.5% on HumanEval-X and from 42.4% to 59.6% on MBXP relative to one-shot prompting; GPT-3.5 and Code Llama-7B-Instruct are also evaluated.
Scope and limitations
Inputs are short exercise functions from the C++ splits of HumanEval-X (161 of 164 compiled) and MBXP (773 of 974), built as Emscripten 3.1.46 side modules at -Oz with function names kept by --profiling-funcs and converted with wasm2wat 1.0.33; minified, obfuscated or whole-application WebAssembly is not evaluated. Correctness is the share of outputs that compile and pass the benchmarks' test cases after a test oracle renames functions, inserts tests and adds namespace directives, with manual pass-by-value edits. Passing these tests is not proof of equivalence with the original module. Summaries are scored by BERTScore and SBERT; the user study had 15 participants on 20 snippets per dataset. wasm2c and an earlier neural decompiler were excluded for producing no compilable output, and results depend on specific hosted model versions.
Why it fits this collection
Adds LLM-based WebAssembly decompilation guided by static stack analysis, covering the bytecode format shipped in web applications alongside the native-code decompilation papers in this section.
Publication and review details
- Venue
- ICML 2024
- Status
- Conference paper
- Publication / posting date
- July 8, 2024
- Date source
- PMLR states that Volume 235 was published on 08 July 2024, and the paper page gives the same citation date. The July 21–27 conference dates and the June 2024 arXiv submission are not used as the publication date.
- First arXiv submission
- June 7, 2024
- arXiv
- 2406.04568
- Reviewed arXiv version
- v1
PMLR lists Proceedings of the 41st International Conference on Machine Learning, PMLR 235, pages 13010–13028; no DOI is given. arXiv has a single version, v1 of 2024-06-07, whose comment cites the ICML 2024 proceedings. The models are gpt-3.5-turbo-1106, gpt-4-0125-preview and Code Llama-7b-Instruct; GPT-4 was compared only with one-shot prompting, while the GPT-3.5 and Code Llama baselines also use 3-, 5- and 10-shot prompts. The appendix gives the emcc and wasm2wat commands and the prompts. The paper links third-party tools such as WABT and Emscripten but no author code, prompts repository or evaluation harness, and none was found. No code or experiments were run.
Sources reviewed
Source review only; experiments were not rerun for this collection. Cite the original authors using the linked paper or BibTeX.