AwesomeREA.org

Learning resource

Frontkick vs Rikugan: a CREBench agent comparison

Matthieu Breuil’s Synacktiv article explains tool-calling and code-executing agent designs and describes Frontkick, a CodeAct agent that runs generated Python inside Binary Ninja. It compares Frontkick with the open-source Rikugan plugin on two CREBench cryptographic binary tasks, then attributes most of Frontkick’s advantage to a prompt that listed installed Python libraries rather than to its architecture. It is a small evaluation write-up with per-run results, not a released tool or a general benchmark finding.

Requirements and limitations

  • The article on Synacktiv’s publications site is dated 13/10/2026. It describes Frontkick as a CodeAct agent that executes generated Python against Binary Ninja’s API, with a recipe-retrieval corpus, a single-level subagent and truncation of older turns; no download, repository or license for Frontkick is linked. Rikugan is an MIT-licensed community agent plugin for IDA Pro and Binary Ninja built on schema-validated tools, while Vector 35’s closed-source Sidekick was excluded because its internals are not accessible.
  • The comparison used Claude Sonnet 5.5 with CREBench’s AES-128-CBC and ChaCha20 cases in the hardcode_plain/O0 setting, supplied as stripped binaries, with four runs per case for each agent and a 1M-token budget. Each run had to identify the algorithm and key and reimplement decryption to recover the flag. The article does not state the Binary Ninja, Rikugan or Frontkick versions or publish the run transcripts it inspected.
  • In the published run table Frontkick fully solves six of eight runs and Rikugan none; the means are about 86 versus 31 out of 100 and about 0.60M versus 0.95M tokens, which the article summarizes as roughly half the tokens and nearly three times the score. On AES-128-CBC, Rikugan repeatedly tried a Python crypto package that was not installed and fell back to a manual implementation, whereas Frontkick’s prompt listed the installed libraries. The author therefore calls the outcome a draw driven by prompt design rather than evidence for CodeAct, and proposes a rerun with aligned prompts.
  • Two tasks, one model and eight runs per agent cannot support general conclusions about either agent, about tool-calling versus code-executing designs, or about other CREBench settings. The separate CREBench paper record covers the benchmark itself; the article reports no REA use and no reconstruction. This collection read the article, Rikugan’s README and the CREBench README without running either agent or the benchmark.

About this entry

Type
Learning resource
Category
Learning resources
Relationship
Learning resource
Targets
Binary

Sources reviewed

The description was reviewed against primary sources. This does not establish runtime testing, security auditing or verified compatibility.

Primary sources

  1. https://www.synacktiv.com/en/publications/ufc-666-frontkick-vs-rikugan
  2. README.md GitHub · buzzer-re/Rikugan · Revision a2ba669
  3. LICENSE GitHub · buzzer-re/Rikugan · Revision a2ba669
  4. README.md GitHub · wangyu-ovo/CREBench · Revision 9fc4016