AwesomeREA.org

2025 · Conference paper

PoE-World: Compositional World Modeling with Products of Programmatic Experts

Wasu Top Piriyakulkij; Yichao Liang; Hao Tang; Adrian Weller; Marta Kryven; Kevin Ellis

PoE-World represents an Atari world model as an exponentially weighted product of many small Python programs, or programmatic experts, each synthesized by an LLM to explain one aspect of the observed dynamics; learned weights and optional synthesized hard constraints combine them. From object lists that OCAtari parses out of a demonstration of fewer than 1,000 frames, the authors learn stochastic, partially observed models of Pong and Montezuma's Revenge and use them in a planning agent, including on rearranged Pong-Alt and Montezuma's Revenge-Alt variants that have no demonstrations.

Scope and limitations

Models consume symbolic object lists, not pixels, and OCAtari had to be patched manually for each game, so evaluation covers only Pong and Montezuma's Revenge plus two author-built variants. The learned programs approximate observed dynamics and are not the games' original code: next-observation prediction accuracy on 1,000 random test frames is 0.06 for Pong and 0.31 for Montezuma's Revenge, and object-attribute accuracy is 0.66 and 0.76. Synthesized experts build on manually written helper classes and contact functions and were generated with gpt-4o-2024-08-06. Exploration and reward learning are outside the method; agent scores use 5 seeds and at most 3k training steps after the demonstration, and the planner reached the key in 5 of 9 Montezuma's Revenge tries.

Why it fits this collection

Adds compositional program synthesis of executable game-dynamics models for non-grid Atari games, extending the code-world-model line beyond grid, text and puzzle environments.

Publication and review details

Venue
NeurIPS 2025
Status
Conference paper
Publication / posting date
2025 (exact date not verified)
Date source
The publisher-deposited Crossref record for the NeurIPS 2025 proceedings gives 2025; no publication month or day is asserted. Neither the proceedings page's 2026-04-23 citation date nor the November–December 2025 conference dates are used as a publication date.
First arXiv submission
May 16, 2025
arXiv
2505.10819
Reviewed arXiv version
v4
DOI
10.52202/085713-0896

Title, author order and DOI agree across the NeurIPS proceedings page, the Crossref deposit, the final PDF and arXiv. The proceedings metadata and Crossref give the first author as Top Piriyakulkij; this record uses Wasu Top Piriyakulkij as printed in the final PDF, on arXiv and in the repository README. The proceedings page lists pages 26609–26638 while Crossref lists 30165–30194, so no page range is recorded. arXiv v4 (2025-11-19) was reviewed. The README headline calls the paper a NeurIPS 2025 Spotlight, which the proceedings page does not state, and its arXiv badge text reads 2401.02739 while linking to 2505.10819. The MIT-licensed repository snapshot at b499fb1 (2026-02-05) targets Python 3.10, pins most dependencies, and uses MIT-licensed submodules for OpenAI API access and a patched OCAtari fork; runs need a user-supplied OpenAI API key, and the Atari environments come through Gymnasium and the Arcade Learning Environment under their own terms. The snapshot includes seed-0 learned world models for Pong and Montezuma's Revenge and a Pitfall run script that the paper does not evaluate. No code, model calls or environments were run.

Sources reviewed

Source review only; experiments were not rerun for this collection. Cite the original authors using the linked paper or BibTeX.

Report an outdated entry Suggest a translation fix

Primary sources

  1. https://papers.nips.cc/paper_files/paper/2025/hash/262dd62fd1bbb30d6a6b4d578f5e65ff-Abstract-Conference.html
  2. https://proceedings.neurips.cc/paper_files/paper/2025/file/262dd62fd1bbb30d6a6b4d578f5e65ff-Paper-Conference.pdf
  3. https://doi.org/10.52202/085713-0896
  4. https://api.crossref.org/works/10.52202/085713-0896
  5. https://arxiv.org/abs/2505.10819
  6. https://arxiv.org/pdf/2505.10819v4
  7. https://topwasu.github.io/poe-world
  8. README.md GitHub · topwasu/poe-world · Revision b499fb1
  9. LICENSE GitHub · topwasu/poe-world · Revision b499fb1
  10. requirements.txt GitHub · topwasu/poe-world · Revision b499fb1
  11. .gitmodules GitHub · topwasu/poe-world · Revision b499fb1
  12. run_pitfall.sh GitHub · topwasu/poe-world · Revision b499fb1