2025 · Conference paper
Diffusion Models Are Real-Time Game Engines
GameNGen learns to simulate DOOM interactively with a diffusion model instead of game code. A PPO agent first plays the game in ViZDoom, and its recorded trajectories, from random to trained play, become the training data. Stable Diffusion v1.4 is then fine-tuned to predict the next frame from the last 64 frames and actions, with noise added to context frames to keep long autoregressive rollouts stable and a fine-tuned latent decoder for detail. With 4 sampling steps it runs at 20 frames per second on one TPU-v5. The authors report next-frame PSNR of 29.43 and human raters picking the real game in 58–60% of short clips.
Scope and limitations
A neural approximation trained on recorded gameplay of an existing game; it recovers no game logic, source code, assets or state representation and is not an exact simulation. The model sees just over 3 seconds of history, so state outside that window survives only through on-screen cues and learned heuristics, which can be wrong; for example, repeated shooting may spawn enemies. Locations and interactions the agent never explored are simulated poorly. Training used 128 TPU-v5e devices and 70M agent frames at 320×240, plus an environment-specific reward function. PSNR is a teacher-forced single-frame measure on 2,048 held-out trajectories from 5 levels; autoregressive rollouts diverge from ground truth after a few steps. Human evaluation used 10 raters on 1.6- and 3.2-second clips and 150 later 3-second clips; the authors say they could often spot the simulation within seconds. Only DOOM was tested, and the model cannot readily create new games.
Why it fits this collection
Adds learned, non-symbolic reconstruction of a game's interactive behavior from gameplay, contrasting with the collection's code-based world models and visual game replication research.
Publication and review details
- Venue
- ICLR 2025
- Status
- Conference paper
- Publication / posting date
- 2025 (exact date not verified)
- Date source
- Publication year stated by the official ICLR proceedings; no finer publication date is asserted. The proceedings page's citation date of 2025-05-01 is not adopted, and the arXiv dates are not substituted.
- First arXiv submission
- August 27, 2024
- arXiv
- 2408.14837
- Reviewed arXiv version
- v2
Authors and title follow the ICLR 2025 proceedings page and PDF, which list pages 73754–73776 and print “Published as a conference paper at ICLR 2025”; no DOI is listed. arXiv has two versions with the same title and authors: v1 of 2024-08-27 and v2 of 2025-04-24, whose comment reads “ICLR 2025”. The OpenReview forum P8pqeEkn1H could not be retrieved automatically because of a challenge page. The project page links the paper and videos but no code or weights, and no author-provided code release was found; unofficial reimplementations are not linked. The paper's reproducibility statement relies on open Stable Diffusion v1.4 weights, the open-source ViZDoom environment and documented training settings. Its reported 50 frames per second uses a distilled single-step model with some quality loss. Sources were read only; nothing was trained or run.
Sources reviewed
Source review only; experiments were not rerun for this collection. Cite the original authors using the linked paper or BibTeX.