GameNGen is a fully neural model-driven game engine capable of real-time interaction with complex environments while maintaining high quality over extended trajectories. It can interactively simulate the classic game 'DOOM' at over 20 frames per second, with its next-frame prediction achieving a PSNR of 29.4, comparable to lossy JPEG compression. Human evaluators only slightly outperform random chance in distinguishing between game clips and simulated clips. GameNGen is trained through two phases: (1) an RL-agent learns to play the game and records the actions and observations from the training sessions, which become the training data for the generative model; (2) a diffusion model is trained to predict the next frame, conditioned on the past actions and observation sequences. Conditional enhancement allows for stable autoregressive generation over long trajectories.