On: Diffusion models are real-time game engines

[darrinm]

So… is it interactive? Playable? Or just generating a video of gameplay?

[vunderba → darrinm]

From the article: We present GameNGen, the first game engine powered entirely by a neural model that enables real-time interaction with a complex environment over long trajectories at high quality.

The demo is actual gameplay at ~20 FPS.

[darrinm → vunderba]

It confused me that their stated evaluations by humans are comparing video clips rather than evaluating game play.

[furyofantares → darrinm]

Short clips are the only way a human will make any errors determining which is which.

[darrinm → furyofantares]

More relevant is if by _playing_ it they couldn’t tell which is which.

[Jensson → darrinm]

They obviously can within seconds, so it wouldn't be a result. Being able to generate gameplay that looks right even if it doesn't play right is one step.