Its incredible I can check out my small home town on Google Earth now entirely in 3d (which wasnt the case just a few months ago). Yet the trees/cars are still these blocky low resolution things sticking out of the ground. Imagine Google Earth procedurally generating a high resolution mesh as I zoomed in. Train it with high resolution photogrammetry of other similar locations for the ground truth - and let me zoom in endless in their VR app
With that said I bet this would choke on lots of actual Minecraft worlds, because people often build things using blocks where the semantics get thrown completely out the window in favor of aesthetics. Want a big mural on the wall? You're going to be building the wall itself out of differently-colored blocks of wool
Maybe they'll solve that part one day :)
Edit: That said, it could choke in some really interesting ways...
If you ignore the implementation, its basically a procedural texturing technique? Those are widely used now.
If you're talking about a real time post effect, it would probably be a bit too slow for a few more years.
If you count SLAM techniques that label camera feeds for AR games, those are very close but I dont think most run at a full framerate.
The paper in question can apparently render at 2K and 30fps though. Or at least that’s what the videos claim.
Even real-time today it should be doable if you create your game with that in mind. You really don't need to generate all the textures, just a compact representation of the level which is to be rendered normally after the fact.
We've had procedurally generated worlds for a long time, but this would take it from roguelike top-down or isometric to immersive fps.
If you mean asset / level generation. Then yes. It is the next step in procedural generation imo.
NVidia also released their RTXDI SDK for global illumination at a scale of millions of dynamic lights in real time. Combined with GANCraft, anyone could become a world class environmental artist using only Pixel Art tools.
See the OG transfer algorithm called "Image Analogies" from decades before the GAN boom:
If papers did that, they'd be a thousand pages long. The target audience is people intimately familiar with the state of the art.
And wouldn't it be possible to simply take the original minecraft map as a height map and texture map and then regenerate a new world with the original world data and more advanced post processing? You could interpolate and randomize more detail into the scene than you started with.
Not to minimize the awesomeness of that... doing it stably in 3D while moving the camera is the point of this paper, and is amazing.
But it’s not really adding detail beyond “these are the kinds of pixels that grass has and the AI figured out we can put them in this arrangement without making things jumpy”
The renderer seems to be adding some resolution, smoothing, and mipmapping. Shaders can do the same thing, and in real time.
before: old streetview pre bikelanes
after: streetview with new bike lanes
profit: now you can see what any town would look like with complete streets. I call it Complete Street View.
Please do implement. Of course it would be dreamlike, this is a strength as you wouldn’t want the gan to make design recommendations, just a plausible feel.
Myself and my son absolutely love it and spend months in this pandemic deep in minecraft worlds
Your complaint could be made about just about any new technology. It's usually worse than what came before it at first, but the value is in the potential to eventually become better than what came before it.