Does anyone know of work towards automatic 3D model generation and, ideally, rigging for animation? It seems like the future, but it also seems like it’s not getting the attention it should.
I think it might take the form of artist augmentation rather than replacement. Think of it like a new sort of power tool. And it's true there are a lot of steps for triple-A engines, but most gamedevs (like myself, once) simply don't have any art assets, and need them greatly; a tool similar to MSPaint for models might enable the next Notch to emerge more easily.
On the other side of it, from the triple-A point of view, automating even one of the steps you mention will be a huge time saver. Artists tend to worry about machines replacing them, but I think it will only unlock more, better art. You'll be able to do so much. So there is a big financial incentive to get this right; the first one to capture the market will be in a strong position.
> [...] most gamedevs (like myself, once) simply don't have any art assets, and need them greatly;
But isn't that exactly why market places like the Unity Asset Store, TurboSquid, and UE Marketplace exist?
The asset stores for Unity and UE in particular offer assets that are ready for use with these frameworks and can also be easily modified if need be.
I rather think the problem is the expectation to get high quality assets for free, which just isn't a realistic assumption, unless you're willing to team up with an artist who is willing to volunteer their time.
It's also funny you mentioned Notch, as Minecraft and its aesthetic are a direct result of lack of artistic talent, proving that you don't need great looking 3d models at all to create the best selling video game in history...
I agree, though that ML/AI tooling for asset generation would be a major improvement for the gaming (and film!) industry.
I've been looking for that for the last couple of days and couldn't find much. Great resources linked as comments.
I had found that one myself: https://github.com/mishig25/3d-posenet
This almost touches the field of A-Life, rather than neural methods for 3D geometry. There's OpenAI's humanoid walker, that taught itself bi-pedal motion. But as far as I know there is no MeshNet or ShaderNet, analogous to Imagenet. The research data sets like ShapeNet consist of CAD models for object detection and scene understanding.
Chimera Painter demo itself is unbelievably fun. When AI Dungeon and GPT-3 launched the observation was made that we are all "prompt engineers" now. It's the quality of the input that differentiates final results ;)
Since your comment became a list of AI links, here is one from me. I'm not an expert in this area, but it looked like a very interesting project in the space of artificial worlds creation: https://www.prometheanai.com/
