Funny thing, the more people comment that it doesn't load, the more trending the post is and more traffic it gets and the loading wheel spins even more and more.
Oh my GOD, the horror :) I'll leave it alone now, the dev server loaded for me a bit. Some observations from one who tried to do weird things with it…
This generator expects realistic proportions, in a big way. If you try to create giant anime eyes you're liable to find they have become mouths, even though you colorcoded them to be eyes. Auxiliary eyes seem more do-able. Mouths that are too small and eye-shaped become eyes: ambiguous shapes become who knows what. Fun ensues :)
The generator's doing interesting things with the overall contour of the face. It'll shade pretty realistically for whatever you make; the thing is, you have to be a good artist to understand what the REAL shape of, say, a nose is. If you think 'triangle' or any sort of abstraction you get monster creations: if you go for a roughed-in representation of ALL the real features of a nose covering the real areas you get something pretty realistic. That shape will look very strange to a non-artist because we don't think of a nose as having all those features.
It seems like the colors aren't nearly as important as the shapes and sizes and positionings are. That might be a useful observation. Right now, it's monster fuel but there's also an interesting 'can you REALLY draw a nose, or do you only think you know what a nose is shaped like?' factor. Again, I can't tell if the colors even mean anything but you've got to get REAL close to the actual dimensions of facial features (and scale!) to get remotely photoreal results.
Of course the monstrosities are perhaps even more fun… because the generator will try its best to contextualise them against the background of what a real face would be shaped like to account for those departures. I would enjoy a slower and more painstaking version of this along the same lines.
Thanks for the nightmares… and insights! I'll take my very creditable photoreal nose and be off now :)
My drawing skills are atrocious, I just gave up and started drawing parts where they didn't belong. Experimenting was fun but I kinda regret doing that.
I've been exploring how machine learning can be incorporated into the artistic process or for content creation. This experiment generates faces from segment maps. We have a pipeline setup that could be applied to landscapes, architecture, cars... etc. It feels like to me AI assisted content creation is going to become commonplace.
Didn't get a chance to see the result (due to hug of death), but I think it could be improved with a "clear" function in order to start from a blank canvas, as it's quite tedious to try and remove the existing guide picture.
Would love for someone with some patience and artistic skill to draw Marge Simpson (or the other Simpson family members) and see what pops out. preemptive shudder
1. Pushing now as an option to the vertical [...] menu (give it 10 - 20 mins)
2. Long press on the brush icon for brush menu. Can set brush size numerically or use [-] [+], also can use keys "[" and "]". We will put preset sizes on the list.
3. Does currently generate on mouse up, but pushing fix to wait 3s before generation.
Yeah for sure. I'll write up a blog post, but here is my high-level overview for now. Ollie (ML engineer) from our team will fill in the technical details.
This works by using a “segmentation map”. Deep Learning models are really good at performing per pixel labelling.
You’ve probably seen the Remove Background tools that classify parts of the image either as background or foreground to make an alpha mask. But we don’t have to stop at two classes. We can classify an image with much more detail such as what parts contain an Eye, Ear, Nose, Mouth… etc. Passing an image of a face to a ML classification model can return a so-called “Segmentation Map”, which assigns a unique color to each class of facial feature. This kind of object detection is commonly used in computer vision for manufacturing assembly lines, but we want to use it for art...
Now this is where it gets interesting, we can play this in reverse, give the ML model a segmentation map, and use a GAN to generate the most convincing image of a face that it can (given the dataset it was trained on).
The really cool thing is that this isn’t limited to just faces, you can do this with landscapes, buildings, cars, cats you name it. Anything where you can train a model by classifying the segments or "features" of an image.
I'm getting 504s from the `v1/image/ml/segment-to-face` endpoint with the message "The upstream server is timing out". There were initially auth redirect messages in the console too, but they go away once you log in (as you'd expect).
To the user that clicks only once, there's little value to show a random initial image and compute the corresponding result.
You should prob just always show the same image and cache the result, so the site can always load. Or randomize in a fixed set of 10 if you really like the randomization part.
It looks to be similar to Nvidia Canvas - there's a good demo of that online.
Basically the user paints where they want a feature to be using a colour. The AI replaces the colour with a feature and makes sure that any features touching eachother blend together beautifully to create a believable human face.
Yeah, it's Nvidia Canvas for faces, except not trying to sell you RTXs. Instead we're trying (when we load balance) to offer a realtime service through the browser. I have some more background in a comment here https://www.producthunt.com/posts/face-maker-ai
It's trained with a mixture of publicly available datasets of faces. The final model is several gigabyes in size, so it's fairly large. Actually that's one of the reasons we've made this tool - to test our infrastructure with larger generative models.
The model is learning the features that make a convincing face, and generating a synthetic face from those (controlled by the segment map), similar to Nvidia's GauGAN.
The power in generative models lies in being able to flexibly generate images that belong convincingly to a set (e.g. faces, landscapes), but that are not actually in the input dataset. E.g. you can make images that look like faces, but that don't belong to any real individual.