I think it's fascinating to see sample images generated from these models side by side. It really does give a sense of how fast this field has progressed. In just five years, we've gone from blurry, grayscale pixel arrays that vaguely resemble human faces to thispersondoesnotexist, which can easily fool most people on first glance.
Apart from image samples, I've also included links to papers, code, and other learning resources for each model. So this article could be an excellent place to start if you're a beginner looking to catch up with the latest GAN research.
I hope you enjoy it!
The self attention mechanisms caught my eyes. Going to look into implementing something like that for a toy dataset. Thanks for the inspiration
I know people have been having trouble adapting these kinds of
generative techniques to text. Do you know of anyone making
interesting progress there?
It is difficult to make progress in the field of generative methods
with text alone - it takes effort and creativity to get a generative
system working. A big part of our research focuses on generating
sequences which correspond to handwritten data, and to improving on
that method we have developed generative techniques which allow us to
generate a large range of novel sequences. Our work is still small,
but we are not stopping there, in fact our next major research project
is to generate novel sequences for novel languages.
What I see is that we are still in an early stage when it comes to
the technology used in generative methods to create new words, but I
suspect this is due to a combination of factors. First and foremost is
the fact that the techniques we've developed to generate novel
sequences are highly specialized in a particular kind of context -
we are not going to create random numbers or sequences because that
just doesn't work. Generating a word, for example, uses very specific
computational principles and can only be done if you are aware of the
context in which it is being generated (or "determined" as the
linguists would say). Even so, the general principle has been around
so long, that one could quite easily create several different methods
GANs simply try to replicate a set of features - you can think of this as images or text. Variations in the GAN designs will be present, but the general principles are the same.
I like the mix of images and explanations.
Your point stands though, that it’s obviously not overfitting, and no scientist would publish a result that was just overfitting face generation.
Wikipedia GAN history (and talk) indicates it's not quite as clear cut as your framing, and this answer below on that subject from somebody who blogged the idea several years previously demonstrates there is often a hinterland of discussion that gives rise to the (genuinely) independent ideas of Goodfellow et al.
In this case the central idea doesn't seem to have been completely original to Goodfellow, though the credit is his for fully pursuing it to implementation in the current model.
Note: the linked answer - while in the context of the well known Schmidhuber can of worms, is actually the more obscure and very polite challenge from Niemitalo (who is mentioned in the Wikipedia history). The point stands though, regardless of actor.
The article explains this very accessibly.
I don't know if everybody knows that. Lot of people believe Hillary ran a child-sex operation and should be locked up. They chant it. Trump has lied 10,000 times and still a lot of people believe him. Surely they would believe a fake video as well, many of them.