Failure modes often tell you more about a technology or phenomenon than successes do.
The marketing and promotion will, of course, focus on successes. Failures suggest boundaries, limits, and weaknesses.
There's a similar case for humans. Studies of optical and auditory illusions, cognitive biases, and various forms of pathological behaviours, both individual and group, would be examples of this.
Or any equipment or system. The flight envelope of an aircraft is literally the boundary beyond which its behaviour is unpredictable and/or uncontrollable. For software, limits define the extent to which applications or AI algorithms produce useful results.
How much those limits can be pushed is itself an interesting question. The observations of failures of negation (a map of the world without Europe, a bowl of fruit without apples, a man not running) suggest that DALL-E has something of a "now don't think of an elephant" problem. It's also a problem that's been observed in search contexts --- I am recalling a case where Google and/or Amazon repeatedly failed when requested to find shirts without stripes, consistently presenting striped shirts.
More to the point, it doesn't do semantics at all, only syntax, if you will. It
has learned correlations between images and words but it has no way to map
either the words or the images to their meanings. All it knows is that this
word appears often in correlation with that image. So when it sees the word,
it gives you the image.
What is the word "no" correlated with? Probably, absolutely anything. Most
likely "no" is as probable to appear as the description of an image with apples
as it is to appear as the description of an image without apples. But "apples"
is likely to describe an image with apples, even if it has "no" in it. For
example "An image of apples in a basket with no handle" has "no" and "apples".
So- you get apples, "no" or not.
In a sense, nobody has told it that "no" means "no".
(Also, it doesn't know that "apples" means "apples". "Apples" is only the label
of some sets of pixels in a space. If you scrambled the labels so that "apples"
was correlated with images of elephants... "an image with no apples" would get
you an image with elephants).
One thing I don't like about this article is that it's showing singular examples of Dall-E's failures, when Dall-E really produces 8 possible answers per text prompt (and more if you iterate further). You can cherry pick the worst of the batch and write an article about how bad it is just as, I suppose, you could write an article about the best of each batch and how good it is. However, the fact that multiple drafts are created is similar to the way humans work when doing creative work. So I think it's more instructive to look at the successes. I'm sure if we chose any genius in human history and analyzed their life's work, we'd find tons of failure and false starts and bad attempts. But we don't let those failures detract from their later success. However, in analyzing AI, you can quickly jump to that kind of "it has to be perfect or its fake" analysis and I think it's a flawed way of looking at things.
Okay, so the author had to iterate on a couple of examples. But look at the speed. Most of these examples would take a human a day to a week of labor, for each one. The fact you can rattle off almost any random text prompt like "draw me a walrus in the style of pointillism with a fake looking mustache" and that it can accommodate such requests in one second is absolutely stunning.
The marketing and promotion will, of course, focus on successes. Failures suggest boundaries, limits, and weaknesses.
There's a similar case for humans. Studies of optical and auditory illusions, cognitive biases, and various forms of pathological behaviours, both individual and group, would be examples of this.
Or any equipment or system. The flight envelope of an aircraft is literally the boundary beyond which its behaviour is unpredictable and/or uncontrollable. For software, limits define the extent to which applications or AI algorithms produce useful results.
How much those limits can be pushed is itself an interesting question. The observations of failures of negation (a map of the world without Europe, a bowl of fruit without apples, a man not running) suggest that DALL-E has something of a "now don't think of an elephant" problem. It's also a problem that's been observed in search contexts --- I am recalling a case where Google and/or Amazon repeatedly failed when requested to find shirts without stripes, consistently presenting striped shirts.
Perhaps semantic negation is hard?
"If you please ... draw me a sheep."
http://www.supercoloring.com/sites/default/files/styles/colo...