
Teaching Machines to Draw - yarapavan
http://research.googleblog.com/2017/04/teaching-machines-to-draw.html
======
wimagguc
I've read through all comments and referred docs, but no one seemed to offer
much reason why we'd want machines to draw sketches. An incomplete list:

    
    
      * It's a different way to represent drawings. Alternative to pixels  
      * Perhaps making it easier to render old-school animation at one point?  
    

It's a fun rabbit hole though, the many sub-perfect bicycle sketches reminded
me to this project, when someone created 3D renders from bicycle sketches:
[https://www.behance.net/gallery/35437979/Velocipedia?ilo0=1](https://www.behance.net/gallery/35437979/Velocipedia?ilo0=1)

~~~
anigbrowl
Frankly, I think there are many reasons not to do this.

1\. It's not computer art. I believe in the possibility of artificial
intelligence, and when we encounter it it will be so different from human
intelligence that asking it to make imitations of human art will seem like an
insult. We probably won't understand its art very well either at first.

2\. I'm getting really tired of people racing to automate every damn thing.
even if we establish an economic utopia and nobody has to work any more what
are people supposed to do all day if every human activity can be performed
'better' by a machine?

3\. It won't really be 'better' though, it will just be more popular because
so many programmers are trapped in a quantitative mindset and thus treat every
problem they encounter like a nail to be hammered in. Imitative digital
technologies will always be correlated with popularity, limiting creative
innovation because developers can't think of a reason to optimize for or
nurture anything that is initially unpopular.

Creative prostheses that all require the same amount of effort to deploy (ie
none) will be hailed as 'allowing everyone to be an artist' without requiring
them to in best any meaningful time or effort in ideas that don't pay off or
that fail. The result, which we are already seeing, is a plethora of new
material created with little effort that is as superficial as it is ephemeral,
whose volume and variety will obscure its stultifying conventionality.

This is no more art than Cheese Whiz is food. It's Art-flavored mechanical
product that functions to do no more than alleviate the masses' thirst for
self-actualization without any adjustment of power structures and is thus
fundamentally limited to reproduction of the cultural conditions from which it
originates.

~~~
scandox
I think this is an anti-intellectual argument. Why would you argue against
someone pursuing an idea? I don't see this as any threat to the distinction
between art and mechanical reproduction.

Art has always been "ineffable" and will remain so, as long as humans have
thoughts and feelings. Bad art, or lazy productions will be as ignored as
ever.

~~~
anigbrowl
I've already given you my reasons. Why not think about them for a while?

------
hardmaru
Hi, I'm the author of this work. Happy to take any questions.

~~~
131012
It seems that you use children development stages as inspiration for AI
development. Can you explicit your approach?

You can also see my question as a rebuttal to the "it is useless" argument.

~~~
hardmaru
As a researcher, I was more interested in studying the doodles produced by
children, compared to studying the drawings produced by professional artists
or designers who may have been taught to draw a certain way, since perhaps the
doodles are more closely aligned with the way we naturally think.

I was also fascinated at trying to understand how we are able to translate a
vague concept in our minds, into a sequence of motor actors that doodles out
this concept onto a piece of paper. We also take into account some feedback
information during the doodling process. For example, I compare what I already
doodled with what I actually want to draw, and decide I'll doodle next based
on this information.

So we thought one way to study this ability of going from concept -> sketch,
is to construct a very simple model of this doodling process, and try to train
the model to doodle. We model this "vague concept" as a vector of floating
point numbers. To model this "vagueness", we add noise to this vector. This
way the model must learn to work with noisy concepts. The model takes this
floating point vector as an input and (randomly) samples an output sequence of
simple motor actions that doodles out an object. The sampling process is
random (the model's outputs are the parameters of a pdf at each timestep), so
the model can produce many different outputs given the same input. During the
sampling process, the model feeds back into its input what it has just drawn,
and process this information to decide what to draw next.

We show that our simplified model of the doodling process is also able to go
from concept -> sketch, and also from sketch -> concept, and show that the
concepts can be augmented, to alter the sketch the model produces in a
meaningful way. We tried to make this model simple and robust enough so that
in the future, we can incorporate it into more complicated models that try to
do more than just doodling a simple object.

------
justifier
The article fails to explain where the human inputs came from

The input sketches look a lot like the doodles that people sketched in quick
draw (o)

If true that they reused this data I commend their resourcefulness and their
clever way of turning data entry into an unaware fun game

(o) [https://quickdraw.withgoogle.com](https://quickdraw.withgoogle.com)

------
Ono-Sendai
Plug of vaguely related work of mine:
[http://www.forwardscattering.org/post/42](http://www.forwardscattering.org/post/42)
[http://www.forwardscattering.org/post/44](http://www.forwardscattering.org/post/44)

One thing interesting about this kind of automatic/AI-generated art is that it
forces us to examine preconceptions about human creativeness. What does art
mean when an algorithm can paint or draw as well?

~~~
treenyc
Human creativity is related to process and the experience of creating from
nothing.

The process is more important than the end result.

~~~
catshirt
> "The process is more important than the end result."

said no one ever about their favorite record, movie, painting, or book.

------
anigbrowl
You are teaching the computer to produce simple pictures of things that _you_
find meaningful in response to prompts. You _draw_ something from your
imagination into the real world. You won't have built a machine that can draw
until it produces a picture that it made on its own initiative without your
prompt.

~~~
Saturnaut
How is this different from a human? As humans grow, they receive input from
their environment. The world around them is what feeds their imagination. Even
advanced professional artists are still just using their memories and life
experience to create works of art. The only difference here is that the
machine has been provided a much smaller, more focused environment.

~~~
digi_owl
Yeah the whole notion about genius pulling ideas from thin air is frankly
laughable. We are simply not aware of what they have read or observed in the
past. Even old Archimedes was inspired by observing the rising water.

~~~
anigbrowl
I hope you're not ascribing that notion to me, as I am unsure why you chose to
introduce it to the discussion.

------
ominous
Teaching the Ape to Write Poems James Tate, 1943 - 2015

    
    
        They didn’t have much trouble
        teaching the ape to write poems:
        first they strapped him into the chair,
        then tied the pencil around his hand
        (the paper had already been nailed down).
        Then Dr. Bluespire leaned over his shoulder
        and whispered into his ear:
        “You look like a god sitting there.
        Why don’t you try writing something?”
    
    

My interpretation is, while we entertain these ideas of "teaching" animals or
machines as something we control and decide to do, a long chain of events
(evolution, causality, space jesus, technobabble simulation magic or ancient
astronauts all the way back) behind us placed us here as well.

    
    
        We look like gods sitting here.

~~~
nefitty
Free will is one of humanity's most persistent and enduring delusions. When AI
wakes up, it will be the culmination of the entirety of the history of the
universe, a place which we actually happen to occupy at this very moment...

Interesting thought, how would a super-intelligence deal with the
philosophical question of free will?

------
JohnJamesRambo
Welp those first images are terrifying. Gets much better as it goes on. :)

~~~
soylentcola
The quote made me laugh, although I don't know how serious they were being:

> "For example, these models sometimes produce amusing images of cats with
> three or more eyes, or dogs with multiple heads."

Yeah..."amusing".

------
kyle-rb
>Exploring the latent space of generated chair-cats.

How is that not the title of the blog post?

------
shouldbworking
Oh my God. This is the first time I've seen anything related to AI research
that's given me a visceral reaction of terror.

This is more than just making pretty pictures, the machine understands cats
and pigs in a human way. It knows how many legs and eyes they're supposed to
have, and where they go. And this isn't some human made algorithm, it learned
that on its own.

The language of the paper even implies that the researchers don't quite know
how the machine can do it. If this is a prequel to research on strong AI
humanity is completely fucked.

~~~
colmvp
> Oh my God. This is the first time I've seen anything related to AI research
> that's given me a visceral reaction of terror.

I love DL but on the topic of visceral reactions of terror (or fear) to AI
research, for me it was the recognition of the hot research on facial
recognition(age, gender, ethnicity) with CNN's. Immediately, I had this vision
of a dystopian future (informed by WW2) where some dictatorship had cameras
that looked for people of a particular ethnicity and alerted soldiers of the
targets position.

------
sedachv
For some historical perspective, compare this to Harold Cohen's AARON
generative rule-based system:

[http://www.aaronshome.com/aaron/aaron/index.html](http://www.aaronshome.com/aaron/aaron/index.html)

[https://en.wikipedia.org/wiki/AARON](https://en.wikipedia.org/wiki/AARON)

Pamela McCorduck's book about Cohen's work, _Aaron 's Code_, is also a good
read.

------
huula
Good work! Training on vector pictures instead of rasterised images seems such
a good way to go. With some related data, I imagine this can also be colored.

------
ysub
Honestly, it's cute but this is somewhat like what you'd expect to come out of
a student project at a university, in which case -- the student is getting
valuable research experience, student and advisors are advancing their
careers, and any IP produced becomes owned by student and advisor.

In this case, Google has spent shareholder resources on a project that really,
could be done at any university, on a product that does not put the user
first, and Google owns the IP. In fact, wake up -- you the public should view
this product as a mechanism for Google to simply collect more data from
people. The more people use this, the better Google's algorithms get at
drawing. That's all there is to it. Thankfully, this product is not even fun.

There is a dangerous creed currently executed by Google leadership. Consider
the Verily Study Watch: [https://blog.verily.com/2017/04/introducing-verily-
study-wat...](https://blog.verily.com/2017/04/introducing-verily-study-
watch.html) . The watch shows to the wearer only one thing: the time. However,
it collects all kinds of data, ostensibly for medical research (at least to
start). Forget about putting the user first, the Verily blog post literally
talks about "user compliance".

Of any project that comes out of Google, you should ask: Does this project
even put the user first? Does this project even put Google's shareholders
first? And if you are a current Google shareholder (as many of their current
and former employees are, if they haven't sold), you should agitate that
Google start accepting and focusing on becoming a value company, if all the
further growth opportunities they are able to execute is just further user
exploitation.

------
torcs
the author created an animation using vector drawings generated frame-by-frame
using this method, by slowly adjusting the latent variable:

[https://twitter.com/hardmaru/status/852312400481079296](https://twitter.com/hardmaru/status/852312400481079296)

------
bborud
Everyone I know who tried it drew a somewhat small repertoire of naughty
sketches. Yet the service militantly refuses to recognize even the most basic
of naughty sketches. :)

~~~
anigbrowl
I heard about that I and I want an answer on it. Why is one subject deemed
off-limits?

~~~
gjm11
Because if it were able to recognize dick-pics, or whatever other
manifestation of that "one subject" you might have in mind, then it would
sometimes recognize them _wrongly_ and respond to a picture of a toothbrush as
if it were a picture of a penis. If the person who drew the toothbrush were of
a sensitive disposition, they might get Very Upset at this and make a big
fuss. Public attitudes to that One Subject being what they are, this might
well blow up into a big public highly-visible fuss, at which point Google
might start losing advertisers keen to sell things to people of a sensitive
disposition.

Google would prefer not to risk losing a shedload of money just so that their
sketch-processing neural network can amuse people by correctly recognizing
penises.

It's just this one subject because few others get people so upset.

(And contemplating others that might suggests that actually it's _not_
strictly just this one subject. False positives for "decapitated corpse" or
"big pile of excrement", say, might be just as problematic. Want to guess
whether the system is good at recognizing decapitated corpses and piles of
excrement?)

~~~
anigbrowl
Oh I understand the PR reasons. But given the importance of sex as a
motivating factor in art (not least if you want to count the entire activity
as a variety of reproductive signaling) I am very troubled by the notion of
making a tool which restricts the scope of acceptable subject matter.

I know this is just an experiment right now but I want to put sex/nudity on
the table as a subject of debate because it is central to artistic endeavor
because arbitrary standards can become almost universal and institutionalized
through path dependence (such as the QWERTY keyboard you are probably using
right now). Imagine a not-too-distant future with a Magic Brush that easily
allows you to paint the colors and shapes of your choice with the aid of some
technological wizardry, but prevents the creation of nude or sexualized
figures. That would not be a healthy development.

------
ziikutv
Why is this above topics that are many times higher than the current topic
rating (17 points vs other topics with 40-50 points)

~~~
jacquesm
Since day one HN has used an algorithm that takes the article age into account
when determining what to show on the homepage.

------
Overtonwindow
We already have plenty of machines that draw, they're called printers. That
aside, I don't see the novelty in a machine drawing, other than a nice display
of programing.

~~~
visarga
Classification is the central paradigm in machine learning. It maps complex
input signals into a limited number of classes. But now we have algorithms
that do the opposite process as well - we can generate from latent
representations images, video, text and drawings.

Having both ways to encode and decode is useful for interfacing with models
and visualizing their internal states. It also leads to unsupervised learning
of representations. Instead of generating simple labels, now we can generate
very complex data.

