
Autodraw – Fast Drawing for Everyone - kbyatnal
https://www.blog.google/topics/machine-learning/fast-drawing-everyone/
======
toddmorey
It's not really "auto-draw" as much as it's a visual search in which you
suggest shapes and it looks across the collection for visually similar icons.
Impressive and fun, but not yet a huge advancement over just typing "house" or
"cake" to search the image library.

~~~
pierrec
Their description made it sound like a really cool lower-level tool, so
obviously it ends up being a letdown.

Drawing/art programs generally have a line smoothing feature - just smoothing
your wobbly lines as you draw, using relatively simple algorithms. The
description here made me hope for something more "medium-level", half-way
between the two. It wouldn't just smooth your lines - it would adjust them
according to context, based on a corpus of more precise line drawings, and
perhaps predict/suggest the next strokes. It might be difficult to pull off
though, if implemented naively it would probably just work against the artist.

~~~
applecrazy
This kind of functionality sort of exists in a basic form in the default mail
on iOS using a feature called Markup. Markup tries to guess if you drew an
arrow or a circle and suggests it based on your drawing.

~~~
comex
On a similar note, this site can recognize handwritten mathematical equations
(not just one symbol at a time):

[https://webdemo.myscript.com/views/math.html](https://webdemo.myscript.com/views/math.html)

~~~
pyedpiper
as well as microsoft's baked right into office Ink to Math convert

------
Animats
It's not a drawing tool. It's a search engine for clip art. That's very
Google.

This may be the answer for how to enter emoji. There are now over 2600 emoji,
with more to come. Keyboard selection isn't working and menus are huge.

~~~
tigershark
On iOS I just write the emoji name and it appears in the suggestions. Much,
much easier than trying to draw it.

~~~
euyyn
GBoard does that too.

~~~
simon1573
And SwiftKey as well

~~~
50CNT
And helm-unicode!

------
wyldfire
I was a cog in the machine [1]. You're welcome for my 1e-23'th-sized
contribution.

[1] [https://quickdraw.withgoogle.com/](https://quickdraw.withgoogle.com/)

~~~
jacquesm
Heads up: That fails to connect for me.

~~~
wyldfire
_shrug_ , works for me.

~~~
jacquesm
Yep, it's back up.

------
art1st
This crowd might appreciate the following comic, which I made with
autodraw.com (the tool this post is about) --

[http://imgur.com/a/WWff9](http://imgur.com/a/WWff9)

It took me about an hour, after I got incredibly frustrated that it wouldn't
let me draw anything. Can't draw a robot. Can't draw a sad face (only smiley
face). Can't even draw a stick figure. Can't draw a speech bubble.

I felt like it was fighting with me for what it wanted to draw, while leaving
very basic and fundamental shapes out. There were more things I couldn't draw,
I can't even remember everything.

~~~
art1st
Also a couple of bug report type things:

-> There's an undo button, it works well. But there should be a redo button. (Or the Apple-Y or Ctrl-Y keyboard shortcut for redo ought to work.)

-> See how my smiley face is too big on the right? Well I can't make it smaller: even if I zoom way in (there's a zoom functionality) I can't use the select tool to just select the smiley face (inside the jail) to reduce it in size. I'd have to recreate the parts of this image separately.

-> There is no way to set line thickness on the clip art! This should be one of the easiest things to set - but you can only scale the whole image, not the line width. That makes it hard to work with.

Overall I found the experience frustrating.

I have a challenge for you guys though: for the most common hundred thousand
or so words, use a machine learning algorithm on your own Image Search
results, to try to come up with canonical ideas of what the objects in
question might look like, after sorting them into categories based on
similarity of recognized features. Then have the algorithm create an outline
using the canonical idea it has derived for each category.

What I mean is that if someone Google's "hand" they might get: left hand,
right hand, fist, middle finger, OK sign, I mean there really are only so many
ways to hold a hand, or visual meanings/memes for the idea of "hand", and
other artists already have introduced a canonical version. (Likewise "stick
figure" has a meme around it.)

So for each one of those, the algorithm could learn from every version of that
that it judges as similar to each other -- and then draw it's own for each
one! (Computer algorithms are good at drawing in a learned style, even such as
Van Gogh's, etc.)

Other simple examples include a "peace sign". If you Google image search
"peace sign" you obviously get a very canonical shape. Why can't a machine
learning algorithm draw its own?

This idea of deriving free, creative-commons licensed images (not subject to
trademark search of course), by a machine learning algorithm trained on a huge
corpus of image results date (in a fair use way), without copying any of them
in particular, would be huge.

You have most of the interface to do this. It is a nice next challenge for you
- and a very serious one. I suggest you do it!

------
wehadfun
Whats the process for Google to make this sort of thing? Does some 7 figure
exec say we need to make it easier to draw bikes and then Google gets their
army of 10x engineers to make this happen?

~~~
zippergz
[Edited because I'm dumb and can't count figures] I've mostly seen this kind
of thing happen because some engineer(s) wanted to try an idea, not because it
was imposed from above.

~~~
tlarkworthy
College grads make 6 figures, 7 figures is one million or higher

~~~
zippergz
LOL. Yup. I'm an idiot.

------
cialowicz
This would be great for flowcharts and diagrams. Sketch out a rough diagram on
a tablet, and then have the shapes and lines "snap" to crisp versions as soon
as they are identified. Even better if I could draw it on a whiteboard, take a
photo, upload it, and get a response back as soon as it's done being
converted.

~~~
bronson
There are bunch of apps that do this on ipad and Android... Plus Microsoft's
note taking app, Lenovo/IBM's old X-series apps, and I'm sure others. Heck,
the Newton did it.

If you're curious, try one of them out. It gets frustrating pretty quick.

~~~
yagibear
Could you please name some? There seems to be an even bigger bunch of apps
that do plain drawing, and it can be hard to find the needle (apps that
convert rough sketches to clean line art) in the haystack (many apps for
sketching; most just replicating the paper experience on a screen without
adding functionality). Thanks.

~~~
tedmiston
Paper by FiftyThree -
[https://www.fiftythree.com/paper](https://www.fiftythree.com/paper)

~~~
yagibear
Thanks

------
forgottenacc57
Google needs a better way to lifecycle these things. Clearly this project will
be cancelled, so rather than just reinforce its reputation for killing its
projects, perhaps they need "experimental" projects that might even get spun
out of the company. Or something like that.

~~~
neil_s
From the blog post: "We hope that AutoDraw, our latest A.I. Experiment, will
make drawing more accessible and fun for everyone."

The Autodraw website literally says "This is an A.I. Experiment" in big
letters

------
sly010
Google keeps coming up with ways to use machine learning to do autofills,
suggestions, etc. A month ago Allo [0], then that article in Verge about
computational photography [1], then cameras without lenses [2] and now this.
There is no question that this is all very powerful and awesome, but it also
raises some questions, like who is the creator of a photo / drawing? Is every
photo / drawing going to look the same in the future?

Here is an illustration of what I am concerned about:

My wife downloaded google "Allo" (Yet another chat app where you can change
font size. Innovative, I know.). It also happens to suggest answers so you
don't have to type as much.

Here is how it went:

    
    
      She: Hi!
      Me: Hi how r u
      She: Where r u
      She: Where r u now?
      She: At home?
      She: Working?
      She: I missed u
      Me: Working
      Me: Missed u too
      Me: What u doing?
      She: How are u?
      Me: Fine thank u
      Me: What about u?
      Me: What are u doing?
      Me: Can i see u?
      She: Working
      Me: Oh
      She: Yes
      Me: Where r u from?
      Me: Who are u?
    

And it kept on going for a long long time, none of us actually saying anything
real, but both of us learning a lot about what looks like an average socially
awkward American teenager conversation. It had love, beauty, cuteness, gifs,
it even made us add some daily love quote bot to our thread, but we never
actually typed anything ourselves because it was so easy not to. Of course we
both knew it and thought that it's funny, but I can't shake this weird feeling
that something is very wrong with this and that in the long term we are being
brainwashed to be a dumber, more superficial version of ourselves.

p.s. I never use "r u", I find it lazy.

    
    
      [0] https://allo.google.com/
      [1] https://www.theverge.com/2016/10/18/13315168/google-pixel-camera-software-marc-levoy
      [2] https://www.theverge.com/2017/4/12/15267486/photography-machine-learning-future
    

Edit: formatting.

------
Waterluvian
I was surprised how poorly it ran on my very modern phone. And then how tiny
everything was on my desktop.

When I looked past that and tried to draw a cat, it wasn't all that useful. I
mean cool, you saw I was drawing a face and gave me 50 options. But what am I
supposed to do with that?

It feels like a rehash of what the Newton would do when you tried to draw
stuff. But it does it better. I think if I could skip the "pick what I meant"
step, it would be cool for whiteboarding in the office.

~~~
johansch
That's because your very modern phone has a very puny CPU compared to even the
average desktop CPU. I'm surprised about how few people know that their "2 GHz
multi-core" phone is 5-10x slower than an average 5 year old desktop on common
tasks.

(Edit: hehe, as evidenced by this post being downvoted. The HN audience
doesn't know any better either?)

~~~
photojosh
Probably downvoted because you're wrong? (Not that I did.) But this is with
the caveat that this is comparing a desktop Mac to an iPhone and I haven't the
faintest clue about top Android phones, although I have the understanding that
the A10 destroys the current Qualcomm SoCs.

Here's an instructive article from last year comparing a 2013 Mac Pro and an
iPhone 6s: [https://www.mikeash.com/pyblog/friday-
qa-2016-04-15-performa...](https://www.mikeash.com/pyblog/friday-
qa-2016-04-15-performance-comparisons-of-common-operations-2016-edition.html)

The relevant quote: >The most remarkable thing about this is how similar it
looks to the Mac results above. Looking back at the old tests, the iPhone was
orders of magnitude slower. An Objective-C message send, for example, was
about 4.9ns on the Mac, but it took an eternity on the iPhone at nearly 200ns.
A simple C++ virtual method call took a bit over a nanosecond on the Mac, but
80ns on the iPhone. A small malloc/free at around 50ns on the Mac took about 2
microseconds on the iPhone.

>Comparing the two today, and things have clearly changed a lot in the mobile
world. Most of these numbers are just slightly worse than the Mac numbers.
Some are actually faster! For example, autorelease pools are substantially
faster on the iPhone. I guess ARM64 is better at doing the stuff that the
autorelease pool code does.

>Reading and writing small files stands out as an area where the iPhone is
substantially slower. The 16MB file tests are comparable to the Mac, but the
iPhone takes nearly ten times longer for the 16-byte file tests. It appears
that the iPhone's storage has excellent throughput but suffers somewhat in
latency compared to the Mac's.

------
dvt
This is awesome!

Reminds me of a little toy project I made 5 years ago:
[https://www.youtube.com/watch?v=3WswSywx6TI](https://www.youtube.com/watch?v=3WswSywx6TI)

~~~
tomcam
TL;DR as you sketch on one side of the page, a dynamically updated visual
search appears on the right side. Very cool. Tell us more.

~~~
dvt
Just one of my many fun throwaway projects :P It used to be up at skrch.com,
but went down a while ago. Couldn't figure out how to monetize/sell it so I
moved on -- I'm still not sure what sector could use something like it. The
original idea actually came about in a dream (true story!) and I wondered if I
could actually implement it. Took me about a month or two as I had never used
OpenCV before.

The search was done with a very simple histogram analysis algorithm and the
image database had about 10,000 pictures from Flickr. Results were pretty
decent, but sometimes hit and miss[1]. Database costs were pretty high as I
don't think there's any database out there that has any way to efficiently
hash 2d histograms (so everything was stored in memory). That could be a fun
challenge.

I open sourced it a while ago here:
[https://github.com/dvx/skrch](https://github.com/dvx/skrch)

[1] [https://stackoverflow.com/questions/5332212/feature-blob-
cor...](https://stackoverflow.com/questions/5332212/feature-blob-correlation-
and-histogram-analysis)

------
dheera
This reminds me of Chinese handwriting input methods, which have almost the
exact same UI. You draw a character on the screen, and you get a selection of
results at the top.

------
apeace
Since nobody has mentioned this yet: I found that the core search
functionality is not very good.

I tried drawing a frowny face, a stick figure person, and a puppy face, and it
didn't recognize any of them. I'm terrible at drawing, but I feel these are
objects that have a universally-understood outline.

------
roywiggins
Reminds me of detexify.

[http://detexify.kirelabs.org/classify.html](http://detexify.kirelabs.org/classify.html)

~~~
the_alchemist
That tool saved me many times. It's hard to do obscure symbolic searches in
Google :')

------
chvid
Fun idea but doesn't really work. Just sorta replaces your random doodle with
a random piece of clip art. Any trace of your original drawing is gone.
Disappointing.

~~~
RussianCow
Isn't that the entire point? Or am I missing something?

~~~
iak8god
Yes, that is the point of this tool, and it probably beats slapping together a
bunch of clipart based on google image search, but I was hoping it was
something more than that. The tagline is misleading because it's not really
helping anyone draw -- it's just a visual search for sketches.

I sketched a really rough palm tree and it suggested a bunch of tree drawings,
one of which was a palm. That's helpful, but everyone who wants a palm gets
the same palm. Wouldn't it be great if the tool recognized that I was trying
to draw a palm and then improved mine by adjusting it according to what it
knows about sketches of palms (smoothing the lines, adjusting angles, etc)?

------
neovive
Very fun! Wishful thinking, but I'm hoping they partner with The Noun Project
and add SVG downloads.

~~~
legohead
Everything looks vector-ish, was surprised it didn't download as SVG :(

------
bhouston
This would be even more interesting if one could morph the provided object
with the provided sketch. Sort of like style transfer.

------
intoverflow2
This turned out to be a lot less interesting than it would be in my head.

Was expecting it to maybe use the data from the other drawing experiment to
dream up new creations. Not just search a limited library of glyphs

------
llimllib
There are no butts, what type of auto-drawing tool doesn't include butts

~~~
slig
You can suggest new drawings here
[https://docs.google.com/forms/d/e/1FAIpQLSeWmEs3Tjb2G-31On1l...](https://docs.google.com/forms/d/e/1FAIpQLSeWmEs3Tjb2G-31On1lribhwiDT9PL6SeKrt1FbG7hRAfwpvA/viewform)

~~~
llimllib
I suspect there are no butts because they are a stodgy corporate entity, not
because they haven't thought about butts

------
tbabb
Useless.

[http://i.imgur.com/NmpdbT2.png](http://i.imgur.com/NmpdbT2.png)

~~~
smashed
I concur.

Drew a face and it proposes ovens and random jitters as closest matches???

[http://imgur.com/a/4mjz0](http://imgur.com/a/4mjz0)

------
omurphy27
Very cool, but they need an export to SVG option. Right now it looks like it
only exports to png.

~~~
computerwizard
I thought the same thing. SVG would be killer!

------
TheBeardKing
This could be so much more useful than it currently is. If anyone has used
Microsoft Visio you know what a pain it is to find symbols while searching
through a library, especially if it isn't associated with a common noun. Where
are the simple arrows, Greek letters, schematic components? Perhaps they'll be
integrating this type of technology into an actual useful product in the
future, like a Google Drive version of Visio?

~~~
iplaw
Agreed, I feel your exact pain with Visio.

They should have this for all UNICODE characters, too. There are web services
that attempt to do this, but they fail pretty miserably.

------
victormustar
What I need is betterdraw, you input a bad drawing then it correct
perspective, shapes, etc...

------
zargath
Somebody need to feed the robot overlords with more dirty body parts.

joke aside, Adobe should not be worried just yet. It seems to be just a image
tagging service or a terrible drawing application. When I draw a face of a cat
it suggests a body of a cat. When I draw a rocket is suggests a glass of vine
and so on.

The QuickDraw game was fun and a good idea, but basing a drawing application
on "topics" from that game seems like a bad choice.

------
andreygrehov
Boring. Really, what's the point? It doesn't even make a connection between
more than one "drawing". Try to draw a triangle, select the shape and then
draw another triangle to make the two look like a square - you won't see a
square option in the suggestions pane, since it doesn't see/remember your
first triangle. My kid would probably like it though.

------
njharman
This is awesome tech. And probably useful.

But, art even/esp hand drawn scribbles losses almost everything without the
character and idiosyncrasies of the artist. This is little more than a fancy
ui for a clipart library. Clip art sucks. Although it does have a purpose,
limited as it may be.

------
slig
For some reason they left some `console.log` and you can see if you open the
Developer Tools.

------
peterkshultz
My guess is that they trained the model for this with the data they collected
from
[Quickdraw]([https://quickdraw.withgoogle.com/](https://quickdraw.withgoogle.com/))

------
willhackett
Okay. This is great, but I feel like Google's off doing everything but looking
after their current products. Google Inbox is in dire needs of new features to
bring it in line with competitive mail products, and GMail needs a facelift.

~~~
cschmittiey
Regarding Inbox, I couldn't care less about more features at this point, I
just want better desktop web performance.

------
geluso
Wow, it's really annoying that there's no way to type text in and get that
shape. I get that it's cool that it will (sometimes) recognize what I draw,
but apparently I can't draw the Space Needle for the world! It would be nice
to still be able to search their images with text manually instead of having
to try to draw everything you want when you know you'll use what they have.

------
nathan_f77
This is pretty incredible. The only really important thing that's missing is
the ability to flip the images horizontally or vertically. Otherwise my cow is
unable to wear a helmet:
[https://www.autodraw.com/share/T7HFJ9TVN91J](https://www.autodraw.com/share/T7HFJ9TVN91J)

------
cableshaft
Awesome. As a board game designer, I could see myself using this to make
prototype cards that look decent much easier. Although I'm sure I'll still
need Illustrator to take it to the next level. But for a quick and dirty
prototype, it should work great.

Much better than searching the web for hours for icons that have a similar-ish
art style that have what I need.

~~~
failrate
Thenounproject.com is really good. Also, Daniel Solis sells packs of really
specific board game icons.

~~~
cableshaft
I keep forgetting about the noun project. Also, those icons by solis look like
they could be useful, thanks for letting me know about that. I was aware of an
old card game design series he did, but not his Patreon. I've also picked up a
few of his games that are still on my Wall of Board Game Shame and need to get
played.

------
markc
Two minutes use and color me unimpressed. The first thing I drew should have
been a slam-dunk and it wasn't recognized. (I challenge anyone to draw a
padlock that autodraw can recognize!)

Also "Fill" doesn't fill except with pre-defined shapes. What's with that?

------
6stringmerc
I can do a passable Bart Simpson face and the top three recommendations were
for Teddy Bears. I think that's quite cool. I might throw some other stuff at
it later when I have a stylus to play with. Love finding out about these
things.

------
wolfgang42
This reminds me of the excellent
[http://shapecatcher.com/](http://shapecatcher.com/), except for an icon
library instead of a Unicode font.

------
tambourine_man
"There's nothing to download"

Yeah, that's the web for you. With all the obsession with apps in recente
years, I'm glad to hear that being advertised.

Maybe we're hitting an App saturation point.

------
danmaz74
It's really cute. I wonder if Google is going to use all the classification
that people will do of their own sketches to teach its machines... to
recognize hand-drawn sketches.

~~~
jpeeler
That was my first thought as well. But I do wonder about the quality of
submissions given that most people can't draw with a mouse as well as they can
with a writing utensil (assuming a drawing tablet or touchscreen is not used
either).

~~~
dragonwriter
> assuming a drawing tablet or touchscreen is not used either

Given the prevalence of mobile devices, some of which use styluses—even
ignoring touchscreen laptop and desktop screens—why would you assume that?

------
awjr
One issue I see is that you can only export pngs. It would have been really
useful to be able to copy the selected drawing elements and paste them into a
Google Drawing

------
deepakkarki
What other people think : “Oh! what a cool drawing app, Google is so awesome!”

What I think : “Oh what an interesting way of crowd sourcing ML data, Google
is so smart!”

------
catchup
I can't see the auto suggestions on Google Autodraw. Sort of defeats the
purpose... Anyone with the same problem? I have a macbook pro.

------
therealunreal
Only thing that seems to be missing is a way to re-arrange the shapes
(z-order). I tried page-up/down and right-clicking but no luck.

------
vasco
Tried to draw a dick, failed to be recognized.

------
brilliantcode
interesting, it definitely supports the narrative of AI replacing changing our
jobs, in this case the designer.

For less artistic folks like me, this tool is from heavens. How many times did
you want to illustrate a simple diagram but couldn't draw or use photoshop?

Having said that I can see Autodraw still needs more work done. It failed to
recognize a phallus.

------
aerovistae
Really interesting idea that isn't yet implemented well enough to be that
interesting in practice.

The idea of machine learning refining your drawings as you go, forming a sort
of cooperative artistic partnership, is fascinating.

The idea of machine learning somewhat sloppily matching your drawings to pre-
existing ones and just replacing them.....well, kinda just feels like image
search copy/pasted into microsoft paint.

But it's a start!

------
mijoharas
If anyone else is confused about it not working, check if privacybadger is
blocking inputtools.google.com

------
finid
_With Autodraw, there 's nothing to download, nothing to install..._

An installation one would be nice, though.

------
zepto
Guns don't exist, but swords do.

------
lorenzosnap
unfortunately I get

Over Quota This application is temporarily over its serving quota. Please try
again later.

~~~
johansch
(I guess Google made it using App Engine.)

This is a little embarassing...

------
catchup
I can't see the auto suggestion bar on my Macbook pro. Anyone with the same
problem???

------
ixtli
Try getting it to display fire or flame that isn't attached to a candle or
birthday cake.

------
sharpercoder
Reminds a lot of shapecatcher (shapecatcher.com): Unicode character
recognition by drawing.

------
johnvega
The name AutoDraw sounds like it is created by Autodesk, the creator of
AutoCAD and AutoSketch.

------
trialanderror
Got my hopes up for a really easy drawing program. Instead got an icon finder.

------
Geojim
AI experiment? Bet the world pictionary champ is starting to sweat :P

------
wickedlogic
I'm seeing 404 for the images, wonder if it auto-reports that.

------
vdjkvdwkjb
ok i drew a cock and balls and the top matches were saxophones

~~~
lllllll
Does that mean it works? More importantly, how was the AI trained in genitalia
recognition?

------
mrdrozdov
This is a really clever data collection technique. ;)

------
harrystone
Never suggests anything on linux Chrome 57.0.2987.133

~~~
x0rgz
I had to allow third party cookies to inputtools.google.com in privacy badger
for it to suggest things

------
lightedman
This is literally the "Write the kanji" feature which has been present in
Google Translate for years, with a different 'character set.' Not exactly
impressive.

------
nzjrs
Wow, the blacklist of classes is really long.

------
halloij
Don't get used to it... If it's an "experiment", and "free", then it will
likely get shutdown.

------
shirro
Great, more clip art. /s

------
cozzyd
tried drawing some circuit components but they weren't in the library :(

~~~
ORioN63
I couldn't get a horse. It was a horrible drawing, though.

~~~
iak8god
It thought my horse was pliers, a camel, or a hand gesture:
[http://imgur.com/a/JWifP](http://imgur.com/a/JWifP)

~~~
failrate
Someone could make a great version of "Eat poop, you cat" with this.

~~~
iak8god
Oh, hell yes.

1\. Make a sketch of your choice 2\. Pick the first AutoDraw suggestion (or
randomly one of the first N) 3\. Feed that to google image search 4a. Google's
_best guess for this image_ is a prompt for the next human sketch. Repeat from
2. OR 4b. Pick a sketch-like image from the results of 3 and reproduce on the
AutoDraw canvas. Repeat from 2.

I got an amazing result on my first attempt at 1 - 3:

Autodraw (Cat > Raccoon): [http://imgur.com/a/Fy00K](http://imgur.com/a/Fy00K)

Google Images (Raccoon > "Suppression of Enemy Air Defenses"):
[http://imgur.com/a/F3z1T](http://imgur.com/a/F3z1T)

It'd be hilarious to automate this, set it running, and just watch it go.

------
eric_Anderson86
Hey

------
eric_Anderson86
nryche0725

------
brianzelip
I didn't realize there were now brand top level domains, such as `.google`.
Here is a list of more, [https://en.wikipedia.org/wiki/List_of_Internet_top-
level_dom...](https://en.wikipedia.org/wiki/List_of_Internet_top-
level_domains#Brand_top-level_domains)

