I can show this to someone and say:
1. The software can recognize a feather, as long as it looks similar to what it thinks a feather looks like.
2. The software can't recognize a feather if it's never seen a feather like that. It's not a sentient being.
This is good, because most examples focus on point #1 and -- if enough marketing is involved -- don't go enough into point #2.
People read news articles like "X can recognize cats in a picture with Y certainty!" and are quick to assume that this "AI" can make sense of a picture and understand it, when all it does is apply certain methods for a certain use case.
This does a much better job by letting people write (or draw) their own test cases and figure out the limits intuitively.
I was prompted to draw a hurricane. I drew something that looked like the typical hurricane doodle used on news reports.
The software didn't recognize it.
When the game was over and I was able to look at all of the doodles that were used to train the software to recognize a hurricane ... the majority of them instead looked like tornadoes!
So maybe we should more precisely say:
1. The software can recognize a feather, as long as it looks similar to what the humans who contributed its training set think a feather looks like.
I'm also ashamed to admit I drew some less than ideal stuff due to forgetting details on things and then panicking because of the timer. Like the spots on a panda's face for some odd reason.
Hopefuly my drawings were treated as outliers.
1. The person can recognize a feather as long as it looks similar to what the other people who contributed to it's learning think a feather looks like.
Idiocracy was prophetic -- except it missed the aspect that "Idiocracy" would first manifest on the Internet.
If I had a dollar for every time someone pattern matched me or a phrase I wrote, then jumped to conclusions about my ideas of internal emotional state then even insisted I am lying when I tried to disabuse them of the notion -- I'd have a whole lot of dollars. (Hint, if you start sniffing around trying to justify that they're right, I haven't left you sufficient evidence and you're probably also doing that.)
Apolitical people? Mostly just as bad.
On the one hand, the network should eventually learn to classify high heels as shoes.
On the other, when these classification system actually get used, they're always at some arbitrary point in their training, so you can't just wait for "all the biases to go away."
 Not just the young fashionistas that specifically dress up, but every woman.
This is the kind of thing AI researchers write papers on (source: AI MSc), not some SJW topic, yet you saw the word "gender" and assumed it didn't belong?
For recognizing relatively simple entities, are there advantages humans still have over neural nets (assuming the same scope of knowledge)?
An idea: We can also run several cat photos through image processing algorithms to filter out details. The output would be outlines similar to the drawings in the Google Quickdraw app. We put those through the app to generalize (perhaps the app needs some training with a few categories of objects, not necessarily animals). Voila! Software can now recognize drawings based on photo examples.
Of course, there's severe bias here, in the sense that what we consider abstraction is by definition "human shaped" abstraction
If multiple humans try to "abstract" a cat, the overlap in underlying processes will be pretty big, making it more likely that we can recognise each other's abstractions.
I can read the words here, but I don't understand the meaning.
We abstract to find a common set of features in things that are supposed to be the same but that are not present in things that are not supposed to be the same. Grouping these features then produces higher level abstractions, and so on.
Where would the bias be?
Even if the features differ, the process is the same.
And even the features are often the same. If you reverse a DCNN to see what it uses to classify things as "cats", expect to see whiskers and fur.
Computers don't look at things from a human perspective; they're still good at abstraction, just different to human abstraction. i.e. there's a human bias in there.
That's OK though; the objective is to make a computer that sees things the way people do; so it's a bias we want.
However the issue isn't that the computer's not a sentient being and therefore can't abstract things it's never seen before; only that the algorithm hasn't been written to sufficiently take account of human bias.
I don't see a fundamental difference between biological and electronic neural nets; so please take the following with a physicalist grain of salt. Imho, precisely because NNs will be fed with nothing else than the reality (physical or virtual) we live in, it should gradually develop the same familiarity as humans have; i.e. nothing more and nothing less than elements of our lives/civs. Visually lots of cats, lots of cars, mountains and coasts; functionally all the tasks we accomplish daily, like driving or cooking or cleaning.
I don't really think you can hard-code "human bias" as it's an emergent property of our biology: too complex (we don't really understand much of it, imho you're bound to miss the mark and induce subjective biases), and somewhat contradictory to how NNs are supposed to evolve (thinking long term here). Basically, I don't think it would be practical nor cost efficient to induce too much perturbations in deep learning, better work on refining the process itself. Think of plants: you can tweak the growing all you want, but the root deciding factors lie in genetics (their potential, and in understanding how to maximize it).
I realize another wording is that we should apply sound evolutionary (Darwin etc.) principles in "growing" AI at large. Because AI and humans share the same environment, we should see converging "intelligence" (skills, familiarity, etc). It's a quite fascinating time from an ontological perspective.
General purpose machine translation is harder, for instance. Brute force algorithms have gotten decent, but aren't in the same ballpark as humans (though professional translation services now often work by correcting a machine translation). However, MT systems trained on a specific domain do much better (medical or legal docs, etc).
What would be the hardest task for machines that's trivial for humans? Maybe deciding if a joke is funny or not?
A literate human scores 100% on this test. No computer system so far scores better than 60%. (And remember that random guessing gets 50%.)
The book, "We are all Completely Beside Ourselves" is fiction, but refers to findings from real studies.
Human perception is heavily biased towards features that had evolutionary advantages, and limited by whatever technical flaws our eyes/brains/etc have. That's a selection bias in our perception of information, in our processing of said information, and therefore in the abstractions that result from it.
I presume it's possible that the limitations of our visual system means we may miss powerful features and hence the ability to build some more powerful abstractions. (I didn't even argue this, just pointed out the process is the same even if features differ)
But I don't see how this supports your original claim of bias, which was: "If multiple humans try to "abstract" a cat, the overlap in underlying processes will be pretty big, making it more likely that we can recognize each other's abstractions."
If humans are good at recognizing each others' abstraction, that's a validation that low-pass (for lack of a better term) filtering the features due to human's physical design still creates very good abstractions and classifiers. That is to say, if anything you're confirming that humans are designed in a way that makes the abstractions they can make maximally useful.
... to other humans.
Are you arguing that the classifications themselves are biased?
Think of the Turing test and its criticisms; it's kind of has the same issues.
PS: I've upvoted every comment of yours; asking questions like this should be encouraged :)
My point is that "good" and "bad" are not objective here, but depend on human use-cases.
Now to be clear: I'm not disagreeing with you! These are good abstractions, for humans. It lets us communicate concepts easily, which is great! But it might not be the best abstraction in every circumstance.
For example, I recall reading an article that said that AI is better at spotting breast cancer from photos (which is essentially interpreting abstract blobs as cancer or not). The main reason seems to be that it is not held back by the human biases in perception.
Second, when we look at a picture of a cat, we're looking at a human's interpretation of what a cat looks like. If we asked a computer to draw a cat, it might look nothing like a cat to us, but another computer could look at it and go "Oh sure, that's a cat." I seem to recall Google did a thing with this a while ago, where they effectively created a feedback loop in a neural net - feeding its own drawing back into itself. As I recall, the result looked like the computer had done way too much LSD.
Google doesn't recognize it as a feline, it recognizes it as Garfield.
The software does:
Sure, you can fool a human. But there are things AI is missing that would be embarrassing if a human made the same mistake. It's hard to say, based on anecdotes like this, how big that gap is, but it's there.
I think we do. We see a building we've never seen before and we know it's a building because it has certain features that we use to classify it as a building. The examples aren't scarce.
I also think a good indicator of us doing it is the use of "y" and "ish" and "sort".
As for sthlm's point 2:
>2. The software can't recognize a feather if it's never seen a feather like that. It's not a sentient being.
This is Asimo in 2009:
I feel there is an immense difference between recognizing simple sketches and deriving what an object is based on extended characteristics.
The video you linked furthers that by showing that ASIMO was using three-dimensional observation to calculate certain features and ascertain what that object was.
If you'd give these doodles to people that are not Western males it'll do a lot worse. Someone already pointed out it doesn't recognize woman's shoes.
It is unmistakable how much the difficulty level ramps up when you're paired with those of an unlike-nature to you. Sometimes that level of abstraction is taken way outside of generic context clues.
We use very generic "words" (eg egg, tree, bike, cloud, plate).
When you're using your foot to draw you really have to distill down to the essence of the item. Yes there is a deal of guessing but in some way the image (however unlike the object) has to have some element of the Platonic nature, if you will, of the object being drawn.
As for advantages over neural nets, one of the primary ones is that humans can recognise things from unusual angles much more easily. When I tried QuickDraw and doodled things from non-stereotyped angles (like a three-quarter view of a car rather than the usual 2D side view), it had no idea.
The dalmation optical illusion is another example of human ability to pick out patterns and assign them to belong to certain objects. Neural nets have different abilities, and are sometimes better at picking out different sorts of patterns than humans.
Why did this word "sentient" sneak in to your comment? I don't see what "sentience" has to do with what you just described; it's just a more sophisticated form of pattern matching.
"See, it can't do this! It's not self-aware!" is almost never the correct answer, because whatever thing it is you want to do will probably be solved in the future with more of the same techniques. Just about the only thing "sentience" or self-awareness is good for is an entity's private experience, which you wouldn't ever be able to see anyway.
>People read news articles like "X can recognize cats...
may assume sentience when it's not there
Like humans brains?
>are quick to assume that this "AI" can make sense of a picture and understand it, when all it does is apply certain methods for a certain use case.
Like human brains?
Remember, this child has never been on the road, never driven a car, never had the mechanics of locomotion taught to them. All they know is that objects that are longer than they are tall with a flat bed on one side and wheels on the bottom are classified as cars.
Once the child (or machine) has more information to associate with the 'vehicle idea' it can call on this information when it sees shapes that are also associated with the 'vehicle idea' in order to extrapolate without having direct previous experience with that object being seen.
So when that machine learning algorithm recognizes wheels in a picture and recognizes seats in the same picture, it searches for results that include both wheels and seats.
The human brain does not inject any magic in to this process.
ML trained on bunch of static pictures is like humans dealing with those abstract geometrical riddles that are used on IQ tests. They're difficult for us, because they're not related to our normal, everyday experience.
Any neural net, artificial or not, can only recognize things as long as it looks similar to what it thinks the thing should look like.
Times are changing...
It's told in the first chapter of the famous childrens book, "the little prince"
> The narrator explains that, as a young boy, he once drew a picture of a boa constrictor digesting an elephant in its stomach; however, every adult who saw the picture would mistakenly interpret it as a drawing of a hat. Whenever the narrator would try to correct this confusion, he was ultimately advised to set aside drawing and take up a more practical or mature hobby. The narrator laments the crass materialism of contemporary society and the lack of creative understanding displayed by adults. As noted by the narrator, he could have had a great career as a painter, but this opportunity was crushed by the misunderstanding of the adults.
Ignoring the cultural reference everyone here is talking about:
How do you get that perception? We apply the adjective adult to animals to whom we (rightfully or wrongfully) don't attribute intelligence, and even to other lifeforms which we usually categorize as non-sentient (e.g. adult trees).
I think it is perfectly legitimate to assign the adjective adult to a neural network that has either left the training phase, or that is only undergoing marginal changes in further training. This seems to be mostly in line with how the word adult is used in other contexts.
Not that I'm opposed to calling software intelligent, in fact I think it would be weird if we couldn't call something intelligent just because it's silicon-based instead of being based on organic neurons. I just find it odd that you associate "adult" with intelligence at all.
So in this sense, I found it surprising to associate a word that I use for "living"/biological things to a digital thing; specially in the context of A.I.
Maybe the term adult is, for me at least, very loaded with lots of meanings that go far behind the simple notion of "maturity"
Love Le Petite Prince! Read it both in French and English.
Also with the sweater drawing I'd drawn just a T-shirt accidentally then when drawing a long sleeve it said it knew it was a sweater when it could have been similar things.
So this demonstration isn't about it identifying the drawing correctly, it's just about saying when it's found something considered close enough in an overly-broad range of ambiguous things based on the answer it's been given already.
It did identify some partially drawn things, like a line or a circle or more complex things the partial drawing could have been, but the pre-loaded rigging made me stop the test.
It's interesting to consider whether it would end up making guesses that it sometimes gets correct, or whether it would actually learn to discriminate based on very early stages of drawing. It's conceivable to me that people might start out drawing "police car" differently from "car" in some subtle but detectable way, even if it doesn't involve what we humans would consider to be the distinguishing features of a police car.
Exactly the same thing happened to me (I did a CTRL+F on 'police' to find this post), which immediately turned me from 'this is cool' to almost quitting, and coming here to see if anyone else experienced the same thing (perhaps with different images).
It also didn't guess my drawing of a bulldozer, which I was surprised at since my drawing wasn't bad, and easily recognizable to any human of any age as a bulldozer.
I went back to complete the test, and it tells you what it guessed, which helped to explain both things.
Here's what it thinks a bulldozer looks like:
Here's the difference between "car" and "police car":
You, like me, obviously don't draw cars from the Soviet Union.
Edit: I just noticed (thanks to another post here) it also shows several examples of what it thinks an object looks like, and now I'm not sure why it didn't guess bulldozer, when couple of the images look like mine, certainly far more than the things it guessed.
At first I was thinking, how could it know this is a circle, but for the sun, it knew circle wasn't it when I had only drawn a circle. But it just occurred to me that the program probably tells it whether or not it's right before it actually verbalizes the guess/answer.
Then a few drawings after that, I got bus, and it guessed school-bus about 15 seconds in, and it wasn't until 10 seconds after that when it finally guessed just bus. So, I'm guessing this is the same for yours, it was simply guessing police car before car (probably because the NN takes probability into account and has gotten that as a right answer more often or something), and in your case it just happened to be right so it stopped guessing before it tried just "car."
I drew the exact same doodle for each and for each it guessed correctly on its first try.
If what you're supposed to be drawing is ranked 'sufficiently high' the game/training side of it moves on to the next.
The first guess for each doodle was the word that was provided -- if this software was completely legitimate, I would think it should have guessed the same word for each (and then moved on to other guesses).
A good test for that would be to script the game and give it pixel-identical answers for each questions and see if the result is different.
I guess a star is a zig zag looping on itself, kinda.
For instance, our sub-conscious tendency when drawing stairs would be to have straight lines at 0 and 90 degrees but a "zig-zag" wouldn't necessarily have that feature. Thus, even though you thought the drawings were the same, there were sufficient detectable differences for the AI to recognize.
I couldn't tell the difference between my basket and most of the successful baskets, but the robot didn't like my drawing.
For reference, here's my carrier
The drawings at best look like churches (a building with a cross on top), but most just have giant crosses on the side.
I guess if I was trying to doodle one, I'd do something like this: http://med-fom-emerg.sites.olt.ubc.ca/files/2012/06/surrey.j...
A carport with ambulances parked in front.
I'm also amused that one of the aircraft carriers has wheels :)
It sounds reasonable to call the Crawler-Transporter  a spacecraft carrier. From there it's not a big jump to wheeled aircraft carriers
But I'm probably giving people too much credit here.
After sometime I start scribbling randomly. Then it says "I see motorbike". And, wow! that was a motorbike. I din't see a motorbike in the drawing until it told me what it resembled. I didn't freak out, but closed the window immediately and told myself - "its a machine, I'm a programmer, I know how NNs work and there's nothing to worry about". The kind of feeling non-techies have when they experience predictive-text for the first time and they they say "it knows what I am thinking"
Nice try, Skynet.
I was asked to draw 'eyeglasses', kept guessing 'glasses'.
I guess it would have been more of an A.I. challenge if the premise was; draw anything and I'll try to guess what it is.
It could figure out most of my drawings but it would get them well in advance of me completing anything substantial (like others, I would be asked to draw a leg and draw just a curved line and it would guess leg before I finished).
Trying to draw what it asked for but with some unusual features (like lines or dot patterns around what it asked for before drawing it) and it gets extremely confused; it doesn't really seem to be good at filtering out any noise: http://imgur.com/a/oE1j2 (gallery of results and what it thought it saw.
Drawing things it didn't ask for just to see what it was guessing resulted in some really strange responses and fits. The answer set it has is extremely limited, so something like a hand giving the horns (\m/) was last guess a duck. A moose was a scorpion, then a duck, then a hand. Godzilla (or a bipedal dinosaur if you prefer) was a vase, then a scorpion, then a boat. My loaf of bread was a washing machine, an anvil, then a postcard. The Deathstar was a bandage, a helicopter, and a lighthouse. And a chainsaw was considered an aircraft.
Between the disruptive patterns and drawing things outside of it's vocabulary, the system seems really confused. Looking at the comparison results, I can see how when drawing some things it got it real fast. (Tennis Rackets were mostly defined by a crosshatch pattern, Harps by a series of parallel vertical lines). This makes sense. For other things, not as much.
It might be a more convincing presentation to give the user a list of items the machine knows (the full list) and tell the user to try to draw some, and then the computer could check it off as it gets them. That seems like a better way of presenting this than "Draw a box. hey! you drew a box! Isn't that cool?"
To me this is another interesting distinction on the NN recognition versus a human recognition - QuickDraw having a limited "vocabulary" to refer to really highlights this, as does my own lack of knowledge of Roobarb. Some of these things can really blindside us, and I suspect that it's going to require a lot of human hand-holding for awhile for the machines to get a strong vocabulary.
For some time I've been pondering how far you could take a machine's tabula rasa learning, for something like language, and how closely it would mimic a child's learning. (Language, color, math, etc).
Also, I realized how incredibly hard I suck at drawing.
This. I drew an Ant but the system couldn't guess it. Later it showed me how normal people draw ants.
Man, i suck at drawing ants
It should give you 8 ~10 words to choose from.
It also seems to constantly have a "best guess" to some degree and if that happens to be correct it confirms pretty quickly.
EDIT: Drew a "cake", it guessed "birthday cake". Wrong answer apparently.
So if you draw a simple shape it start to go trough the list of things he recognize and end the game there.
It works great for this game because he can have very fast answer but only work win the cases when you actually have a feedback that eliminate all the wrong guesses.
Basically if it simply went trought the whole english dictionary fast enough he could get the same result without even looking at the pictures.
As a designer/amateur artist, the most interesting part about this to me was seeing other people's drawings/most-basic-concepts-of-a-thing drawn out: which lines they draw, which ones they don't, how they express an idea in the most essential way, basically. (the 20s really helps)
It's a great window into how people think about things.
As a reference: I couldn't understand why it didn't recognize my keyboard (http://imgur.com/fXk6Gg9) but then it hit me hard when I saw other people's drawings (http://imgur.com/J7Lqodw).
So it is not just how people think about things, but at least in part also how people think other people will have thought about things guided by seeing the examples of other drawings.
Did we all see the same picture book growing up? Are most righties drawing it one way and lefties another (to which I would be a counter-example)? the guitar from top-left to bottom-right didn't look wrong at all, but why were most drawing it the other way?
Also, no one drew an electric guitar. Shame on us humans. This is fascinating to me more for the human-view than the ML piece...
I am sorry, my Vs are not very consummate.
We've been seeing that since forever with human children raised on doctrines etc. that have no basis in reality.
When I briefly permitted my utter goober of an inner child to have his way, its top two guesses at the result were "syringe" and "cannon", which...really aren't quite as far off the mark as I'd have expected, I suppose.
(No, I won't try a swastika and report back. Immature as I occasionally permit myself to be, I do have some principle and taste.)
Although I am a bit disappointed by the fact that it did not recognise my amazing whale :(
Especially if you compare it to what it thought a whale should look like:
I guess this is like everything in AI: bad data in, bad data out.
That said, I'm a bit disappointed that it has a lot of trouble recognizing anything outside of "how people normally draw things" (especially if you go out of your way to do more than "the bare minimum", draw things at a different angle, in 3D, etc).
 Umbrella (wheel, binoculars, pizza): http://i.imgur.com/jr9mexS.png
 Fork (sword, paintbrush, baseball bat): http://i.imgur.com/3uLIPkH.png
 Envelope (phone, eraser, leaf): http://i.imgur.com/pemgKpD.png
There were lots of innocuous things like [house, teapot, dog, zebra, moustache] but there was also a substantial amount of things like [rifle, aircraft carrier, submarine] that are military-related. I sort of got the feeling that I was helping to train an AI that would later be deployed in combat. I played another ten rounds or so to see what else might come up, and aside from one or two repeats nothing did and I realized I was being silly.
omg, I just trained this thing to bust some ghosts
The only preprocessing done is likely size reduction - the QuickDraw canvas is quite high resolution, so it's probably scaled down with (just linear interpolation)
Similar complexities arise in relation to perspective (should 'bridge' be side-on or top-down?) and even something as obvious as homonyms (I drew 'nail' as the small metal thing you use to hang a picture, but I'm sure others would have drawn the thing at the end of a finger).
I guess my point is that this comes across very much as a game (maybe that's all it really is), which will probably give poor 'neural net training' results, as opposed to if it weren't a game - e.g. by increasingly the ridiculously short time limit. I'm sure the results will still be interesting, but I think they'll end up very abstract rather than very accurate - maybe we'll just 'invent' a new set of logograms.
Or is it? Beyond the most simple shapes, humans heavily rely on context for image recognition. Maybe a neural net should do the same to be successful.
When it couldn't even guess my envelope I was suspicious and disabled pb. Then found out it "live" guesses as you go!
* In Chrome, open the developer tools, and find the style for "#challengeword-text" (you can hover over the challenge word, hit right-click and then "inspect" to get there).
* Set that style to opacity: 0 or something similar. (Be sure it's the #challengeword-text style and not just the element.style)
* Then, in your console, type "Array.prototype.slice.call(document.getElementById('challengetext-word').getElementsByTagName('span')).map((v)=> v.textContent).join('')". You should see the clue word appear after that.
* On two screens, show the browser window with the game to other humans, and keep the console on a screen only visible to you.
* Keep repeating that JS command in the console to get the word every time the challenge word screen appears (you can just hit up and enter, if you've already typed it.)
* And... bam! Now you can do a human vs computer round.
For Quick, Draw specifically, I'm interested in a "search" functionality where I can input a term such as "computer" and see not only the average representation, but more importantly, the fringe.
The possibilities for creative brainstorming are tremendous.
After I did add the hole, it looked nothing like a steak to me. The only thing I could think of was to put a pan around it. As soon as I started drawing the pan it said, "Oh I know, steak". Makes zero sense to me.
Picture of my drawing:
Does that look like a steak to you? In no way.
Here are the rest of my badly moused submissions if you're interested. I literally had no idea how to draw asparagus.
I was pretty shocked it didn't figure out my pencil. In the last few seconds (I was already done) I started coloring in the eraser but it still didn't get it.
Just did another one. Shocked it didn't get this!!:
Completely obvious what it is. First I drew the first 3 circles. It didn't get it. I added flashing. Didn't get it. I added a pole. DIdn't get it.
Added a car and arrow. Still didn't get it.
Everyone else just had in the center of their image, the three circles. Some people included "shining" lines.
(third from bottom and second from bottom rows.) Obviuosly mine looks a LOT more like a traffic light than these other people's, because I contextualize it!
The single time it worked without freezing it nailed correctly what I drew though :)
EDIT: tried again
performance still bad... but I saw one interesting, it requested me to draw a Knee, and... failed to recognize it.
Except after it ended and I went to see other people drawings, mine was identical, but flipped! I guess because I am left-handed or something like that...
Still, it is funny that it failed hard to recognize a flipped drawing.
As for the explanation: everyone drew a leg, seen from the side, slightly bent (me included). But I drew like this ">" while everyone else drew like this "<"
EDIT 2: someone asked me why it is not in portuguese :(
also I saw the reply below, I am on a Chrome on OSX right now.
And I'm done.
So nobody do that, right.
Seriously, don't even think about it.
(More seriously, I think I might have been de-trainging it by trying to draw the correct drawing using a touch-pad... rather too many of my phones seem to look like tornados by the time the dang thing's got stuck in draw mode trying to click-and-drag to draw)
We are so far from general AI. It's so strange how our brains can refine existance down so easily... how many CPUs is the human brain worth as far as we know?
It also didn't get my hexagon.
My bathtub looks too ridiculous for it to be able to guess it with confidence.
I'm not sure if it's because I'm left handed, but I tend to draw everything backwards from all the examples that the bot has seen.
I also think that the other examples it's seen are horribly drawn in many cases, making me think perhaps a 1 minute time frame would yield better results. I can't draw anything meaningful in 20 seconds haha.
The microsoft chat experiment from a few months ago shows that people can hack these systems to train them into racist slur-spewers.
Similarly, i wonder how this project aims to get accuracy given that many people are going to draw the porno objects no matter what the prompt was to mess with it.
This must be using a relatively small corpus of terms to guess from.
I also noticed it was very laggy on my phone, making it hard to draw anything before the end of the countdown.
Amusingly, the one I had drawn looked pretty similar to one of the examples drawn by other people.