
Something About Cats, Dogs, Machine and Deep Learning - skazka16
http://kukuruku.co/hub/image-processing/something-about-cats-dogs-machine-and-deep-learning
======
dkarapetyan
So at what point do any of these tools build a naive model of 3d and
projective geometry. I don't think when people look at pictures all they are
doing is extracting 2d features because in my head I am imagining some kind of
3d space and placing things in them according to how the picture shows them.
One obvious reason pictures with weird angles and perspectives are hard to
understand is because I can't properly orient myself in the made up picture
world.

Proper AI or deep learning or whatever it is that is the fad these days should
account for this kind of model building that brains are good at. Looking at 2d
pictures and only extracting 2d features doesn't feel like any kind of model
building but more like really good data mining but deep data mining doesn't
sound as sexy.

~~~
anigbrowl
An excellent summary. I believe that a lot of the work we're doing on NN
classifiers is based on a rather shaky notion of bootstrapping a collection of
platonic ideals, one at a time.

Lately I've been thinking about this from the opposite perspective, as I have
a lot of cats and a large and rather neurotic German Shepherd (all rescue
animals that basically came up to us). I mention the dog being neurotic
because he went through a phase of reacting to _anything_ in his size class,
such as rocking horses and small goods trolleys, and is still inclined to
react to things like his own shadow or reflection in a window if it catches
him unawares. He's very interested in the cats but given his size and volume
they don't appreciate his attentions and I worry more about him catching a
stray claw in the eye than I do about him biting one of them, so currently
they have to admire each other from a distance. I spend a lot of time with the
dog every day (partly because my wife isn't able to handle him physically) and
we also look after and train a neighbor's Shepherd pups.

Not being a dog person I've had to spend a lot of time learning how to think
like a dog, or model the dogs' model of the world, and make gradual
modifications to it, often by subverting existing behaviors. ISTM that the
trouble with back-propagation algorithms in NN classifiers is that we are
establishing and then polishing a recognition reflex, and at best this is
going to yield only insect-like behavior. Now such hardwired behavior in
insects can be pretty elaborate and impressive, more so in social insects; but
it's an inherently reactive model. Consider instead a neural network with two
outputs, GET and IDENTIFY. The IDENTIFY part is more or less the same as what
we have now; when stimulated it is trained to classify input as DOG or CAT,
and if it does so correctly we 'reward' it by propagating those output
weightings back into the network.

But I'd like to see (or be made aware of) one that also tries to GET input,
either input it has already classified or new and therefore 'exciting' input.
My dogs sometimes want predictable rewards, such as snacks, access to the
outside, or various kinds of play (so do my cats but they have different and
rather more zero-sum approaches to getting what they're after). But they also
need a degree of novelty and mental challenge which requires them to expand
their world model. I've been experimenting for a while now with hide-and-seek
games using a ball for the oldest dog and (casually) monitoring how he divides
his time between iterating over known hiding locations and novel ones,
sequences of known locations and so on. Of course he can 'cheat' by using his
sense of smell to rack where I've been, but I've managed to train him out of
that by using various cheats of my own to make that unreliable.

My takeaway from this is that successful world-modeling is not its own reward
but seems to be emergent from a framework of interdependent reflexive
behavior, and require a temporal as well as a spatial dimension. Indeed, one
of the trickiest aspects of this game was getting the dog to wait for me.
Right now I have to show him the ball, then have him stay while I wander
away(completely out of his sight) for some indeterminate length of time,
following which I return and tell him to start searching, which I may or may
not monitor directly. So he has to model both that I'll be coming back _and_
that I've left something in the environment for him to retrieve. He worked
this part out quite quickly and it took some extra time to get him to wait for
my say-so before doing the retrieval as soon as he sees me reappear.

I feel that for our machine learning progress to advance we need both first-
order classifiers of the kind described in this article and second-order
classifiers that are orthogonal to the first and classify the first-order
classification events into meta-patterns.

I'd be grateful if anyone could point out work already being done in this
area. I don't keep up with the academic literature in any sort of organized
way so this is probably a case of reinventing the wheel.

------
taliesinb
There seems now to be a cottage industry producing fuzzy, non-technical
articles about machine learning and particularly deep learning. I guess we are
now in the rapid inflation phase of the hype cycle.

~~~
bluebur
I thought that was a very well written article, with code examples and
results. It was describing an interesting approach to a classification problem
in a way that you could follow and reproduce.

~~~
taliesinb
You're right. That was unfair and mean. The article is actually pretty good --
as you say, nice examples, code snippets, and some history -- and I shouldn't
have implied that it falls into the category of fuzzy and non-technical
blogposts. The latter category does seen to be a growing thing, though.

I suppose people at all levels of discovery are excited by what they're
learning, especially because we all possess a brain and it is intriguing to
begin pulling back the curtain of how it might work.

