

Two definitions of a kernel? Why "kernel trick"? - snippyhollow
http://syhw.posterous.com/two-definitions-of-a-kernel-why-kernel-trick

======
daniel02216
This is a great example of why mathematicians have difficulty getting others
to use and understand their work. What the OP says is perfectly true, yet
completely incomprehensible unless you already know what he's talking about,
at which point you don't need to read what he's saying.

~~~
snippyhollow
I'm not claiming to explain why kernel machines are called this name to the
layman. I just think it could be beneficial to people with a background in
mathematics or CS (BA would suffice I'd say) that are interested in ML. If one
is interested in ML (particularly kernel methods) but does not want to learn
optimization and algebra, one will not be able to use (perform ? ;)) kernel
tricks. I would have loved to read that during my Master for instance. (But
perhaps it is just me scratching my own itch... Not a big deal anyway.)

~~~
Figs
I'm an undergrad CS major in my fourth year, and I have no idea what you are
writing about, although it sounds like it might be interesting or useful if I
understood it. The first stumbling block came right away, since I don't know
what the notation you used here is supposed to mean:

"In algebra (kernel of a linear transform, of a homomorphism, of a matrix), a
kernel is the set of elements that map to the neutral element. T : V → W
ker(T) = {v ∈ V | T.v = 0}"

What is T.v? Is 0 the "neutral element"? What is meant by neutral element? (I
am not an expert at algebra, since I am a CS major, and my major has been
focused more on things like compilers, machine architecture, programming
languages, algorithms, etc. rather than machine learning) What does V -> W
ker(T) mean? I assume you're saying something like "T is a kernel of a
function that maps V to W if T.v (where v is a V) is the neutral element", but
this means nothing to me. Aren't you trying to explain what a kernel is...?
How is the reader supposed to understand this if they don't already know what
it is?

~~~
snippyhollow
T is a linear transform/map or (it is the same thing!) a homomorphism, or a
matrix: an application between 2 vector spaces that preserves vector addition
and scalar multiplication (i.e. f(x+y) = f(x)+f(y), f(a.x) = a.f(x)).

T : V -> W is the signature of this homomorphism, whereas ker(T) = {v \in V |
T.v = 0} means "ker of T is the set of vectors v in the vector space V such
that T applied to v is the neutral element of W". The neutral element is noted
0 because, in a vector space, it is such that v + neutral_element = v. So
neutral_element is the null vector.

I learned algebra kernels in my first year of college, so I assumed wrongly.
But that's not the point: I was not thinking about explaining what a kernel is
but instead (higher level) trying to group what seemed to be scattered
definitions into explaining why we call "kernel machines" this (and why a
kernel trick involves a kernel).

Now, I'm thinking about explaining (with pictures) what an algebra kernel
is...

~~~
snippyhollow
Here it is: <http://i.imgur.com/8V7Xg.png>

~~~
ableal
I'd suggest tossing that figure into the original post. It's a nice visual
hook.

------
colanderman
The video you linked "explains" it very well to visual learners such as
myself: <http://www.youtube.com/watch?v=3liCbRZPrZA>

If a picture is worth a thousand words, a second of video is truly worth
29,970 words.

~~~
snippyhollow
This video helped me too. Understanding something with one explanation is very
good, but making the link with a second and a third one is even better.

