
Tomáš Mikolov on Word2vec and AI research at Microsoft, Google, Facebook [audio] - tim_sw
https://rare-technologies.com/rrp-1-tomas-mikolov-on-word2vec-and-ai-research-at-microsoft-google-facebook
======
cs702
Great podcast.

The world owes a big THANK YOU to Tomáš Mikolov, one of the creators of
Word2Vec[0] and fastText[1], and also to Radim Řehůřek, the interviewer, who
is the creator of gensim[1].

The number of software developers and researchers in industry and academia who
rely on the work of these two individuals is large and growing every day.

[0] [https://code.google.com/p/word2vec/](https://code.google.com/p/word2vec/)

[1]
[https://github.com/facebookresearch/fastText](https://github.com/facebookresearch/fastText)

[2] [https://radimrehurek.com/gensim/](https://radimrehurek.com/gensim/)

------
faizshah
Interesting podcast, I wished they talked more about how he came up with
word2vec and how he went from the initial idea from the Czech language (I
think that's what he said) into the word2vec we all know.

I also liked how this podcast was conversational like the interviews in the
talking machines podcast. I look forward to the future episodes.

~~~
nl
I think most people who have worked in computational linguistics or natural
language processing for a while would have some idea of words existing in some
kind of vector space.

The things which still amaze me are that meaningful vector operations work
(Queen + Man ~= King), that you only need 300 dimensions (or sometimes even
less!) and that it is possible to build this vector space so "easily".

------
andrewshadura
So, what is this word2vec, after all?

~~~
quonn
After putting some documents in word2vec, it gives you a vector representation
of each word (1). Words that are close to each other in this space might be
similar, for example. Famously, it's possible to do neat computations, e.g.
king - man might result in a vector that's close to the one for queen.

(1) This representation is learned, essentially, by trying to predict words
from the surrounding words in a sentence or the other word around.

~~~
versteegen
You need to compute king - man + woman, since, in a well learned model, the
king - queen vector is similar to the man - woman vector.

These gender-based vector differences can be explained as capturing syntactic
rather than semantic information, but semantic relations (Rome:Italy ->
Paris:France) can also work.

