

Google Is Working on a New Type of Algorithm - gobinath-mani
https://wtvox.com/robotics/google-is-working-on-a-new-algorithm-thought-vectors/#.VeJmdkCIaBk.twitter

======
cs702
This is not about word2vec, an algorithm for learning vector representations
of words.

This is about learning vector representations of _thoughts_ (ideas, concepts,
etc.) in an unsupervised manner.

For reference, two months ago some of Hinton's colleagues at the University of
Toronto published a paper describing an unsupervised algorithm for learning
vector representations of sentences:
[http://arxiv.org/pdf/1506.06726v1.pdf](http://arxiv.org/pdf/1506.06726v1.pdf)

Once trained, this algorithm is given a sentence and asked to find similar
sentences in a corpus of half a million sentences, and it produces impressive
results. For example, when asked to find the sentence with most similar vector
representation of sentence 1, it finds sentence 2:

* Sentence 1 (sentence given to the trained algorithm): "He ran his hand inside his coat, double-checking that the unopened letter was still there."

* Sentence 2 (sentence with most similar vector representation found by algorithm in corpus): "He slipped his hand between his coat and his shirt, where the folded copies lay in a brown envelope."

The paper has more examples. The vector representations learned by this type
of algorithm truly appear to be capturing the meaning of sentences.

This looks promising.

\--

Edit: Code used in the paper I mention is/will be here:
[https://github.com/ryankiros/skip-
thoughts](https://github.com/ryankiros/skip-thoughts)

~~~
JD557
> This is not about word2vec

Thanks for the clarification. I actually thought they were talking about
word2vec.

The "Paris - France + Italy = Rome" example is really similar to other
word2vec examples.

------
Someone
More info (references, source code, data sets) at
[https://code.google.com/p/word2vec/](https://code.google.com/p/word2vec/)

Edit: I couldn't figure that out from the text, but according to a comment by
cs702, this isn't about word2vec, but to me, it seems 'just' a more advanced
version of it.

------
sctb
Previously:
[https://news.ycombinator.com/item?id=10100910](https://news.ycombinator.com/item?id=10100910)

------
fauigerzigerk
_> "If you take the vector for Paris and subtract the vector for France and
add Italy, you get Rome"_

I would have liked to hear more about the parts that are not purely based on
associations between words given some context. Associations only get us so
far. Eventually, we will need to extract logic as well.

------
tajen
Very interesting. It is based on a technology of semantics or is it a neural
network where each neuron represents a concept?

~~~
PaulHoule
It sounds like this is something like Word2Vec on steroids, i.e., some kind of
dimensional reduction based on seeing lots of instances.

It is an interesting idea but my suspicion is that it is a way to get where
current NLP technology is with less work but it won't quite "cross the chasm"
to produce commercially useful systems.

Deep learning is an academic bubble that is going to sweep through the
problems that is good at, but 20 years from now people are going to look back
it like many look at symbolic AI, which swept through the problems it was good
at and then ran out of steam.

------
petepete
Pity the article is so frustrating to read.

[http://i.imgur.com/youRjpv.png](http://i.imgur.com/youRjpv.png)

