Words with similar vectors have similar meanings. You use this in search, sentiment analysis, topic detection, finding similar text, and classification.
Of course there are tests for this "words with similar vectors have similar meanings" property, and I'm finding that the fastText vectors aren't doing that well on them, especially outside of English.
I'm glad they released them, particularly so anyone can run a fair comparison between different word vectors, but these things should come with evaluation results the same way that code should come with tests. These vectors are performing worse than ones that my company Luminoso released last year [1] (a better, later post is [2]), and if you don't believe me plugging my own vectors, I know that Sapienza University of Rome also has better vectors called NASARI [3].
fastText covers more languages, but most of these languages have no evaluations. How do you know the Basque vectors aren't just random numbers?
I think that performance hits a plateau when the vectors only come from text, with no relational knowledge, and especially when that text is only from Wikipedia. Text exists that isn't written like an encyclopedia. Meanings exist that aren't obvious from context. My research at Luminoso involves adding information from the ConceptNet knowledge graph, producing a word vector set called "ConceptNet Numberbatch" that just won against other systems in SemEval [4], a simultaneous, blind evaluation. The NASARI vectors are also based on a knowledge graph.
We evaluated using ConceptNet Numberbatch but in the end went with fasttext because of the treatment of OOV words using sub word information. This is important for us because we work with Social Media where misspellings are very frequent and we have found this helps a lot. Are you also looking into these sort of enhancements? How do you usually deal with OOV words?
Our OOV strategy was pretty important in SemEval. The first line of defense -- so fundamental to Numberbatch that I don't even think of it as OOV -- is to see if the term exists in ConceptNet but with too low a degree to make it into the matrix. In that case, we average the vectors from its neighbors in the graph that are in the matrix.
For handling words that are truly OOV for ConceptNet, we ended up using a simple strategy of matching prefixes of the word against known words (and also checking whether a word that's supposed to be in a different language was known in English).
fastText's sub-word strategy, which is learned along with the vocabulary instead of after the fact, is indeed a benefit they have. But am I right that the sub-word information isn't present in these vectors they released?
There's a paper on the SemEval results that just needs to be reviewed by the other participants, and I'm also working on a blog update about it.
I think there have been some attempts to explain, none of them fully satisfying.
But there have also been systems that create vectors with that property more directly instead of as a side-effect of a neural net. Examples include Stanford's GloVe, or Omer Levy's Hyperwords, which is made entirely from old-school ideas such as mutual information and dimensionality reduction, and was for a while the best system if you limited it to fixed training data (still Wikipedia).
A gripe: reviewers don't even seem to like explanations. If you can explain your system with well-understood operations, it's boring and "not novel". But when Google publishes magical mystery vectors, they lap it up.
Edit: I see that I didn't answer your question at all. However I'll leave it here for people who are not that familiar with ML
If I'm not completely wrong, these are so called latent factors of words. That pretty much means computer representations of the meaning of a word.
Words with similar meanings would have similar factors, for example the word "Rome" and the word "Italy" will probably in one or more of these dimensions be quite similar.
These vectors usually take a lot of time to train if done properly, and they come out quite similar anyway, so having them precomputed makes it easier for other people without the resources of fb to do NLP.
Another cool thing is that they are available for so many languages, this is the first time I've seen precomputed vectors for my native language.
TLDR: computer representations of words, which makes it easier for people to make machine learning models.
[EDIT] I reply to myself here: https://news.ycombinator.com/item?id=12226988