

How do machines learn meaning? - benjaminwilson
https://blog.lateral.io/2015/06/how-do-machines-learn-meaning/

======
fsargent
This isn't learning by any means - this is just similarity. Machines don't
understand what the words mean, nor the nuance of how to use them via this
method.

~~~
tensor
You say this as though you have a definition of what it means for something to
"understand" words. It's easy to make a claim, why don't you try to back it
up?

~~~
sqrt17
usually, we assume that a meat grinder does not "understand" meat, that a rice
cooker does not "understand" food, and that a telephone or a CD player does
not "understand" human language.

Wiktionary terms "understand" as (i) be aware of the meaning of something (so
we've just reduced it to the meaning of awareness and meaning), or (ii) to
impute meaning that is not explicitly stated.

To put it in more colorful terms, a blind person can understand the difference
between green, yellow and brown bananas, but they usually cannot understand
the visual aspect of color.

By the same rationale, a vector does not understand the word it describes any
more than the telephone book understands the people listed in it or the city
that they live in.

~~~
tensor
We are not talking about just a vector, but rather vectors trained in a
specific way along with cosine similarity. You could easily argue that this
system can tell you that one words has a different meaning from another word,
or that two words have similar meaning. Further, it learned this without these
relationships being explicitly stated.

Once you start to try to define aspects of cognition explicitly, things very
quickly get ambiguous. Also, these conversations usually go along the lines
of:

1\. State a definition.

2\. See that computer matches.

3\. Decide it's wrong after all and try to change it so that computer can't
match it.

4\. Repeat until we find a definition that excludes computer.

I think it's a fascinating topic, but the above pattern is fairly
disappointing.

------
alexchamberlain
I'm not ML expert (especially in the natural language space), but I can't help
thinking something is missing here. I know a ship 'is-a' boat. Cats and dogs
are unrelated animals etc. Maybe the colocated words are a very good
approximation of a dictionary or real meaning?

~~~
slashcom
So this is my research area.

Distributional vectors are a proxy to word meaning. Words with similar vectors
have similar meaning or are semantically related in some way. But usually, you
just measure similarity by a single number: the cosine similarity between the
two words' vectors.

This number can tell you words are related or not, but it can't tell you _how_
they're related [1]. There's been a good deal of work in automatically
identifying that "ship is-a boat" (which is called hypernymy) or cats and dogs
are unrelated animals (cohyponomy), but it's still being perfected.

But it is useful. Words that are similar in meaning can be treated similarly.
As a bad example: maybe I know that "anger" has negative sentiment, but I
don't know what sentiment "furious" has, but I can infer it probably has
negative sentiment since it's so similar to "anger".

[1] There's a good deal of evidence that words that have high cosine
similarity are more likely to be cohyponyms.

