
Context Is Everything: Finding Meaning Statistically in Semantic Spaces - ghosthamlet
https://arxiv.org/abs/1803.08493
======
Radim
The article seems a little rough (draft?), but the money shot is the algo in
Section 5. Looks pretty neat and straightforward!

TL;DR:

1\. Build a special distance(word1, word2) metric using a set of word vectors
trained elsewhere (such as GloVe). This distance works better than cosine
distance.

2\. Given a document (a sentence, a paragraph… basically, a sequence of
words), calculate the "importance" of each word as a sigmoid over
distances(word, avg(all_words)).

3\. To embed a document, simply do a weighted average of its word vectors,
where the weight of each word equals the importance above.

~~~
ContextArxiv
Hi, author here! I can confirm everything you said. A conference version is in
the works, though this one is essentially complete

~~~
braindead_in
Do you plan to release an implementation? I would definitely like to play
around with it.

~~~
ContextArxiv
Absolutely. The current priority is conference submission, but releasing a
simple interface is definitely the plan, as appropriate

~~~
ContextArxiv
[https://github.com/ezelikman/Context-Is-
Everything](https://github.com/ezelikman/Context-Is-Everything) The code will
be posted here when available!

------
sizzle
Very cool. Any interesting uses for this? I want to develop a 'cognitive
debiasing' ML system. It will take real time spoken languages and parse it
syntactically to output a dibiased version of the input. Let me know if it
have any resources or insight on this. Thanks!

~~~
3131s
By "debiasing" do you mean word sense disambiguation?

~~~
sizzle
Not sure what the technical term is but by debias I meant the ability to
detect cognitive biases [1] from real time spoken input and remove them from
the output. The goal would be to help humans understand their biased thinking
and over time make us less biased in our decision making process.

[1]
[https://en.m.wikipedia.org/wiki/List_of_cognitive_biases](https://en.m.wikipedia.org/wiki/List_of_cognitive_biases)

~~~
rhizome
Good luck!

~~~
sizzle
Thanks! Even if I get could a few smart individuals to put together a 'bias
detector' mvp that would be an awesome portfolio project!

