

The Search-Recommendation-Notification Spectrum - nikhilbd
https://medium.com/@nikhilbd/the-search-recommendation-notification-spectrum-66c3f8fd5030

======
PaulHoule
The trouble here is that the technology is "good enough" for the first two but
the last one is just a little bit beyond the state of the art.

~~~
storgendibal
Do you mean that it is hard to determine user intent, context, etc. in order
to push highly relevant notifications to a user?

~~~
PaulHoule
What I can say is (i) I don't see anybody succeeding at this an academic
environment (TREC took two years to get better than chance, but this nothing
to be proud of) and (ii) I don't see anybody succeeding at it commercially.

It's very connected to the problem of scoring in search. In almost all IR
systems, the relevance score is meaningless except in that if you sort by it
descending you will see results that get worse as you go down the list.

What you really want is to get a probability score as a relevance value and
you can convert a conventional score into that with logistic regression but if
your system is honest you wind up with 70% or so precision at your maximum
score and that is not so good, particularly in the filtering scenario.

I'd imagine a system based on today's machine learning techniques might do
pretty good if you supplied a training set of 1000 or so articles (think of it
as developing a classifier) and you could probably use dimensional reduction
to do better-than-chance with 50 articles, but I think most people would
expect to be able to learn from 10 or so examples and would judge the system
"unintelligent" if it can't do that.

~~~
nikhilbd
Notifications, as an active research area, just hasn't developed yet. It's
mostly because the concept is very new. But expect much more research and much
smarter commercial systems in the next few years. I won't be surprised if
notifications become our primary mode of interaction with the digital world.

Also, the training data easily goes into thousands or even millions of samples
for even modestly used applications. Unless you mean train a different model
per user, in which case, yeah, it won't work.

