
The Promise of Hierarchical Reinforcement Learning - ghosthamlet
https://thegradient.pub/the-promise-of-hierarchical-reinforcement-learning/
======
canjobear
What I find more promising is methods where hierarchical perception and
control emerge naturally from information bottlenecks.

Eg
[https://www.frontiersin.org/articles/10.3389/frobt.2015.0002...](https://www.frontiersin.org/articles/10.3389/frobt.2015.00027/full)

------
guineamax2000
I think I read about three articles like this per day.

Still haven't seen anyone actually using machine learning in any practical
application yet, anywhere. Lots of neat toy demos, no real work seems to be
getting done.

~~~
jononor
All the useful machine learning is under the hood of a product. Google Search
and Google Translate alone are used by hundreds of millions daily. Probably
millions use voice control of Siri/Alexa/etc weekly. Amazon, Facebook, Twitter
all use machine learning for recommending content. Billions of people use
those monthly.

It is not just practical, it is a huge industry.

~~~
guineamax2000
Converging everything towards the median (google results) is not machine
learning, and is making search results and the internet far worse.

Speech recognition on my phone hasn't improved much since Dragon in the 90s.
I'd argue the tech today is worse because it never works without a network
connection, so it basically never works unless you're in a big city.

Google translate is a running joke. I don't think that I'm allowed to comment
further on Google's use of machine learning without violating NDAs, but
manually fiddling with weights until you get the magic number you want is not
machine learning.

Again, not really seeing any benefit. Lots of hype though.

~~~
whymauri
You're just pulling a No True Scotsman. I'm surprised someone replied to you
in good faith.

~~~
scottlocklin
He's very much not engaging in any rhetorical fallacy: it's an uncommonly
clear observation of the actual reality of the current year in machine
learning. Which probably means he labors in the field.

The main businesses which seems to truly depend on ML (other than maybe FICO)
is that of tech journalist and PR dweeb.

~~~
wenc
Are we talking about ML or DL in particular? Also, does ML have to be
sophisticated in order for it to "count"?

Just off the top of my head, 3D/HD mapping companies like HERE rely on DL-
based image recognition to recognize objects at reasonable levels of accuracy
-- totally infeasible to do manually at scale.

Also, one of the triggers of the renaissance of NNs is that it was shown to
outperform traditional computer vision techniques around the early 2010s. USPS
does handwriting recognition every single day -- maybe not with DL -- but
definitely with some ML algorithm (I was at a talk given by one of the
originators of said algorithm).

ML is more than just DL. If take the definition of ML encompass to statistical
learning -- which it traditionally does -- production ML deployments is
extremely pervasive, from industries as varied as finance to chemical
manufacturing.

The exact FICO algorithm is proprietary, but algorithms of its ilk are
pervasive. As you know, at least two other companies (Transunion and Experian)
also have their own algorithms. And credit scoring algorithms are among the
least sophisticated ML deployments.

I'm not able to talk about the ML models I work on, but they're in production
and the business relies on them.

I guess I'm not seeing the finer points of the argument -- as it stands
without further qualification or refinement, it does not seem to be a valid
conclusion.

~~~
scottlocklin
I haven't spoken to the Here guys lately, but while they did hire a bunch of
DL weenies, it looked like they mostly put them to work doing more useful
things.

I'll say it again: no company depends on machine learning, other than, maybe
Fair Isaac. Many use such things. They don't depend on ML. IMO the trend is in
the other direction; many companies will begin to realize what they've spent
on models isn't worth the returns.

