
Extreme Learning Machine - sytelus
https://en.wikipedia.org/wiki/Extreme_learning_machine
======
finndark
"ELM" = "Extreme Learning Machine" is apparently heavily based upon previous
work, if not outright plagiarism. The Wikipedia article on ELM discusses this
only briefly:

[https://en.wikipedia.org/wiki/Extreme_learning_machine#Contr...](https://en.wikipedia.org/wiki/Extreme_learning_machine#Controversy)

Yann LeCun' opinion on ELM: _' What's so great about "Extreme Learning
Machines"?'_:

[https://www.reddit.com/r/MachineLearning/comments/34u0go/yan...](https://www.reddit.com/r/MachineLearning/comments/34u0go/yann_lecun_whats_so_great_about_extreme_learning/)

A sort-of "How-to plagiarize successfully" guide including "Easy but Proven 5
Steps to Academic Fame" with Huang's work as a gleaming example:

[https://elmorigin.wixsite.com/originofelm](https://elmorigin.wixsite.com/originofelm)

Two very detailed plagiarism analyses of two of Huang's papers. They are quite
sarcastic, funny and enlightening:

[https://docs.wixstatic.com/ugd/5256f1_ac3a0f6b08524a3ca6101d...](https://docs.wixstatic.com/ugd/5256f1_ac3a0f6b08524a3ca6101d4f3b74f54e.pdf)

And finally, as one poster says, the provenance of "Deep Learning" itself has
some controversy:

[https://www.researchgate.net/post/Why_Extreme_Learning_machi...](https://www.researchgate.net/post/Why_Extreme_Learning_machine_is_not_so_popular_as_Deep_Learning)

 _" But, just recently I also found that there is also a problem with the
inventorship of Deep Learning. [http://people.idsia.ch/~juergen/deep-learning-
conspiracy.htm...](http://people.idsia.ch/~juergen/deep-learning-
conspiracy.html) For this, I post a question for discussing these issues.
[https://www.researchgate.net/post/Any_enlightment_on_the_ori...](https://www.researchgate.net/post/Any_enlightment_on_the_origin_or_inventorships_of_ELM_and_Deep_Learning)
[https://docs.wixstatic.com/ugd/5256f1_556b42c00d0d4d5bb199fb...](https://docs.wixstatic.com/ugd/5256f1_556b42c00d0d4d5bb199fb2e45def79f.pdf"*)

~~~
screye
As much as people dislike Schumidhuber for being bit of a curmudgeon, his
contributions have been undermined in the Deep Learning community at large. He
did not deserve to be shafted for the Turing award.

His contributions were as impressive as the other 3, and a lot of it was done
in isolation away from the support of the NA AI/ML community.

------
lenticular
ELMs are just neural nets with random hidden weights. This idea has been
around since the 60's before being rebranded to get unoriginal papers
published.

This is equivalent to a low-rank Fourier approximation of Gaussian process
regression (also known as random features) for some kernel. Random kitchen
sink approximations are similar, but better performing since the weight value
distribution actually has theoretical justification. RKS is a really great way
to use approximate Gaussian process regression on larger datasets.

------
stochastic_monk
I really enjoyed the discussion here [0] on Yann LeCun’s comments.

In some ways, it’s like random Fourier features/kitchen sinks (Cf. Ben Recht).
They’re more insightful than they are useful.

[0]
[https://www.reddit.com/r/MachineLearning/comments/34u0go/yan...](https://www.reddit.com/r/MachineLearning/comments/34u0go/yann_lecun_whats_so_great_about_extreme_learning/)

------
bigred100
Is this an example of reservoir computing?

~~~
T-A
The two first sentences of the respective Wikipedia article answer that:

[https://en.wikipedia.org/wiki/Extreme_learning_machine](https://en.wikipedia.org/wiki/Extreme_learning_machine)
"Extreme learning machines are feedforward neural networks"

[https://en.wikipedia.org/wiki/Reservoir_computing](https://en.wikipedia.org/wiki/Reservoir_computing)
"The reservoir consists of a collection of recurrently connected units"

So, no.

~~~
bigred100
The ideas seem quite relate. A common reservoir computing setup involves
learning a linear map from a back box dynamical system (which can be a
feedforward network if we really want it to) to some output. The only
significant distinction I see from a short observation is that the input and
output size in reservoir models are the same from what I’ve seen

