We published a paper a while back in a genetic programming conference, and before the paper had been published (only the title was announced) Schmidhuber guessed that we had not cited his work. He very publically put us through the wringer for not citing him as the inventor of the concept were were examining. In fact we had not cited him: but among our dozen or so previous examples, we had cited two others which long predated his work and were the actual seminal papers. He had failed to cite them himself.
But it is possible that he ought to have been cited as well? It doesn't matter if he's a hypocrite. The justification ultimately has to be, "Our paper did not use your work in any substantial way therefore a citation is not needed."
I can see how citations can get very complex for the purposes of scientific credit, traditionally there's a self-policed aspect to it in every academic community; however, in this century as advanced research gets more complex and globalized, it might be time for a more rigorous and neutral process of doing so.
Whenever you cite papers just because you're "politically supposed to", even though you don't think it's really intellectually useful for anyone to be aware of the cited paper, you're making the citations less useful for everybody.
Schmidhuber is only the worst of many abusers of the citation system. Everyone knows that anonymous reviewers are more likely to approve of people who cite their work. Someone is politically powerful? Better cite their paper, even though you don't really think it's a good paper and you didn't use it.
Schmidhuber is cited far more often than he should be, because people just don't want to deal with this crap, and the cost of sticking in one more citation is very low.
> Whenever you cite papers just because you're "politically supposed to", even though you don't think it's really intellectually useful for anyone to be aware of the cited paper, you're making the citations less useful for everybody.
The audacity of deciding on behalf of readers whether information is valuable to them when publishing an academic paper is quite an interesting concept to me.
I'm not published in journals, but I do have a "professional" writing background. There's only a few ways I can imagine to read into your statement. Either you think you're above citation, or so much smarter than the reader you're qualified to decide on their behalf whether it's relevant, or simply hiding it for nefarious reasons.
Any of these options, or even the perception of them, is precisely the type of behavior fostering anti intellectualism and prejudice for academia.
It's also quite ironic you repeatedly mention "politics" while your entire argument seems to hinge on seeking to deny empowering further those already in power. Isn't that exercising political capital?
> ... the worst of many abusers of the citation system.
Is this ad hominem argument by user "lacker" meant to distract from the omissions of the awardees Bengio, Hinton, and LeCun? Especially the first two got tons of citations for work that should have credited Schmidhuber's lab: the analysis of vanishing gradients in neural networks, the principle of generative adversarial networks, attention in neural networks, distilling neural networks, speech recognition with LSTM neural networks, self-supervised pre-training, and more.
The disputes with LeCun are more recent and of lesser magnitude IMO.
My godfather is an academic that does some arcane nonsense (I mean that in the nicest way possible) involving topology and number theory. He told me a story about how he and his coauthor had a paper in review for 3 years because the reviewer insisted on adding citations to what they assumed were his own work because they were not particularly relevant. My godfather after a round or two didn’t have the patience for it and said whatever let’s just put them in by his coauthor refused hence the three years of back and forth.
>> Schmidhuber is only the worst of many abusers of the citation system. Everyone knows that anonymous reviewers are more likely to approve of people who cite their work. Someone is politically powerful? Better cite their paper, even though you don't really think it's a good paper and you didn't use it.
The reason that this is not the norm in academia is because there is a substantial number of researchers who care very, very much about academic integrity and even more so about getting their work properly recognised, just like good old You_again.
Btw, "everyone knows" is a running joke in Game of Thrones, it's what the Dothraki always say to show that they're pre-scientific barbarians who believe whatever they like, or at least that's how I get the joke.
Whether Schmidhuber is a bad actor taking advantage of the "system" in general is separate from the specific case I was responding to (not sure if that was your comment or somebody else's.)
But as I said before, we should take the long view. Globalized scientific research ought to have a more neutral way of assessing citations. The 20th century way of doing it may not be the right way any more. I don't see anyone else suggesting a problem of this scope.
> Schmidhuber is cited far more often than he should be
You haven't even read the paper, have you? Otherwise you'd see that it's Hinton and Bengio who are cited far more often than they should be. Just look at disputes B1, B2, B5, H2, H4, and H5 to see how they republished parts of his work again and again without citing it. No honest scientist can approve of something like that.
Our paper was an analysis of an already established technique used nearly identically by a number of previous papers. We dug up about a dozen IIRC, including the seminal ones, but we missed his. He didn't go after us for not citing him (which he had divined). He went after us for not acknowledging that he invented it (which he did not).
He really should be cited. The paper is supposed to give context to the reader, so to not cite a notable paper that is relevant is doing a disservice to the reader and it does seem pretty sketchy to me (I mean only if it's notable, idk who Schmidhuber is).
Sometimes this just happens by mistake. The literature is vast and people rediscover things fairly frequently that they honestly didn’t know existed. The related work section is a good faith effort to contextualize your work in the space of extant works. You will not know every work even seminal ones (it happens, especially if it came out a few decades beforehand), and you don’t have space to include everyone. Sometimes you’re coming from an adjacent field that doesn’t literature from other sources. I study a lot of control theory, and optimal control theory is something that shows up in engineering and economics. An economist writing on the topic may not be familiar with all the engineering literature. Recently, with the machine learning explosion, I’ve come across reinforcement learning papers from time to time where they rediscovered something from optimal control and are proposing it as a whole new thing because, well, they don’t know a lot about the control theory literature. Personally, I don’t think it’s a large problem because it brings the idea to a new audience and it’s impossible to know everything.
Another aspect that makes it difficult is that different people even in the same field use different terms for the same thing. As an example, engineering optimal control folks like to optimize cost functions, economists might refer to that as the utility instead, and RL folks like to use rewards (which is just a negative of a cost and functionally equivalent). That makes it difficult to run a search. I’ve come across a whole slew of new papers to read on topics just by changing out key terms for synonyms.
A reviewer may suggest other works to include in your discussion. Sometimes the recommendations are appropriate, other times it’s a backhanded way to fish for citations and the works suggested aren’t actually relevant. I would say harping over not including one is poor form unless it is so similar that is important for you to explain the novel aspect in your paper.
Are you talking about the 1985 Genetic Programming paper by Cramer? Unlike Hinton and Bengio, Schmidhuber has corrected himself:
> BTW, I committed a similar error in 1987 when I published what I thought was the first paper on Genetic Programming (GP), that is, on automatically evolving computer programs[GP1][GP] (authors in alphabetic order). At least our 1987 paper[GP1] seems to be the first on GP for codes with loops and codes of variable size, and the first on GP implemented in a Logic Programming language. Only later I found out that Nichael Cramer had published GP already in 1985[GP0] (and that Stephen F. Smith had proposed a related approach as part of a larger system[GPA] in 1980). Since then I have been trying to do the right thing and correctly attribute credit.
Some context for people unfamiliar with ML research:
the author, Schmidhuber, is well known for claiming that he should get credit for many ML ideas. Most ML researchers think that:
- He doesn't deserve the credit he claims, in most if not all cases.
- There's a few cases where his papers should have been cited and weren't. That's fairly common.
- People do not get much credit for formulating an abstract idea in a paper or implementing it on a toy problem. Credit belongs to whoever actually makes it work.
- Credit assignment in ML is not perfect but roughly works.
>> Credit belongs to whoever actually makes it work.
That is according to whom? Is it a rule you just came up with or accepted practice? And if it's accepted practice, in what community is it accepted practice? Because where I publish and review there's really no such rule and credit belongs to the people who deserve credit for the work they've done that was useful to others.
> credit belongs to the people who deserve credit for the work they've done that was useful to others
Certainly agree. The point is that coming up with the idea, writing it as an equation, or an architecture diagram in a paper, is a small fraction of the effort that goes into making the idea work in a model showing good performance on real life datasets.
For example, just taking a random paper that Schmidhuber claims should give him credit for GANs, https://people.idsia.ch/~juergen/FKI-126-90ocr.pdf hopefully you can easily see that a lot of work would be needed to turn this into a realistic image generation model. And that is, even if you admit that the idea is strongly related to GANs, which I'm not convinced of but won't spend time on.
> Credit belongs to whoever actually makes it work.
>> That is according to whom? Is it a rule you just came up with or accepted practice? And if it's accepted practice, in what community is it accepted practice?
It is accepted practice in the ML community. If it weren't, Schmidhuber wouldn't be complaining.
I agree a significant amount of work (and often insight too) is needed to translate an architecture idea into something that works in practice, and there are certainly plenty of ideas that are obvious in the abstract. But I also think it's important to avoid dismissing work only on the basis that it doesn't involve "real life datasets".
Deep learning is a relatively unexplored field and there are many open mathematical and scientific questions to ask that involve only model equations or contrived datasets. Novel theoretical results are not just about some architecture idea but about proving facts that can be useful for understanding how the model class would perform in different scenarios. Which in turn can help shape the search space for applied work.
Additionally, I don't think credit assignment should be so discrete. 100% agree that vomiting out vague ideas shouldn't grant claims to credit, but academic science much too often gives only a single author the "real" credit.
Incidentally, in other fields the person who actually makes it work very well may not be the person that receives this credit. Like biology can involve a lot of hard manual work (that isn't really intellectual) in order to realize a project plan. It varies how much of the credit those people receive, and I'm not even sure how much they should receive. This topic is extremely nuanced.
"Introductory theoretical work in GAN was done by Schmidhuber [1], but it was not until large experimental efforts [2,3,4] on image generations that the power of GANs was revealed."
I don't buy that this is standard practice in the ML community, and even if it is it's BS. If the basic idea/principle has been published previously but in a different context you should cite it and say why the solution is not directly applicable or has not been evaluated in the current context. Anything else is unprofessional.
Agreed. The piece anticipated this straw man argument:
> "the inventor of an important method should get credit for inventing it. She may not always be the one who popularizes it. Then the popularizer should get credit for popularizing it (but not for inventing it)." Nothing more or less than the standard elementary principles of scientific credit assignment.[T22] LBH, however, apparently aren't satisfied with credit for popularising the inventions of others; they also want the inventor's credit.[LEC]
Basically ideas are a dime a dozen. Sure, your idea might be a good one, but how do we spot your grain of sand is special when it looks the same as the rest of the desert? Essentially having an idea isn't useful to others. Demonstrating that your idea has legs is useful to others.
I don't have to deal with citing papers, but I once had to deal with people pitching me ideas, wanting me to sign an NDA, in exchange for 50% of the revenue after I did all the actual work. Just out of curiosity, I signed one once. It was a fart app, IIRC. They thought a fart app needed an NDA, and that I'd then go do all the work and give them 50% because they "had the idea". It was so laughably sad.
If you think these ideas are valuable, I have a beautiful clock for you. It is right twice a day. You'll have the same problem: you won't know when it's right. You'll need someone else's work to tell that.
There's a spectrum of ideas, from groundbreaking to "dime a dozen". In tech startups, and in almost all of computer science, most ideas are a dime a dozen, and the value is in the execution.
But clearly, some ideas are groundbreaking. Einstein rightfully gets the credit for an on-paper hypothesis that wasn't proved until decades later via a chain of critical discoveries and experimental innovations by other people. It's legit to call it Einstin's relativity, and not Mossbauer/Hay's relativity.
Ideas are a dime a dozen in the sense that the same idea will often occur to dozens of people, on a dime's worth of effort. Relativity theory wasn't anything like that. Einstein made predictions that no one else was making. When one of them from GR was confirmed a few years ago, Lenny Susskind famously marveled at the foresight, saying "they didn't call him Einstein for nothing!".
Problem then goes to how do I decide whether this particular idea is a dime a dozen or a unique idea... Everyone ends up going by feels when answering this question for any particular problem.
This is nothing to do with ideas. Schmidhuber is complaining that his published work was plagiarised. In machine learning research, in order to publish your work you have to show that your proposed approach works and to do that you have to beat some benchmarks and establish a new state of the art, otherwise there's no publication. That takes work and that's the work that Schmidhuber claims was inappropriately left uncited. In fact that's exactly the kind of work that Hinton, LeCun and Bengio have always done. That's what machine learning researchers do.
This ... idea that Schmidhuber is an ideas man who's never done any real work is Hinton's allegation, and it's clearly designed to misrepresent both Schmidhuber and his work in order to discredit his complaints. And I'm sorry to say that people on HN have fallen for it hook, line and sinker, I guess because that's what social media says.
Btw, the point I make, that you don't get published in machine learning without beating some benchmarks and establishing a new state of the art, I can attribute that to none other than Hinton himself, in an interview with Wired, whence I quote, by the by:
>> What we should be going for, particularly in the basic science conferences, is radically new ideas. Because we know a radically new idea in the long run is going to be much more influential than a tiny improvement.
So that's the guy accusing the other guy of being nothing but an ideas man and that you don't need to cite someone who first came up with an idea, saying that "new ideas" are important.
But that's just Hinton presenting things just the way he likes. Now ideas are important, now they're not, as he pleases.
I don't know enough about this area of research to have an opinion on this particular topic, but I've noticed a trend with this sort of thing with my colleagues. They both claim at different times, depending on whether it benefits them, that
(1) Ideas are a dime a dozen, and making it work or bringing it to fruition, is the important thing
and
(2) The idea is the important thing; the specific implementation by someone doesn't matter, as they're just doing what the idea creator or discoverer laid out for others to follow.
Sometimes I feel like there's a fundamental paradox there that arises a lot in numerous areas of work, business, and economics.
>Most ML researchers think that: He doesn't deserve the credit he claims, in most if not all cases.
That deserves a source. Especially for "all cases"; I don't think anyone who understands machine learning could read some of his earlier papers and still think Ian Goodfellow invented GANs.
Frankly, I do, and it comes across as quibbling about categories and trying to define unnecessarily general taxonomic categories, as a reaction to positive reactions to other people.
ex. in the article:
"Goodfellow eventually admitted that my PM is adversarial...but emphasized that it's not generative. However, [it] is both adversarial and generative (its generator contains probabilistic units)...It is actually a generalized version of GANs."
When you're at "actually, probabilities means generative, and actually you know what, even my initial claim was too specific: turns out its a generalized version of GANs", all in service of arguing a paper should have been cited in another paper, years after the other paper has been published, there's not much room for sympathy.
Agreed, Ian Goodfellow wasn't the first to come up with the idea of jointly training a generator and discriminator. But he was the first to make it work for image generation with modern neural networks. For that Ian Goodfellow deserved to get most of the credit, and he did.
While I generally agree with you in these specific cases, I feel like this is a shortsighted form of argument nonetheless. Coming up with an idea begets a scalable implementation of it and it's not very wise to ascribe absolute value solely on one side of the equation.
I think it's less about scalability and more about identifying specific choices that are promising. A lot of Schmidthuber's work paints out broad ideas in grand strokes and suggests hundreds of potential neural networks without evaluating what choices in that massive space are good. He then claims credit when other people identify the specific one or two of those hundred models (often with minor variations that make it not immediately obvious whether it perfectly fits the broad definition or not) that are actually promising.
But if other people identify and further develop one or two out of his models, he should still be cited. If they did not use his work at all, coming ith one or two models independently, then that's a different situation. It's a bit of an honor code thing as well, it's hard to prove if somebody has a read a paper or not. But then there's a more stringent standard where one cites as part of surveying preexisting work, which can result in a vast list.
Let's take an example: his "unnormalised linear Transformer," a neural network with "self-attention" published in 1992 under another name. It wasn't just an idea, it was implemented and tested in experiments. However, few people cared, because the computational hardware was so slow, and such networks were not yet practical. Decades later, the basic concepts were "reinvented" and renamed, and today they are really useful on much faster computers. Unquestionably, however, their origins must be cited by those who are applying them.
Why are some people here even debating the generally recognised rules of scientific publishing mentioned in the paper:
> The deontology of science requires: If one "reinvents" something that was already known, and only becomes aware of it later, one must at least clarify it later, and correctly give credit in all follow-up papers and presentations.
The sad part is that Schmidhuber is not a grifter by any means. If the turning prize for deep learning could go to 5 people instead of 3, he would very likely be on that list.
His lab is excellent and was easily Europe's best deep learning lab for decades before it blew up.
Some of his complaints are valid too. European labs often get ignored, and he has been sidelined despite being one of the most important people in deep learning himself.
But man doesn't know when an argument runs out of gas. His claims get grander with every passing year.
He would've just been the 'get off my lawn' grandpa of deep learning, but he somehow comes across as even more insufferable than that.
I wonder if 2023 schmidhuber was created because the polite one from a decade ago was ignored. A sort of evil phase, if you will.
I feel bad for him. He did get passed over of some deserved awards and recognition. But he reeks of resentment and thats never a good look.
> I feel bad for him. He did get passed over of some deserved awards and recognition. But he reeks of resentment and thats never a good look.
It's a terrible look. We're in the middle of one of the biggest gold rushes in tech history and he's wasting time complaining when he claims to be one of its pioneers? That effort is much better invested in building stuff but I suspect he's fallen into the classic PI trap of writing grants all the time and leaving the real work to the rest of the faculty, atrophying his skills too much to do anything now that the industry is moving so quikcly.
His students you mean, otherwise he would be first author. Academia is just like capitalism, with credits substituting for capital. The capital owner (laboratory head/professor) always gets a cut of everything published.
Authorship conventions vary a lot within the academia. In general, the closer the name is to either end of the author list, the more significant their contributions likely were.
Things also vary from paper to paper. Sometimes the first author just did the actual work for somebody else, and sometimes they also made significant intellectual contributions. (If the first author is listed as the sole corresponding author, it usually indicates the latter.) Sometimes the last/senior author just brought the money in, sometimes they were primarily mentoring the first author, and sometimes they were the driving force behind the project.
In some fields and some situations, last-author papers are actually more valuable than first-author papers. A first-author paper tells that you are capable of working as a junior researcher, while a last-author paper is a signal of your success as a senior researcher.
I once had a paper where I shared first authorship with four other people. That implies that the project was large enough that different people were in charge of different subprojects. Which in turn implies that the senior author must have made major contributions by being in charge of the entire project.
This is getting off-topic, but I've given up making heads or tails of authorship.
I know senior authors who are last on a paper just because they're senior and realize they (1) didn't really contribute much at all (they might have just inserted themselves on a paper), and (2) invoke the old meaning of last author knowing that the meaning has changed, so they end up being "the senior author" who gets a lot of credit just because they're senior.
Even producing the data takes on new meaning in an age of open science where datasets are distributed freely. What's the difference between citing an original study paper to give credit to the study PIs, and having them as a last author? Should someone who generates a dataset be last author on every paper using that data?
> In general, the closer the name is to either end of the author list, the more significant their contributions likely were.
In ML, NLP, and many Humanities too, the supervisor (supervising professor/postdoc, lab head, PI) is put last regardless of contribution. The rest of the author list is ranked in descending order for contribution. Often the last author's contributions are very limited to obtaining funding or proof-reading.
Now this practice is controversial with many venues stating that obtaining funding and only supervising is not a valid reason for authorship, but in my experience this practice doesn't die out.
Agreed, although I personally prefer when actual experts are communicating to the public. But that's not all he's doing, Tegmark also intends to influence global policy on AI research, random example: https://www.theguardian.com/technology/2023/sep/21/ai-focuse...
I independently arrived at[0] something close to Max Tegmark's idea of the Mathematical Universe, almost nobody noticed and fewer still cared because I published it as a LiveJournal blog post whereas he fleshed it out into a whole book.
I didn't get credit because I didn't do the hard work that deserves credit, I had the flash of inspiration and stopped after a few paragraphs of mediocre student philosophy.
This whole comment section is full of absolutely unacceptable ad-hominem attacks on Schmidhuber, from people who most likely haven't even read any of the works in question and are certainly not showing any of the "intellectual curiosity" this site is supposed to be about.
Anyone who cares about academic integrity should at least not attack someone complaining of plagiarism. That sort of attack is the academic equivalent of blaming the victim. If even half of Schmidhuber's accusations have a basis that's still a major academic scandal of epic proportions.
I came to Schmidhuber's work learning about his approach on creativity and curiosity and I loved it so much. It’s still so relevant. And beautiful, actually.
I tried reading one of his papers that he claimed contained an important concept avant la lettre. It was unreadable and it required an almost Marxist torture of the text to make it confess (weakly and unconvincingly) that it contained anything remotely like the idea he claimed it did.
I can see some angry comments here, but so far I have not seen any facts that refute his claims. Once I spent a long time reviewing a related paper on Hacker News, and I think he is right about disputes B1, B2, B5, H2, H4, H5. I'd have to study the others more closely:
B: Priority disputes with Dr. Bengio (original date v Bengio's date):
B1: Generative adversarial networks or GANs (1990 v 2014)
B2: Vanishing gradient problem (1991 v 1994)
B3: Metalearning (1987 v 1991)
B4: Learning soft attention (1991-93 v 2014) for Transformers etc.
B5: Gated recurrent units (2000 v 2014)
B6: Auto-regressive neural nets for density estimation (1995 v 1999)
B7: Time scale hierarchy in neural nets (1991 v 1995)
H: Priority disputes with Dr. Hinton (original date v Hinton's date):
H1: Unsupervised/self-supervised pre-training for deep learning (1991 v 2006)
H2: Distilling one neural net into another neural net (1991 v 2015)
H3: Learning sequential attention with neural nets (1990 v 2010)
H4: NNs program NNs: fast weight programmers (1991 v 2016) and linear Transformers
H5: Speech recognition through deep learning (2007 v 2012)
H6: Biologically plausible forward-only deep learning (1989, 1990, 2021 v 2022)
L: Priority disputes with Dr. LeCun (original date v LeCun's date):
L1: Differentiable architectures / intrinsic motivation (1990 v 2022)
L2: Multiple levels of abstraction and time scales (1990-91 v 2022)
L3: Informative yet predictable representations (1997 v 2022)
L4: Learning to act largely by observation (2015 v 2022)
In the past few hours I have had more time to look at the entire piece and download some of the referenced papers. So far I haven't found any claim that's factually inaccurate.
I think there is a reason why the ACM Turing awardees have never tried to defend themselves by presenting facts to the contrary: because they can't.
This might get interesting:
> The "Policy for Honors Conferred by ACM"[ACM23] mentions that ACM "retains the right to revoke an Honor previously granted if ACM determines that it is in the best interests of the field to do so." So I ask ACM to evaluate the presented evidence and decide about further actions.
If a guy becomes famous almost exclusively for having a beef with someone else (and this someone else really doesn't need to beat any drum in order to be famous), then something is wrong. Basically these days this is what Schmidhuber is associated to. Which is kind of sad, really.
>If a guy becomes famous almost exclusively for having a beef with someone else
He was famous well before that for having one of the best labs in Europe, and many key papers in early deep learning. He's only famous for the beef among people who aren't familiar with his earlier work and the huge contributions it made to the field.
It's hard not to think of the beef when you think of him now though, even if you already knew some of his work. To be clear I don't think that should take away from what he did accomplish, but it's like a Pavlovian association at this point to also have his behavior come to mind.
Reading the article and some of the links make me feel like the author Jürgen Schmidhuber is the academic version of the patent troll.
It sounds like he published some theoretical musings back in 1990s without any real practical implementation that did anything useful and since then has run around accusing AI researchers who actually produced concrete research and techniques to get are actually in use today of plagiarism.
Regardless, it should be easy to verify. He cites a paper and says it’s done in this earlier paper.
On the other hand, if his claims are verified, some of these research “discoveries” are simply common sense and would occur to most people working on the subject. HLB were awarded to a good extent because they worked on deep learning at the right time. Deep learning became hugely practical, and outperformed the state of the art in many applications. HL also worked for major companies.
>It sounds like he published some theoretical musings back in 1990s without any real practical implementation
He published working models back then, the problem was compute power was very limited. In the past decade deep learning took off, people took his models, renamed then and ran them on vastly more powerful computers, to great success, then failed to cite him.
It's actually a common problem in lots of computing related research communities. Papers older than 10 years ago, sometimes even 5 years ago, are ignored. Because the results do not compare in terms of computing power, and because reimplementing old papers is often very hard because they are often too vague.
The machine learning field as a whole has a huge credit assignment problem. This post seems to encourage other ML researchers to come out with their own priority disputes. Tomas Mikolov just aired his grievances:
> I wanted to popularize neural language models by improving Google Translate. I did start collaboration with Franz Och and his team, during which time I proposed a couple of models that could either complement the phrase-based machine translation, or even replace it. I came up (actually even before joining Google) with a really simple idea to do end-to-end translation by training a neural language model on pairs of sentences (say French - English), and then use the generation mode to produce translation after seeing the first sentence. It worked great on short sentences, but not so much on the longer ones. I discussed this project many times with others in Google Brain - mainly Quoc and Ilya - who took over this project after I moved to Facebook AI. I was quite negatively surprised when they ended up publishing my idea under now famous name "sequence to sequence" where not only I was not mentioned as a co-author, but in fact my former friends forgot to mention me also in the long Acknowledgement section, where they thanked personally pretty much every single person in Google Brain except me. This was the time when money started flowing massively into AI and every idea was worth gold. It was sad to see the deep learning community quickly turn into some sort of Game of Thrones. Money and power certainly corrupts people...
It's nice of Schmidhuber to point out the quality papers with theoretical advancements and actual validations that fix the intellectual problems with his weak musings.
>But yeah, you honestly have to wonder what it is like day-to-day working with someone with this level of delusions of grandeur.
It's not delusions of grandeur; anyone with half a brain who read his early papers would see he clearly came up with the idea of a GAN well before Ian Goodfellow.
As an outsider to this space, it seems suspect that all of the people he has a beef with (like LaCun) will gladly cite others and predecessors, but not Schmidhuber.
Has he explained his reasons for thinking there is some conspiracy? Otherwise it reflects badly on his assessment of himself, or possibly his mental state.
Copying and pasting Urban Dictionary's definitions of "to schmidhuber"[a], "schmidhuber"[b] and "schmidhubered"[c]:
---
to schmidhuber
When you publicly claim that someone else's idea that is remotely resembling your own is stolen from you.
---
schmidhuber
1) To interject for a moment and explain how one's recent popular idea is a few transformations away from your 1991 paper.
2) To miraculously produce fifty years of relevant literature after someone claims to trace the origin of an idea in a particular work.
---
schmidhubered
Being "schmidhubered" looks something like this:
1) Invent something brilliant that no one cares about. Experience derision.
2) That thing becomes popular years later. Someone else is given credit for inventing it. That person appears in the New York Times and is declared smartest person alive.
3) Go on a campaign explaining the situation and how you are the rightful inventor and thus the rightful Smartest Person Alive.
4) Everyone accuses you of being a sore loser and no one takes you seriously.
