This is embarrassing. I would say Hopfield networks aren't even very revolutionary in neuroscience, but they're so old I can't tell. In terms of AI... they've been irrelevant for thirty years. I guess you could argue a transformer is a generalized Hopfield network, but of course that's a post-hoc understanding. None of this has anything to do with physics.
So what if an energy function lets you approximate the number of macro-states it can capture? Should every mathematics paper with Lagrange multipliers be put up for nomination? Every poll that uses the law of large numbers, and thus, entropy? Surely the computer scientists building the internet need to be included as well, since their work is based in information theory.
Or maybe, hear me out, we reserve the Nobel Prize in physics for advances in the physical sciences, understanding physical reality or how to bend it to our will.
“These artificial neural networks have been used to advance research across physics topics as diverse as particle physics, materials science, and astrophysics,” Ellen Moons, chair of the Nobel Committee for Physics, said at a press conference this morning.
The goal here is to attribute a very important area in contemporary technology to physicists. This prize advances physics in terms of giving it higher importance in the minds of lay people and journalists.
There were some predictions that Peter Shor could win this year for quantum computation. I'd say his work is a lot closer to physics than Hinton's or Hopfield's.
Had they wanted a good ML relevant physics Nobel, the committee had decades to award a prize to Marshall and Arianna Rosenbluth for the Markov Chain Monte Carlo method. Would have been self-evidently important and relevant to both physics and ML. Too late now -- Arianna died in 2020.
As a physicist, my reaction to this is how bizarre is that. Maybe he deserves a nobel prize but in physics?
Also arguing that NN is used in physics so we can argue nobel prize is okay is like asking for Stephan Wolfram to be awarded Nobel prize for Mathematica which is much more used in physics as a tool. And he is a physicist and had contributions to the field of numerical relativity (The reason he created Mathematica in the first place).
The royal science academy fucked up so much with this choice.
By this definition Claude Shannon (the father of Information Theory) clearly deserves a Nobel in Physics. The central concept in Information Theory is Entropy which is defined literally the same way as in Physics. And Shannon's Information Theory clearly revolutionized our life (tele-communications) much more than Hopfield network or Hinton's Boltzmann machine.
In 1939 Claude Shannon won the "wrong" Nobel prize -- The Alfred Noble Prize award presented by the American Society of Civil Engineers [0]. It causes a lot of confusion.
Claude Shannon never won a "real Nobel".
Someone changed the Wikipedia article today to call Hopfield a "physicist". Previously the article called him simply a scientist, because his main work wasn't limited to physics. I changed it back now, let's see if it holds up.
>where we all know that mathematics and/or CS deserve the honor
Or semiconductor manufacturers.
All the math and CS needed for AI can fit on a napkin, and had been known for 200+ years. It's the extreme scaling enabled by semiconductor science that really makes the difference.
That's absurd. The computer science needed for AI has not been known for 200 years. For example, transformers were only invented in 2017, diffusion models in 2015.
(When the required math was invented is a different question, but I doubt all of it was known 200 years ago.)
TBF backpropagation was introduced only in the 1970's, although in hindsight it's a quite trivial application of the chain rule.
There were also plenty of "hacks" involved to make the networks scale such as dropout regularization, batch normalization, semi-linear activation functions (e.g. ReLU) and adaptive stochastic gradient descent methods.
The maths for basic NNs is really simple but the practice of them is really messy.
Residual connections are also worth mentioning as an extremely ubiquitous adaptation, one will be hard-pressed to find a modern architecture that doesn't use those at least to some extent, to the point where the original Resnet paper sits at over 200k citations according to google scholar[1].
> All the math and CS needed for AI can fit on a napkin, and had been known for 200+ years.
This isn't really true. If you read a physics textbook from the early 1900s, they didn't really have multivariate calculus and linear algebra expressed as concisely as we do now. It would take several napkins. Plus, statistical mechanics was quite rudimentary, which is important for probability theory.
The Nobel Committee doesn’t represent the field of physics. I talked to a few former colleagues (theoretical physicists) just now and every one of them found this bizarre.
I don't know if this is a travesty that they awarded the prize to work on non-physical systems to jump on a bandwagon or that there was nothing else obvious enough to the board in actual physics to give this to.
If I was the awardee I'd consider declining just out of respect to the field.
The linked document connects their work to physics as follows:
"The Hopfeld network utilises physics
that describes a material’s characteristics due to its
atomic spin – a property that makes each atom a tiny
magnet. The network as a whole is described in a
manner equivalent to the energy in the spin system
found in physics, and is trained by fnding values for
the connections between the nodes so that the saved
images have low energy"
"Hinton used tools from statistical physics, the science
of systems built from many similar components."
100% agreed as I can't think of any one individual since(1) who has done as much for all of science and engineering as he ultimately did; alas, they are not awarded posthumously.
(1) Newton would be a strong contender on a "for all time" basis, but even he would've probably needed to share it with Leibniz, which would have driven him absolutely ~b o n k e r s~, like wet hornet in a hot outhouse mad, LOL.
I think this is the Royal Academy of Sciences way to admit that Physics as a research subject has ground to a halt. String theory suffocated theoretical high energy physics for nearly half a century with nothing to show for it, and a lot of other areas of fundamental physics are kind of done.
I think this is (very) inaccurate. It feels more like them trying to jump on a "hot topic" bandwagon (machine learning/AI hype is huge).
Physics as a discipline hasn't really stalled at all. Fundamental physics arguably has, because no one really has any idea how to get close to making experimental tests that would distinguish the competing ideas. But even in fundamental physics there are cool developments like the stuff from Jonathan Oppenheim and collaborators in the last couple of years.
That said "physics" != "fundamental physics" and physics of composite systems ranging from correlated electron systems, and condensed matter through to galaxies and cosmology is very far from dead.
I don't know exactly what they hope to gain by jumping on that bandwagon though; neither the physicists nor the computer scientists are going to value this at all. And dare I say, the general populace associated with the two fields isn't going to either - case in point, this hn post.
If there weren't any noble-worthy nominations for physics, maybe skip it? (Although that hasn't happened since 1972 across any field)
It really has not, though. There is more to physics than high-energy and cosmology, and there is no shortage of deserving contributions of smaller scope. It really is bizarre that deep learning would make it to the top of the list.
Could you give me some examples of areas of fundamental physics that are vital and have done some significant discoveries lately? I genuinely would like to know, because I really can't think of any.
I'm probably not the right person to ask, but off the top of my head: superconductivity of high-pressure hydrides; various quantum stuff like quantum computing, quantum cryptography, quantum photonics, quantum thermodynamics; topological phases; rare decays (double beta, etc.); new discoveries in cosmic rays, etc.
My point was that physics is a big and active field, stagnation at the smallest and largest scales notwithstanding. Note also that the Nobel committee is not in any way limited to "newsworthy" stuff and has in many cases awarded prizes decades after the fact.
"Vital" is completely subjective but I'd throw stuff around quantum information into the ring. Maybe you'd consider the loop-hole free Bell tests performed in 2015 and awarded the 2022 Nobel prize to count?
I think the prize in 2022 was a nice prize, but it could still be considering just tidying the corners. In the end it just proved that things really work as most of us has thought it worked for decades.
> Physics as a research subject has ground to a halt
Max Planck was told by his professor to not go into Physics because "almost everything is already discovered". Planck said he didn't want to discover anything, just learn the fundamentals.
My sense is that we might have reached the limits of what we can do in high-energy or fundamental physics without accessing energy levels or other extreme states that we currently can't access as they are beyond our capacity to generate.
From what I've read (not a professional physicist) string theory is not testable unless we can either examine a black hole or create particle accelerators the size of the Moon's orbit (at least). Many other proposed theories are similar.
There is some speculation that the hypothetical planet nine -- a 1-5 Earth mass planet predicted in the far outer solar system on the basis of the orbits of comets and Kuiper Belt / TNO objects -- could be a primordial black hole captured by the solar system. A black hole of that mass would be about the size of a marble to a golf ball, but would have 1-5g gravity at the distance of Earth's radius.
If such an object did exist it would be within space probe range, which would mean we could examine a black hole. That might get us un-stuck.
If we can't do something like that, maybe we should instead focus on other areas of physics that we can access and that have immense practical applications: superconductivity, condensed matter physics, plasmas / fusion, etc.
This is totally bizarre, no precedent for it really. The reality of the prize means that less and less are the winners names every physicist has heard of, but even today they're still big names in each subfield. For e.g. Kosterlitz, Thouless and Haldane weren't exactly household names but they really deserved the prize in 2016.
In this case, there's a good argument that Hopfield had conducted strong work as a physicist and in physics, but Geoffrey Hinton has never worked as a Physicist, at best adopting some existing things from physics into cognitive science use cases. In any case, what they've been given the prize for is work where they've not contributed to the understanding of the world of physics - it's not even really an arguable case where this is work that crosses over between Physics and another field either. It'd be like if Black or Scholes had been given the Physics prize rather than Economics because their famous equation can be re-written in Schrodinger equation form.
This does indeed smell of desperation. Which is really, really sad. Advances in _real_ physics are central to the absolutely needed sustainability transition. In a sane society that values its self-preservation you would not need to grasp at second-order straws to justify the need for all sorts of both fundamental and applied physics research.
We need to think seriously whether our collective hallucinations (pun) have got us to some sort of tipping point, undermining our very ability to act according to our best long-term interests.
ps. not to imply anything negative about the worthiness of the awardees in general
May be I should know better, but is there no Nobel category for computer science or mathematics? This isn't physics, this is absolutely embarrassing. May be all those bitter elder physicists who didn't get a prize can feel a little justified in their derision of the institution.
Computer science has the Turing award and mathematics the Fields medal. Neither is exactly equivalent to the Nobel but they're similar levels of prestige.
The Nobel prize fields and criteria are a bit random, they're essentially just whatever Alfred Nobel wrote in his will.
It’s time to consider adding computer science as a category for the Nobel Prize. They have already been awarded prizes for economic sciences and peace, so why not computer science? It's not the same as physics, and its impact on modern life is undeniable
Is that even "possible"? As in, they have to follow the rules of the organization, which have some criteria laid out. Not sure who could stop them from changing, though. Like, I think the original intent was to have done the most the preceding year, but now it's more of a lifetime award. So perhaps they can change or add different categories if wanted.
In some sense it makes sense to award AI researchers on behalf of the physics community because I know many physics PhD’s who thank their job to AI; they work as data scientist now.
Jokes aside, physics is a bit stuck because it’s hard to do experiments at the boundaries of what we know, as far as I understand. So then it makes sense I guess to award people who made useful tools for physics.
> it’s hard to do experiments at the boundaries of what we know
this applies primarily to fundamental physics. There are many areas of applied physics (materials, fusion, biophysics, atmospheric physics, etc. etc.) where the main constrain is understanding complex systems. These areas are quite crucial for society.
Moreso genuine curiosity than as a gotcha: A lot of comments are saying this was the wrong choice. I'd find it really interesting to hear who the nomination should have gone to instead, in your opinions.
This reminded me of my Hopfield networks implementation in Go [1]. The algorithm is rather simple but fascinating nevertheless and works surprisingly well for reconstructing noisy images. I actually blogged about it as well [2]. But as many are discussing here Deep Memory networks based on Boltzmann networks are more powerful yet they don't seem to have found much use case either
My perspective as a PhD in theoretical physics, who's been doing deep learning in the last 4 years:
1. The prize itself makes zero sense as a prize in _physics_. Even the official announcement by the Nobel Prize Committee, taken at a face value, reads as a huge stretch in trying to link neural networks to physics. When one starts asking questions about the real impact on physics and whether the most important works of Hinton and Hopfield were really informed by physics (which is a dubious link to the Nobel prize anyway), the argument stops holding water at all.
2. Some of the comments mention that giving prize for works in AI may make sense, because physics is currently stalled. This is wrong for several reasons:
2.1. While one can argue that string theory (which is, anyway, only a part of high-energy theoretical physics) is having its "AI winter" moment, there are many other areas of physics which develop really fast and bring exciting results.
2.2. The Nobel Prize is often awarded with quite some delay, so there are many very impactful works from 80s which haven't been awarded with a Nobel prize (atomic force microscopy is a nice example).
2.3. It is wrong to look at the recent results in some sub-field and say "okay, there was nothing of value in this field". For example, even if one completely discards string theory as bogus, there were many important results in theoretical physics such as creation of conformal field theory, which was never recognized with a Nobel Prize (which is OK if Nobel Prize is given to other important physical works, but is quite strange in the light of today's announcement).
To finish on a lighter mood, I'll quote a joke from my friend, who stayed in physics: "Apparently the committee has looked at all the physicists who left academia and decided that anything they do is fair game. We should maybe expect they will give a prize for crypto or high-frequency trading some time later".
The contrast in discoveries made in 'core' physics in the first 25 years of the last centuries compared to this century is quite insane, it was never going to be sustainable. If it did sustain we would be colonising a new galaxy by now.
Consider that in 1900 the atom wasn't discovered yet, within around 25 years the basic principles of quantum physics were established, to say nothing about discoveries in cosmology (GR + big bang).
Real physics and its resulting breakthrough technologies have been hidden from society for a very long time. And so they simply need somebody they can give that price to.
Since most papers nowadays are written by AI, and peer reviewed by AI, it only seemed logical for AI to be used by the Nobel committee to award the godfather of AI.
Thanks to AI, you now only have to to ask any GPT for the source code of the universe to get the code. Since physics is now a solved problem, we should recenter ourselves on more important questions like why did AI create the universe ?
Honestly, I am stunned by today's Nobel committee announcement. Hinton's Boltzmann machine is a clever construct that nobody, repeat nobody, in the AI and ML is using anymore in actual practice.
I'm equally as confused; huge WTF moment. Seems like a paradigm breakthrough, in that Nobel Prizes can be given for discoveries in tangential fields. Or perhaps it's due to Dr. Hopfields physicist status, that all his discoveries are considered physics related? Or that NNs are considered a part of physics / nature?
What a bunch of BS, yet another field trying to steal the thunder of CS. How often have I had to listen to physicists sneer at CS as not a proper science!
"This year’s laureates used tools from physics to construct methods that helped lay the foundation for today’s powerful machine learning."
Does this mean if I'd use a deep understanding of birds to design way more aerodynamic airplanes, I could get the Nobel prize in physiology/medicine? Don't get me wrong, their work is probably prize worthy, but shouldn't the Nobel prize in physics be awarded for discoveries in the _physical world_?
> Don't get me wrong, their work is probably prize worthy
I would strongly disagree with you there. It's the exact same idea as the least squares approximation or conjugate gradient method: create an energy function from a quadratic and minimize it.
So what if an energy function lets you approximate the number of macro-states it can capture? Should every mathematics paper with Lagrange multipliers be put up for nomination? Every poll that uses the law of large numbers, and thus, entropy? Surely the computer scientists building the internet need to be included as well, since their work is based in information theory.
Or maybe, hear me out, we reserve the Nobel Prize in physics for advances in the physical sciences, understanding physical reality or how to bend it to our will.
reply