I agree that late-stage capitalism can create really tough situations for poor families trying to afford drugs. At the same time, I don't know any other incentive structure that would have brought us a breakthrough like AlphaFold this soon. For the first time in history, we have ML models that are beating out the scientific models by huge margins. The very fact that this comes out of the richest, most competitive country in the history of the world is not a coincidence.
The proximate cause of the suffering for terminally-ill children is really the drug company's pricing. If you want to regulate this, though, you'll almost certainly have fewer breakthroughs like AlphaFold. From a utilitarian perspective, by preserving the existing incentive structure (the "deeply broken situation" as you call it), you will be extending the lifespans of more people in the future (as opposed to extending lifespans of more people now by lowering drug prices).
Late-stage capitalism didn't bring us AlphaFold, scientists did, late-stage capitalism just brought us Alphabet swooping in at literally the last minute. Socialize the innovation because that requires potential losses, privatize the profits, basically. It's reminiscent of "Heroes of CRISPR," where Doudna and Charpentier are supposedly just some middle-men, because stepping in at the last minute with more funding is really what fuels innovation.
AlphaFold wasn't some lone genius breakthrough that came out of nowhere, everything but the final steps were basically created in academia through public funding. The key insights, some combination of realizing that the importance of sequence to structure to function put analyzable constraints on sequence conservation and which ML models could be applied to this, were made in academia a long time ago. AlphaFold's training set, the PDB, is also a result of decades of publicly funded work. After that, the problem was just getting enough funding amidst funding cuts and inflation to optimize. David Baker at IPD did so relatively successfully, Jinbo Xu is less of a fundraiser but was able to keep up basically alone with one or two grad students at a time, etc. AlphaFold1 threw way more people and money to basically copy what Jinbo Xu had already done and barely beat him at that year's CASP. Academics were leading the way until very, very recently, it's not like the problem was stalled for decades.
Thankfully, the funding cuts will continue until research improves, and after decades of inflation cutting into grants, we are being rewarded by funding cuts to almost every major funding body this year. I pledge allegiance to the flag!
EDIT: Basically, if you know any scientists, you know the vast majority of us work for years with little consideration for profit because we care about the science and its social impact. It's grating for the community, after being treated worse every year, to then see all the final credit go to people or companies like Eric Lander and Google. Then everyone has to start over, pick some new niche that everyone thinks is impossible, only to worry about losing it when someone begins to get it to work.
Why haven't the academics created a non profit foundation with open source models like this then? If alphabet doesnt provide much, then they will be supplanted by non profits. I see nothing broken here.
I work at Open Force Field [1] which is the kind of nonprofit that I think you're talking about. Our sister project, OpenFold [2], is working on open source versions of AlphaFold.
We're making good progress but it's difficult to interface with fundamentally different organizational models between academia and industry. I'm hoping that this model will become normalized in the future. But it takes serious leaps of faith from all involved (professors, industry leaders, grant agencies, and - if I can flatter myself - early career scientists) to leave the "safe route" in their organizations and try something like this.
Individual labs somehow manage to do that and we're all grateful. Martin Steinegger's lab put out ColabFold, RELION is the gold standard for cryo-EM despite being academic software and the development of more recent industry competitors like cryoSPARC. Everything out of the IPD is free for academic use. Someone has to fight like hell to get all those grants, though, and from a societal perspective, it's basically needlessly redundant work.
My frustrations aren't with a lack of open source models, some poor souls make them. My disagreement is with the perception that academia has insufficient incentive to work on socially important problems. Most such problems are ONLY worked on in academia until they near the finish line. Look at Omar Yaghi's lab's work on COFs and MOFs for carbon/emission sequestration and atmospheric water harvesting. Look at all the thankless work numerous labs did on CRISPR-Cas9 before the Broad Institute even touched it. Look at Jinbo Xu's work, on David Baker's lab's and the IPD's work, etc. Look at what labs first solved critical amyloid structures, infuriatingly recently, considering the massive negative social impacts of neurodegenerative diseases.
It's only rational for companies that only care about their own profit maximization to socialize R&D costs and privatize any possible gains. This can work if companies aren't being run by absolute ghouls who aren't delaying the release of a new generation of drugs to minimize patent duration overlap or who aren't trying to push things that don't work for short-term profit. This can also work if we properly fund and credit publicly funded academic labs. This is not what's happening, however, instead public funded research is increasingly demeaned, defunded, and dismantled due to the false impression that nothing socially valuable gets done without a profit motive. It's okay, though, I guess under this kind of LSC worldview, that everything always corrects itself so preempting problems doesn't matter, we'll finally learn how much actual innovation is publicly funded when we get the Minions movie, aducanumab, and WeWork over and over again for a few decades while strangling the last bit of nature we have left.
It is such a surprise when economics and philosophy of morality end up proving that it was a moral duty of large tech companies and billionaires to become filthy rich. Those people were working for the good of humanity all along, we just didn't look at the data close enough to get it.
I think at some point, we will be able to produce models that are able to pass data into a target model and observe its activations and outputs and put together some interpretable pattern or loose set of rules that govern the input-output relationship in the target model. Using this on a model like AlphaFold might enable us to translate inferred chemical laws into natural language.
No, the activations are a combination of the basis function and the spline function. It's a little unclear to me still how the grid works, but it seems like this shouldn't suffer anymore than a generic relu MLP.
I think the hand running through the wheat (?) is pretty good, object permanence is pretty reasonable especially considering the GAN architecture. GANs are good at grounded generation--this is why the original GigaGAN paper is still in use by a number of top image labs. Inferring object permanence and object dynamics is pretty impressive for this structure.
Plus, a rather small data set: REDS and Vimeo-90k aren't massive in comparison to what people speculate Sora was trained on.
This paper reminds me of the Neural Network Diffusion paper which was on the front page of HN yesterday in the sense that we are training another model to bypass a number of iterative steps (in the previous paper, those were SGD steps, in this one, it is A* exploration steps).
On a different note, they choose such a bad heuristic for the A* for Sokoban. The heuristic they choose is "A∗ first matches every box to the closest dock and then computes the sum of all Manhattan distances between each box and dock pair". I played Sokoban for 20 minutes while reading the paper and I feel like this is a very poor exploration heuristic (you often need to move boxes away from goal state to make progress).
"We synthesize 100 novel parameters by feeding random noise into the latent diffusion model and the trained decoder." Cool that patterns exist at this level, but also, 100 params means we have a long way to go before this process is efficient enough to synthesize more modern-sized models.
Important to note, they say "From these generated models, we select the one with the best performance on the training set." Definitely potential for bias here.
I agree that late-stage capitalism can create really tough situations for poor families trying to afford drugs. At the same time, I don't know any other incentive structure that would have brought us a breakthrough like AlphaFold this soon. For the first time in history, we have ML models that are beating out the scientific models by huge margins. The very fact that this comes out of the richest, most competitive country in the history of the world is not a coincidence.
The proximate cause of the suffering for terminally-ill children is really the drug company's pricing. If you want to regulate this, though, you'll almost certainly have fewer breakthroughs like AlphaFold. From a utilitarian perspective, by preserving the existing incentive structure (the "deeply broken situation" as you call it), you will be extending the lifespans of more people in the future (as opposed to extending lifespans of more people now by lowering drug prices).