Hacker News new | past | comments | ask | show | jobs | submit login
The dawn of post-theory science? (theguardian.com)
91 points by bookofjoe 12 days ago | hide | past | favorite | 118 comments

OK so most of the comments seem dismissive, picking on unfortunate statements (between Newton & Zuck) or an ungenerous reading of the claim.

I'll take the other side.

Consider psychology, economics, ecology, climate, many health & biology related fields, etc. These fields have struggled to succeed within the Popperian/Newtonian framework. We haven't found an F=ma or E=mc^2 that is fundamental, predictive and generally useful. This has resolved to a lot of frustrating modeling and unfalsifiable theories. P-hacking, replication crisis & such.

Marginalist economics is a great example. If you squint, it kinda conforms to Popperian models. In practice though, it's not workable. In practice, it's riddled with post fact storytelling, subjectivity and opinion.

The "high science" mode of enquiry is amazing. It has arguably advanced us more as a species than any other idea. It is, however, not the only mode of enquiry. We walk around the world forming tentative theories for how the world works. They're generally not simple, expressible or general. They are, however, predictive. Jane knows how Joe will react to X. She doesn't know it with certainty or accuracy, but she's got pretty high chances of being right. Call that a theory, a model... whatever. It's a real way of having knowledge. Farmers' understanding of the farm.

There's no guarantee that the phenomenon we're interested in will be expressible in an F=ma manner. Complexity exists.

>There's no guarantee that the phenomenon we're interested in will be expressible in an F=ma manner. Complexity exists.

That's a fair point. But having a good theory doesn't necessarily mean that it's expressible as a succinct equation. The theory of how metabolism works is insanely complicated, but it's a good theory nonetheless, because every part of it plays a functional role, it explains a wide range of phenomena and it's logically compatible with other successful theories. None of that holds for deep learning models.

Good points.

Besides being an overly provocative term, I don't think "post-theory science" is a belligerent to "theory-based" science. It's not logically compatible or incompatible with other theories, because it's not logically based.

It may be contradict theoretical predictions, and that's where empiricism comes in. Long term, if successful, deep learning models of metabolism may be useful or irrelevant to the development of a theoretical understanding.

A lot things in practice are using empirical methods and are not looking at F=ma like representations.

The basic issue with going just empirical is that there is no way of knowing if the relationship actually exists or if what is observed is a symptom of something else. (Or, in the extreme case: random. Some bits in finance have that problem, for example, finding an edge in markets).

Most successful application of ML etc. are not relying in areas where there are no ideas about any relationships, but rather where there are a lot of confounders and the relationships are complex/difficult to formulate.

Sticking with on of the examples, we know that proteins fold in somewhat predictable ways, because we do have theories around the "mechanics" there. So good applications are not strictly in the realm of scientific theory but rather in being able to apply and use such theories in previously inaccessible ways.

> Sticking with on of the examples, we know that proteins fold in somewhat predictable ways, because we do have theories around the "mechanics" there

But we were broadly unable to leverage those theories and knowledge into a predictive system. AlphaFold largely dropped all that (IIUC), and works much better without it.

(I was once a PhD candidate in protein folding, and left in part because I felt the field was unlikely to make any substantive progress. It didn't, for at least 30 years, until NN-based "modelling" started to replace all approaches up till then)

The point is that given a solution to protein folding, we can then compare that solution with current theory and even physically test it. The search space is enormous but individual solutions are physically deterministic.

This is a good application of ML since there is an independent “check” on its work. And it’s hard to call it “the end of theory” when theory is still tightly coupled with the system, just at the back end instead of the front end.

That is not true of many applications of ML, like Facebook, which simply spit out solutions at scale with no way to independently check them. We should not think the same way about all ML systems; there is a lot more to it than just the neural network.

How is theory still involved? You don't use theory to check the predictions AlphaFold makes, you use molecular structure determination (x-ray crystalography, MRI, other techniques). Of course, those involve "theory", but not theories about protein folding.

For me, theory enables this to actually work in the sense that we know that structures are not random, i.e. there is some way to predict them and it is inherent in the composition (hand waving). If there is no theory linking input and output then predictions could just randomly stop working, for example.

This is just simply incorrect.

Theory is not involved in this. We know that a given protein will always (more or less) fold into the same configuration every time, generally on their own. We know this by doing experiments in which the protein is denatured (heated to break internal covalent and some ionic bonds, and then left to anneal (cool down), and then inspected at the atomic level for the resulting structure. They always fold back to the same shape. This is an experimental determination, not a theoretical one.

Nobody was ever successful at devising any comprehensive theory that explained how they did this, though they of course identified many different elements that are likely involved in some way (hydrophobicity, local charge, possible compactness and many more).

But there is (currently) no theory that connects protein amino acid sequence to structure, which is why AlphaFold was such a huge deal. It makes the connection (somehow), without any theory builtin, and without any theory to explain.

In fact, there is not even a theory that requires that they always fold into the same shape.

You are misunderstanding me. There is theory about how atoms etc. interact. We cannot use that to predict shape (at present) but at the same time there is no force etc. missing as far as our current understanding is concerned (at most basic level no physics seem to missing). So predictions seem at least possible or rather are not blocked.

On the other hand, trying to use an AI to make predictions which atom of a radioactive element was to decay next would run afoul of current theories.

People have worked on "theory-based" protein structure prediction for 35-40 years. It essentially went nowhere.

It is theoretically possible? Probably. Does it have anything to do with how AlphaFold works? no. Does it have anything to do with verifying AlphaFold's predictions? no.

I never claimed it has anything to do with verifying the predictions. I am saying it allows such predictions.

But it doesn't!

It isn't theory that led us to notice that (almost all) proteins fold naturally into the same shape every time they are denatured and annealed. It was simple experimentation.

The attempts to create a theory of how that happens have been measured by the success of those attempted theories to make successful structure predictions. They have failed (pretty dismally in most cases).

So there is no theory that allows us to make such predictions. We simply have experimental evidence that proteins can adopt their "normal" structure by themselves, essentially every time. We have no functioning theory of how they do this, and no way to use this non-existent theory to make predictions.

You are still misunderstanding me. I am not saying we have a theory of how proteins fold. We have theories on force interactions between atoms. Those theories are pretty complete and do not say that predictions of protein shapes are impossible. We are not blocked from trying, like, for example, in predicting decaying atoms. So trying stuff like AlphaFold is not going against current understanding.

My point is much more basic, if you will. So I meant "allow" as in there is no theory saying that we cannot do it. Not that we have a theory how to do it.

Well, but wasn't AlphaFold at least in part trained based on the existing mechanical models? Or was AlphaFold only trained on ground truth imaging of the folding process?

It was trained on experimentally-determined physical structures. Not predicted models based on various folding engines.

> This has resolved to a lot of frustrating modeling and unfalsifiable theories. P-hacking, replication crisis & such.

It's entirely possible we haven't found such models because of the p-hacking, poor replication and publications with low signal to noise ratios, and not vice versa. Psychology's replication rate was in the mid 30% range. It would be hard to see any general trends with such unreliable data.

You can create a complex system to make predictions about complex systems. But without underlying theory, you can’t put bounds on its accuracy or utility. Such a system will work right up until it doesn’t, with no way to predict in advance where that point is.

We see this with people all the time. Jane thinks she knows how Joe will react to X but Joe does something unexpected.

A system that makes accurate predictions an unpredictable percentage of the time is not that useful. But it might be great for fooling ourselves or others.

Jane being able to predict how Joe will react most of the time turns out to be super useful in real life.

Sure it fails sometimes, but nobody would voluntarily give up that ability.

This is a great comment and what holds a lot of today's AI back is indeed complexity. The only question I would add is whether it's still science (hypothesize, predict and test as stated) if it's not a human performing the expected steps - e.g. is training a neutral network through feedback iterating through these steps (with short cycle times?)

Probably depends on how you define science. If you put science on a spectrum with it on one end and magic on the other (so the spectrum is more of explainable knowledge versus useful but inexplicable rituals), AI definitely moves towards the magic end of the spectrum and maybe into a field commonly called "engineering" for the fact that it bastardizes knowledge with utility.

My question above would definitely not extend to the word engineering (and by association I probably have to withdraw the idea that AIs could do science.)

>> The only question I would add is whether it's still science (hypothesize, predict and test as stated) if it's not a human performing the expected steps

This is possibly just a semantic question. If so, I have no problem calling it something else.

this is my question with a lot of this "science" - how are we ensuring that we're not incorporating researcher biases when setting up and training a new model?

To take a reasonably provocative example: if a new climate model predicted extremely low rates of change, would it be tuned so it showed change in the accepted range? If so, then how useful is this model to science?

> If it's not a human performing the expected steps

Nothing is performing those steps in a NN.

Not sure why you attack economics?

It works really well to eg predict and understand the impact of tariffs or taxes or deregulation.

Yes, economists can't predict recessions in advance. And economics also tells you exactly why: the efficient market hypothesis tells you that if people knew that stock prices were going down next year, prices would already react to that knowledge today.

> Yes, economists can't predict recessions in advance

Aren't they actually pretty good at predicting recessions? Thought the joke is that they predicted 12 of the last 7.

pretty sure the parent is doing the exact opposite. other forms of inquiry aren't valueless because they aren't falsifiable.

cheers. Dead on.

It's not an attack.

OK! I might have misread your comment.

> Somewhere between Newton and Mark Zuckerberg, theory took a back seat.

It is like saying "Somewhere between Ghandi and Wonder Woman, India's independence movement took a back seat"

It is clear that WW has no relevance just as Zuckerberg to science.

Also, comparing one of the most respected human minds of all time to a simply grotesquely powerful active living person this way is something that is very distasteful from a news publication.

It's very weird, but i can't shake the feeling that this article always uses Facebook examples for positive things (well working algorithms, putting Zuckerberg right up to newton), but very negative towards Google (biased search results, misleading algorithms). It might not be on purpose, or it might really just be in my head, but I found it super weird.

What confuses me most about the constant google bashing is that google is the one of the big companies that actively uses it's ressources to go against disinformation, at least when it comes to science (COVID-19) or politics (no political microtargetting e.g. on YT)


Why is Google bashed so much more than FB or Amazon (product disinformation)?

> Why is Google bashed so much more than FB or Amazon

I would attribute that to personal feelings and confirmation bias.

Facebook has received a lot of negative press coverage in recent times, for their help to nationalist parties in India, their behaviour in Myanmar, Burma, for their mishandling of fake news in the US and Europe, etc. They've also been at the heart of multiple legislative investigations in the US and abroad.

Comparatively, Google has received little backlash and attention for their own misdeeds.

> It is clear that WW has no relevance just as Zuckerberg to science.

> Also, comparing one of the most respected human minds of all time to a simply grotesquely powerful active living person this way is something that is very distasteful from a news publication.

A more generous interpretation could be that on social networks, emergent properties are hard to predict beforehand with theory, and require observation of the actual phenomenon. That's actually behind a large body of scientific work, the best known being agent-based models.

Also I think it's biased to simpy discard the scientific contribution of FANGs with emotionally loaded words like "grotesquely": from algorithms to new methods, there's much good to be found in what large companies (that you may like or dislike) have done.

But Zuckerberg hasn't done any of them. It's like ascribing Newton's accomplishments to the royal in power when he was alive

Totally agree with your point, but just because I was curious...

Isaac Newton lived from 1642 – 1726 so technically:

* 1642-1649 : Charles I (until he got the chop)

* 1649-1660 : Interregnum, Cromwell and Son

* 1660-1685 : Charles II (Restored, partied a lot)

* 1685-1688 : James II (Too Catholic, overthrown)

* 1689-1702 : William and Mary ('Leveraged buyout' by the Dutch as Matt Christman puts it)

* 1702-1707 : Anne (Last English monarch... see Acts of Union)

* 1707-1714 : Anne (As monarch of GB)

* 1714-1727 : George I

So quite a few monarchs!

No, there is something objectively ridiculous in that association, also, please don't compare the scientific contribution of individuals at FAANGs to the scientific contributions of Newton.

> A more generous interpretation could be that on social networks, emergent properties are hard to predict beforehand with theory

Butbas far as we know, not a lot of people were actually interested in working on a theory (because why spend the time if you can just slap a neural network on it?)

Even if some people are, they are likely working inside Facebook and we might not know about it.

I agree.

The analogy there is more in comparing time periods than persons relevant to the development.

Blame popularity and popular culture, it skews history. He didn't even seem to promote the narrative, but people at large think Zuck invented the internet like elon invented rocketry.

They just used him as a placeholder to represent something modern in my opinion, likely with dismissive intent.

There are two approaches to compare: Starting with hypothesis and starting with data collection.

In former, hypothesis specifies which variables are likely independent, dependent, must be controlled, so on, then makes observations to confirm. In latter, data is first collected with as many variables as possible, then we attempt to identify which variables are independent, etc, then confirm.

To be cost efficient, many science experiments chose former. AI models use latter (get data, then find patterns, aka dependencies across variables). But even in latter, 1) human scientists can and do use the approach when appropriate, and 2) someone must have chosen the variables to observe which is a hypothesis (but lower precision is permitted in favor of higher recall).

Long story short: It is sensational and in the best interest of the media to make it seem like some drastic change is happening, but the way we the humankind is growing our collective knowledge is still fundamentally consistent, except with some added techniques thanks to low cost of studying large scale data, and it would serve us well to focus on the fundamentals and basics. The scientific approach.

Excellent comment. So many people seem unable to see that NN-based modelling is really just a new and powerful way of doing correlation analysis, and that we still reject the systems when the results don't line up with what we know of reality.

Ok. So AI is magic and we, as a society don't understand it, and it is used in science. Is there anything else in the article? Cause I don't see it.

Science is the process by which we reject verifiably incorrect ideas of the world. There are a lot of USEFUL tools on this planet which do not fall under that umbrella, and there have always been. AI is just the latest one we have.

Are we going to say that any sort of spiritual practice (which can be seen as form of unscientific therapy that can provide value to people) is "post-theory science"?

Are we going to say that a thousand year old medicine which did not understand placebo effects is "post-theory science"?

Are we going to say that when people meditate to understand how they feel, and then they can't accurately explain the biochemical processes is "post-theory science"?

Just because we can come up with very big functions which can approximate complex data (AI) means nothing about how we use them to understand the world.

The world has always had non-scientific tools to complement science. And always will have. Science always took longer to catch up. Over time some have become obsolete, because science figured out how to explain what they did. Some have not.

Gimmeh a break, there's nothing new here, science ends when we prove things are unknowable. Not when magicians, AIs and humans think other tools perform better.

> So AI is magic

No, this is absolutely the wrong takeaway from the article. NN's are not magic, they are completely understandable in terms of how they work and what they do.

However, they do not (easily, if at all) allow the construction of an explanation of why, given input X, they produce output Y. You cannot (easily, if at all) construct a theory of how they generate outputs. So you end up settling for (fairly, or even remarkably suprisingly) accurate predictions, but without the sense that you understand how they are generated, even though you are 100% clear on the mechanism.

That's not magic.

> There are a lot of USEFUL tools on this planet which do not fall under that umbrella, and there have always been. AI is just the latest one we have.

The point of the article, at its heart, is precisely to ask whether these NN-based approaches are in fact science or not. It's not insisting that they are, but pointing out that they are create a gray zone that is qualitatively different from the 3 examples you offer as other non-scientific approaches to understanding. Why? Because they operate in a domain still legitimately filled with much of the language of science, but not much of the technique. That's not true for spiritual practice, traditional medicine or meditation.

> Are we going to say that any sort of spiritual practice (which can be seen as form of unscientific therapy that can provide value to people) is "post-theory science"?

Are we going to say that a thousand year old medicine which did not understand placebo effects is "post-theory science"?

Are we going to say that when people meditate to understand how they feel, and then they can't accurately explain the biochemical processes is "post-theory science"?

I mean, we kinda do - in the sense that, yes, there is a lot of knowledge about actually effective medicine in those, but a lot less why particular remedies work and why others don't work.

This can easily lead to "cargo cult" like behaviour, where all you can do is blindly repeat certain rituals and hope they work, but you cannot really refine your methods or develop new methods.

(Actually, that's not quite correct. A lot of pre-modern science absolutely tried to develop theories and models behind their observation - as even the original "cargo cult" did - but those were wrong and didn't allow predictions)

> This can easily lead to "cargo cult" like behaviour, where all you can do is blindly repeat certain rituals and hope they work, but you cannot really refine your methods or develop new methods.

An important note. I an not in favor of those practices. All I claim is they can live alongside science, despite all their flaws. So can AI. Science is not becoming post-theory. It always lived alongside non-theory stuff.

My professor made fun of this article in yesterday's lecture. Nice to see it discussed here, too.

The article does make a few solid points, but the conclusion isn't supported. Even automated Deep Learning based science draws from theories. The data, the underlying statistics, the model, and how we interpret the results are all theory-driven.

We are not entering a new, objective era of science. We are merely chosing to ignore the theory and hiding behind a curtain of statistics.

>hiding behind a curtain of statistics.

If there's one thing that Covid has taught us, it's that The Modelling is a vengeful God who must be appeased with many sacrifices.

This is a histrionic way of saying you've been inconvenienced.

That is a very understated way to describe one of the single biggest upheavals in life and culture in the history of Western civilisation.

FWIW, a significant part of the "inconvenience" as a parent of two very young kids comes from my refusal to accept that putting kids in front of screens for extended and frequent sessions is an acceptable compromise when balancing the pressures of life under Covid restrictions. Apparently, most people have decided this is ok, along with other horrors like enforcing isolation requirements on young children.

> That is a very understated way to describe one of the single biggest upheavals in life and culture in the history of Western civilisation.

Come now. Black Death? The Depression? Two World Wars? And your main complaint is that your kids are having too much screen time?

I mean, don't get me wrong: in-person interaction is critical, and having it replaced by screen time is certainly detrimental. But not nearly as detrimental as having to sleep in bomb shelters every night, or killing 50% of the population.

You've made my point - the overall outcome post-Covid is comparable to a handful of other singular events in our history. I strongly believe that the ramifications of what we have done will take years to play out, but I accept that is just my opinion.

My kids are not having too much screen time. My complaint is that Covid restrictions have made the already-tough job of being an accountable, responsible parent almost impossible. Screen time is just the tip of the iceberg. You talk about sleeping in bomb shelters - how many kids have been told they now carry a disease that could kill if they get too close to someone, and what will that do to their mental and social development? How many parents now believe this justifies what would previously be called child abuse? Many of the ones I know, that's for sure.

At one time, it was considered brave to hug a HIV-positive person (and when the risks were unknown). Now, we selfishly impose social restrictions on those closest to us, to reduce an already-tiny level of personal risk.

In my editing I deleted the phrases "doesn't even register in comparison" and "I could go on". "Western Civilization" is littered with plagues and pandemics, wars, revolutions, and economic shocks which had far larger impact on people than COVID has. You're comparing having buildings in your town blown up, your friends and relatives and parents killed, your father sent across the ocean to be shot at, to be told to social distance. Your comment reeks of entitlement and privilege.

>You're comparing having buildings in your town blown up.... to be told to social distance

I very specifically didn't. I compared the overall societal outcome, to those other singular events.

>Your comment reeks of entitlement and privilege.

Yours reeks of poor reading comprehension and sweeping assumptions, as well as an inability to comprehend the scale of the different experiences different people have had under Covid restrictions. In my previous comment, you'll note that most of my concerns are for others, and for the society I live in as a whole. I don't know why I bother, as once again I am branded "selfish", "entitled", etc etc. In fact, due in part to the choices I and my close family have made over the past two years, it looks like we are going to do just fine. I absolutely cannot say the same for some of those I know. A close friend who recently had their first child has essentially had a mental health breakdown (I suspect, due to a mix of pre-existing inclinations, and the stress of being a first time parent under Covid restrictions), and due to the state they are now in, are refusing any external help at all.

The sheer number of stories like this will become more widely known in due course, and defending lockdown-style policies will become less and less palatable.

So on the one hand we have:

   1) negative impacts on individuals and societies of SARS-COV-2
   2) negative impacts on individuals and societies of policies designed to tackle (1)
and then we have the claim that because (2) is a non-empty set and contains specific sorts of issues that it inevitably will come to be understood as worse than (1).

That's a hell of claim to make. It seems to hinge mostly on one or both of two theories about the world (and the disease):

   a) deaths due to the virus were going to be about the same no matter what
   b) the deaths somehow count for less than the afflictions of the living

Good observation.

a) will be forever unknowable, although we can make some broad observations across various countries' responses to Covid and the overall outcome. EG: Australia has had strict lockdowns since March 2020, yet their cases are now spiking horribly (as I've heard so many times, with deaths lagging a few weeks). Where I live (Wales, UK) has had a mask mandate since September 2020. We were not allowed other people in our houses from March 2020 till mid-2021. Most places of entertainment have been closed around half the time, etc etc. Yet our observed outcome is objectively terrible.

All I can say is that my subjective experience has been dominated by the side-effects of Covid restrictions. I personally know only one person who has had anything more than relatively minor symptoms from Covid. I accept that my experience is probably biased in one direction, but I doubt there are comparatively many people whose life has been dominated by Covid itself.

b) is a contentious subject, and our inability to discuss it shows that we have become too detached from the reality of life and death as a society. Tell me - is it better that a young child has been correctly "isolated" in their room for many days, and told that it's necessary so they don't kill anyone with Covid, and grows up into a disturbed adult, or that an elderly person with several chronic health issues dies a year or so earlier than "expected"?

Death is not the worst thing.

as of Jan 12th, official deaths from COVID in the below 64 range: 213337

Same date, deaths in above 64 and older range: 621617

(EDIT: numbers are for the USA, but the general proportions seem to apply worldwide)


> it's necessary so they don't kill anyone with Covid, and grows up into a disturbed adult, or that an elderly person with several chronic health issues dies a year or so earlier than "expected"?

is pushing the bounds of reality a little when 1/3 of the deaths are in a demographic that is hardly described by what you've written above.

Consequently, I think a closer summary of your position would be that the psychic/emotional harm to people caused by our attempts to stop COVID from overwhelming the health care system outweighs the deaths of at least 200k people who were far from the end of life.

I don't consider that an indefensible position, but it needs to be stated clearly, with numbers so that people can see the numerical weight behind your moral take on this.

Yes, that is a good enough summary of my position.

I'm not too familiar with data and methods related to Covid deaths in the US, but here's a breakdown of deaths with Covid from the UK for 2020:


The deaths with Covid for the above 65 age group outweigh those in the below 65 group by 10:1 (and continue to drop exponentially with age).

Even with the large elderly and frail populations in the Western countries we inhabit, and the (frankly) enthusiasm for counting deaths as Covid deaths [1], the absolute number of Covid deaths is also still relatively small. It's certainly nowhere near the scale of the great historical pandemics, nor does the disease itself present as much of a horror as diseases that ran rampant within living memory - for example, smallpox.

[1] Early in 2020, medical guidance in the UK was changed such that it became significantly easier to record any viral respiratory death as Covid, the requirement for a confirming test was removed, and the requirement for a cornonary investigation was also removed. I can link the documents if required.

Re: counting. From my perspective, the excess death count is the only sensible count. That generally comes up with numbers significantly higher than the usual numbers.

Also, to repeat myself from so many past threads in so many places: the issue with COVID is a public health problem, not a personal health problem. The risk of death can be (and is) fairly low, but the risk of hospitalization is high enough that the primary public health care motivation is not preventing death from COVID but "flattening the curve" so that health care systems are not overrun. When that happens, normally non-fatal accidents and diseases become vastly more serious, and the overall risk and demographics of risk expand dramatically. This is why it makes little sense to focus on the mortality rate for COVID - that was never the central issue.

You are right about excess deaths, and once again, I am very grateful that the UK makes this data publicly available. I always struggle to find this kind of information for the US.

As for issues with healthcare - this is very pertinent for me here in the UK, as "protect the NHS" was the mantra for almost everything we've done since March 2020. It would seem that one of the side-effects of Covid restrictions (esp. as they apply to healthcare) is that, despite all of us sacrificing so much to "protect" our health service, and despite the NHS admitting record low numbers of patients since March 2020 (the data is publicly available), it has collapsed anyway.

Again, you appear to be blaming policy and models entirely for these problems, as if the virus doesn't even exist in your mind. So yeah, it does look a bit like the gripes of a privileged person.

As I said in another reply to you, the subject of discussion here is statistics and modelling, and it is very relevant because so many restrictions and new laws have been justified using modelled outcomes.

>as if the virus doesn't even exist in your mind

I suppose you'd go on to say I've "done my own research on Facebook", I'm an anti-vaxxer, etc etc. The virus is real. Our response has been vastly overblown and will continue to have have disproportionally massive side-effects (in my opinion). Those side-effects will mostly hurt others I care about (as I have described in all of my comments). Strange that complaining at all about the effects of restrictions, even when it is mostly about others or wide-reaching societal problems, makes me "selfish" or "privileged".

>That is a very understated way to describe one of the single biggest upheavals in life and culture in the history of Western civilisation.

Placing all of this at the feet of "The Modelling", and none of it at the feet of the virus itself from what I've read so far, is so ridiculous that it doesn't even warrant a discussion, sorry.

I don't know about things where you live, but where I am (UK, and more specifically Wales, where we have taken restrictions to an absurd degree), most major Covid restrictions have been justified as a necessary response to modelled data.

The OP, and indeed the article linked, is on the topic of modelling and statistics, so that is the subject of discussion.

I was involved in building an automated drug research lab a few years ago. Part of that was having in-silico simulation of chemical structures. Obviously there was quite a bit of "oh that means we can rapidly return results to customers without incurring the overhead of actually running the tests right? Great for our bottom line"

The answer was obviously no, at some point we still need to run actual tests. Without that we just have a proposition of what might happen. And the longer we leave running actual tests to feed back into the models the more "out of sync" with reality we risk becoming.

I think thats key to where this article just misses the real world situation. We're heading towards an environment where we can rapidly make propositions of outcomes for a hypothesis based on what we already know to help remove low % avenues and focus in on opportunities. Hypothesise and test still need to happen, but hopefully the predict phase will mean researchers hit dead-ends less often and earlier and can move to other avenues quicker and with fewer resources wasted.

And with "we" it's meant the citizen armchair scientist with degrees in google searching and social groupthink.

Statistics are meaningless without models behind them.

Statistics are not theories. Some forms of NN do not require models. And the whole point of the article is the contradiction of the claim "how we interpret the results are all theory-driven."

> We are not entering a new, objective era of science.

TFA does not contain the word "objective". That is not the claim at all.

During my PhD (and even in my jobs since) I have always been the “non-parametric guy”, the one who’s not good at math so left to figure out the models that don’t have any theory behind them. We were trying to estimate locations using Point Spread Function modeling and my boss was known for being the last theory guy in a field that’s pretty much given up on ground-up modeling of the PSFs for estimation. The reason was while we understood the Factors influencing the PSF it was just computationally expensive and there were too many factors (for many of which we wouldn’t know the parameters anyway). So non parametric estimation became a better alternative at least to get initial estimates.

This is no different from alphafold. It’s an acknowledgment that we know the the theory, we now just want a computational shortcut that allows us to move on to more important problems that have been blocked by not being able to predict protein structures from sequences quickly.

This type of non parametric (or in modern parlance ML) thinking is a great new adjunct method of thinking but this has not and probably won’t fully replace hypothesis based research. The author is clearly a science writer but doesn’t look like has actual science experience so they’ve just went and wrote whatever made sense to them. It’s an interesting problem anyway so anything that provokes discussion is a good thing.

What is science other than a set of theories? My quibble here is mainly definitional- I think there could be a world where engineering is post-theory; i.e. where most problems are solved by ai without anyone understanding how. Science though, is about developing an understanding of a phenomena, and science prescribes that proof of understanding should be codified into testable theories.

Also if theory is dead why are the scientists they interview in this piece obsessed with generating new theory...


Andrew Gelman has a great essay reviewing a paper that discusses this in the context of Covid:

>Greenhalgh’s message is not that we need theory without evidence, or evidence without theory. Her message, informed by what seems to me is a very reasonable reading of the history and philosophy of science, is that theories (“mental models”) are in most cases necessary, and we should recognize them as such. We should use evidence where it is available, without acting as if our evidence, positive or negative, is stronger than it is.


Ha, we've had this discussion back with Anderson's "End of Theory" [1]

Until we can come up with a way to automate large-scale robust causal inference, there's no end of theory. Because of course everything is correlated with everything in real life, the problem is isolating reasonable relations/networks/models of causal effects.

[1] https://www.wired.com/2008/06/pb-theory/

Exactly my thoughts

> they trained a neural net on a vast dataset of decisions people took in 10,000 risky choice scenarios, then compared how accurately it predicted further decisions with respect to prospect theory. They found that prospect theory did pretty well, but the neural net showed its worth in highlighting where the theory broke down, that is, where its predictions failed.

Err, what now? The neural net now substitutes for experiments as well? Replication crisis solved, I guess. Just don't deal with humans at all and social science finally makes working predictions!

I stopped reading after this sentence.

I can't wait until "trust the science" is finally replaced with "trust the AI"! /s

IMO the major revolution was the move towards statistics, especially when using observational data. Moving from a parametric to a non-parametric model or whatever is comparatively not a big deal.

Chucking a bunch of variables into a linear regression, getting an r^2 of 0.3 and calling it day is so far removed from Newtonian science that I'm not sure it should even be called science.

> Chucking a bunch of variables into a linear regression, getting an r^2 of 0.3 and calling it day

In reality chucking a bunch of variables into a linear regression is the easy part (well, with exceptions), getting ,or creating, good, interesting and important data that were not already analysed to death is a problem. Imho the revolution is the realising value in observational data and not using statistics. It's all about the data.

When discussing machine learning, data-mining or statistics with somebody far removed from the machinery (so for example from linguistics to biology), they always focus on what's possible with the tools but in reality the available data is what's constraining everything. Most tools (especially fancy ones like neural networks etc.) only work under certain constraints the are hard to achieve, it's not a hammer. You can always use a hammer and get results.

> Facebook’s machine learning tools predict your preferences better than any psychologist. AlphaFold, a program built by DeepMind, has produced the most accurate predictions yet of protein structures based on the amino acids they contain. Both are completely silent on why they work: why you prefer this or that information; why this sequence generates that structure. [...] They offer up no explanation, no set of rules for converting this into that – no theory, in a word. They just work and do so well.

Err, there is a theory: "What if we have so much computer power that we can try every combination and optimize for what fits best?". These algorithms aren't completely silent either, there's people behind them that write them, feed them data, guide the learning process, predict an outcome, etc - this breathless "We just don't know how it works???" is ignorant AI fearmongering, because the next line will be "What if it decides humans should be destroyed???" or whatever.

Some AI algorithms ended up doing racial profiling; that wasn't because "we just don't know how it works???", it was because it was fed racially profiled data. It's not magic, it's not unscientific randomness, and the outcomes can be predicted.

Engineering is not science, journalism is not philosophy, even calling things by their name is getting too much for people these days.

Insofar as the algorithms in the machine are adjusted to overcome past inaccuracies (whether they are adjusted by the machine itself or by the machine programmers) we are merely witnessing a new tool for use when developing scientific theories.

Whether or not we understand everything that the ML systems are doing, we can still observe if events corroborate their predictions, and will be forced to debug or reprogram them if they are not learning well.

Also, I am guessing that many if not most of these ML systems do not start learning from zero, but have some pre-ML naming, sorting, and classification scheme baked into their algorithms. If that is the case, increasing the accuracy of the scheme - refining the concepts of the theory - should only increase the accuracy of the systems' predictions.

If we consider "theory" for "sciences" that can actually measure outcomes, one could argue that they should be possible to formulate as a mathematical model, somehow.

For instance, kinetic energy can be given by: E_k = (gamma - 1) * mc^2 (technically, this is just one part of SR, and the fact that is consistent with the rest of SR is a benefit beyond simply the accuracy)

Another "theory" predicting cancer (or some other outcome) could be on the form: P=logistic(sum(beta_i * x_i)), where x_i are identifiable risk factors.

The latter would be the theory presumed in logistic regression, and is usually a gross oversimplification, but still very usable. Arguably, this is the same situation that we have with Newtonian kinetic energy (E_k=mv^2/2), which is also known to be a simplification.

I think the above direction extends naturally to Neural Networks, where the core assumption (linearity of logit(P) in X) is replaced by some given NN topology. In fact, logistic regression is just an example of (a very simple) such network.

In other words, the topology of the network (possibly even abstracted to a set of networks) becomes the new "Theory".

If we define "Theory" that way, I would argue that when comparing two such "Theories", and assuming equal explanatory power, the one that requires less computational power (or similarly, fewer parameters) is the better one.

As the technology matures, I expect Machine Learning / Neural Networks to be able to re-descover most of the theories where Science seems to be 100% accurate (like General Relativity), and to be able to "learn" the specific solution that minimizes the number of parameters. That solution may indeed by identical to Einstein's formulation.

In cases where our best theories are quite poor approximations (ie predicing cancer), machine learning models would be more complex/have many more parameters. I don't think that is a problem, as if we think we have a better understanding with current theories, we are simply fooling ourselves.

The protein structure prediction example (AlphaFold) is interesting, because the problem they’re addressing is not that we have no theory per se, but solving the geometry optimization problem directly with ab initio quantum mechanical calculations is not computationally tractable.

Protein structure prediction is not a matter of finding the best geometry (lowest energy). That's simply not true with the except of the simplest two state folders that form trivial geometry.

Sure, but you can search for local optima, and then there’s all the environment specific factors, solvation, finite temperature, that sort of thing. Really you want the local energy landscape so you can understand dynamics and stability. (It’s not really my field)

My main point is, AlphaFold isn’t displacing any general theory that predicts structure from sequence. It’s displacing other machine learning systems, which are mostly displacing just getting/making the protein and doing the X-ray crystallography (where you really do have a straightforward geometry optimization problem)

Again not my field so maybe I think about it a bit weird

The Guardian might have been good once, but I stopped reading after it became obvious that they now shill for the establishment, push irrelevant click-bait headlines, and beg for money every time you open the page.

There's no such thing as post-theory science because the job of science is to produce knowledge. Knowledge is the sum of methods, facts, and models that we use to understand the world.

When you're using ML models to reason inductively from data you can produce systems that do some work, they can aid with the practical and empirical aspects of science, but they don't replace science because they don't produce knowledge.

This is like the Skinner behaviourism debate from half a century ago but with a dog from Boston Dynamics instead of the organic version

But they do produce knowledge, just not in a form humans can easily reason about. We may not know how AlphaGo plays Go, but I think we can say AlphaGo knows how to play Go. The equivalent applies to AlphaFold and other algorithms that are producing useful knowledge.

Edit: Uh oh. HN may not be ready to accept this, so maybe it’s less progressive than I thought. The metaphysics of the Enlightenment went out the window a long time ago. Functionalism does not require Reason. Knowledge isn’t necessarily anthropocentric.

Not saying I like this either, but it’s certainly the transhumanist future we’re currently charting. Personally, I think we’re peering into a nihilistic abyss and need to fuse a metaphysic back into the loop to get back on the right track.

It's more of a philosophical question really, but are AlphaGo and AlphaFold "producing" knowledge? AlphaGo plays go, AlphaFold predicts protein structure. They are useful tools, but I wouldn't call what they produce knowledge. However, as you say, their underlying models have certainly captured knowledge that we, as humans, do not comprehend yet.

So I would reformulate the observation of the article as follows: we have become extremely good at building useful tools that embeds knowledge that we do not understand.

If I may attempt an analogy, it reminds of Ptolemaic tables: quite useful at predicting planetary motions with some accuracy. What they lacked in understanding, they made up with accumulated data and complexity. Maybe AlphaFold is just that: a Ptolemaic table built on top of a huge swath of data.

We will probably build a lot of these tools in the future, as we have very powerful tools to build them. But I don't think they will replace a more fundamental understanding (that is to say science), that will simply lag behind.

If someone or something produces knowledge, then yes, they are scientists and they need not be human. However it must at least be intelligible to them, and in that case it is that entity doing science.

ML systems as they exist do not produce knowledge for humans, and they do not understand anything themselves. They're mechanisms that do work. A chess computer doesn't understand chess any more than a lever understands physics, even if they can beat humans or lift heavy objects. Your TI-83 is not a mathematician despite the fact that it can calculate much quicker than you can.

If you're suggesting that at some point we may built some sort of AGI-scientist that can actually reason about something, potentially at a level we can't understand, sure. But that is not what ML is, that is science fiction, and it is not how ML systems will be used in the scientific process, they will aid as tools as they're doing right now, and that is what everyone will tell you who worked on AlphaFold.

You could say the same about humans; that we do not understand anything ourselves; that we’re mechanisms that do work. But it’s a moot point.

What I’m saying is that “reason” is an anthropocentric concept and is not a prerequisite for the production of knowledge, or even conducting science for that matter.

Reason isn't an anthropocentric concept at all. Knowledge isn't even produced. Knowledge is the subjective representation of facts, objects, skills and so forth. What's being produced by something like Alphafold is data, interpreting that data by means of reason and abstracting it into a model is what we call knowledge. The entire process is science.

And the first point you make is very important. The fact that we have neural nets in our heads that we didn't and don't really understand hasn't changed anything about what science is. We reason about the world and we do science because having general models about the world we are in is more powerful than merely collecting or correlating data. And that will be the case for any kind of intelligence.

We don't have a perfect model of reality to do science completely virtually using unsupervised ML. Science by definition is not mathematics as it is grounded first and foremost in empiricism. You can use ML to help produce science. E.g. hook up a sufficiently sophisticated "AI" system to do experiments and analyse data and generate hypothesises. But the AI itself won't obsolete the scientific method. I am surprised that the posted article is written by a science journalist, I suspect it is mostly due to her not understanding how ML is implemented rather than misunderstanding the scientific method.

> But the AI itself won't obsolete the scientific method

I wouldn't put a lot of money on that in the future given that even 3 years ago AI could prove about 58% of a set of theorems: https://mathscholar.org/2019/04/google-ai-system-proves-over...

From that to inferring then proving new theorems in the future or even inferring new theories in other fields (that can then be tested empirically if needed) I think we're not very far from what the person you replied to suggested

However, it just modeled a known algorithm for solving theorems mechanically. 3-SAT predates ML, and it was trained using it. Essentially parroting an algorithm is not science.

Call me again when AI develops a new theorem from scratch, in a concise manner

We have a tendency to reify science as something sacred or uniquely human, but the scientific method is essentially an algorithm. Like the article said, AI does not need to develop theorems in a post-theoretical mode of science. AI only needs a systematic way to make discoveries that are in the self interest of the system in which it's embedded.

Humans may need theory to perform science, but only because we're slow and have bad memories is it necessary for us to rely on faint signals like intuition, eureka moments, and pulling all-nighters in the lab. Machines don't need narratives and metaphors to represent their knowledge of the world, nor do they need reason or explanations of how and why. They just need to function.

Or it may be that the author has an understanding which you are lacking.

> We may not know how AlphaGo plays Go, but I think we can say AlphaGo knows how to play Go.

To be honest, before I can agree or disagree with that, I'd really like to know more about how AlphaGo actually plays Go. Like, how the network is structured, how an actual decision is produced, etc.

I feel, we're talking about increasingly lofty and abstract concepts while we don't have much of a clue what actually goes on on the ground.

Oh boy, just wait until you discover that all of science is based on induction

Is it though? In science one uses deductive, inductive and abductive inferences right?

The laws of thermodynamics, for example, just like all the laws of physics, are just consistencies we have observed not breaking after decades of trying every corner case we can conjure up. They are quite literally the definition of inductiveness: you see that something holds for a solid number of cases so you assume that it just holds for everything else too. Of course any scientist who hasn't just memorized a bunch of formulas but has actually grasped the reasoning behind science will tell you that even the most fundamental law of the universe has a possibility of being false, just like the Riemann Hypothesis can't be considered proven just because it works for an unfathomable amount of computed values.

Science is about creating models and then studying them and making predictions off of them. The making of the model is an inductive and empiric ordeal, whereas working on said model is a matter of deduction.

Karl Popper just turned in his grave.

Science does not produce knowledge. Science tries to attach a model for people to reason about what we observe. Its like solving x1 + x2+ ...+ xn= 10, where we dont the extent of what x values can be, or even what operations are being used, all we can observe is the result: 10. Science is saying it could be n=2 and x1 is 3 and x2 is 7.

That isn't knowledge, we as a species can never actually know anything, since that's saying that some things can never be proved wrong, which goes against the philosophy of science.

Fact is the enemy of truth, knowledge the enemy of science.

No and this was discussed like, years ago and was just as invalid then. What is motivating this junk?

"Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something."


I guess being that ML is a developing field it's worth checking back in from time to time.

But with counter-intuitive claims like:

> Facebook’s machine learning tools predict your preferences better than any psychologist.

breezily thrown in without anything to back them up, it doesn't feel like a very serious piece

This is from The Guardian, a popular (as in, for the general population) news publication. Expectations should probably be calibrated accordingly.

People are so reactionary here when they read the title it's disappointing.

"Post theory science" really probably means "explanations for complex behaviours are complex and in some cases beyond simply testable theories". We've seen this in quantum physics, where things like string theory are on the edge of testability.

In the first example in the article, they find limits of the behavioural-economic Prospect Theory. This isn't surprising - the tools used to develop that were pretty primitive compared to what we have now.

In this case this is a lot more like how science used to be done: collect data, make some observations, discover they don't align with the current theory and then people might be stuck for a while before they come up with a new one.

Learnt computer models are helpful because if they are good enough they simulate the real world better than many simple mathematical models do.

> We've seen this in quantum physics, where things like string theory are on the edge of testability

I mean... Quantum theory itself has actually given lots of solid predictions. They are statistical in nature, sure, but easily testable, and some can even be done at home (stuff involving polarized light filters).

String theory... Is just math that works out. I don't even know if it predicts anything. However it's certainly more complex than to allow us to say "we've seen this in quantum physics" without a PhD in the field.

Honestly, I find the article superficial. A cute but ultimately shallow idea looking for arguments to support it.

> People are so reactionary here when they read the title it's disappointing.

Indeed, I'm sad too when reading some of the comments here

> "Post theory science" really probably means "explanations for complex behaviours are complex and in some cases beyond simply testable theories"

Case in point: agent based modeling


> "Agent-based models are computer simulations used to study the interactions between people, things, places, and time. They are stochastic models built from the bottom up meaning individual agents (often people in epidemiology) are assigned certain attributes. The agents are programmed to behave and interact with other agents and the environment in certain ways. These interactions produce emergent effects that may differ from effects of individual agents. Agent-based modeling differs from traditional, regression-based methods in that, like systems dynamics modeling, it allows for the exploration of complex systems that display non-independence of individuals and feedback loops in causal mechanisms."


> "ABM enables an examination that would take years, decades, or centuries, to be performed in a matter of minutes. An important topic that must be addressed in any ABM application is the identification of the bounds of the work and the relationship between the model and the related real-world situation."

> An important topic that must be addressed in any ABM application is the identification of the bounds of the work and the relationship between the model and the related real-world situation

Here I think is the core of the whole problem: In case of people (as quoted "often people in epidemiology") how can we verify if ABM is correctly modeling people if we still don't know a lot about ourselves? For example we don't yet have a theory of consciousness, or we still are not able to accurately explain the process of how a child learns (yes we have multiple theories that we switch when one is not working) and so many other things.

The same can be said when we model other things from planets to animals and particles: How can we know that the model is accurate?

I am not saying the new way of doing science using "AI" is not science, but I am saying we need to move pass bold headlines and make sure that we results we take out of this have value and hold out.

Yeah, I think it is important to stress the point that the true strength of models like ABMs is that by systematically testing their behaviour for different circumstances and parameterisations can lead the experimenter to new qualitative insights!

To just run some simulation and trust it's output in any even remotely quantitative sense is no more scientific than reading the future from tea leaves!

Pretty much

The Popperian view that everything is testable and falsifiable (given that in the real world there are limited resources and time) is counter-productive. Going by that view no medicine or treatment are scientific (since none of them work 100% of the time)

Give me models that work and have predictive power. Skip the pedantry.

What? "This medicine produces X effect in Y% of the experimental group with Z p-value" is absolutely testable and falsifiable.

> "This medicine produces X effect in Y% of the experimental group with Z p-value" is absolutely testable and falsifiable.

Correct. It is, up to a range. If I replicate the experiment I'll have a different Y% which might be a slight difference or not.

But since it's not 100%, your theory about that experiment has been falsified.

It's easy to talk about falsifiability giving examples about black swans but that's a naive view in terms of results which are not clear yes/no answers and where multiple variables interact.

You’re describing reproducibility in checking a hypothesis about statistical association. There doesn’t need to be any theory involved to generate that hypothesis. Often there is, but often in modern drug discovery pipeline the lead generation comes first and theory comes later.

A theory-based hypothesis in this case would be something like, we think this disease is caused by this specific protein causes a disease, and this drug candidate will bind well to the protein and inhibit the disease causing reaction, so the drug will be useful to treat the disease. There are experiments to be done to check each element in that chain of theory, including the statistical association of disease outcomes

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact