In terms of public policy you have to decide what you're optimizing for and that decision can't be made with data alone because it does not help resolve questions of value and fairness.
A lot of arguments about science are really arguments about confidence. E.g. most climate change scientists are fairly sure about their models, but the lack of absolutely certainty makes it possible for deniers to cherry pick a tiny collection of outlier scientists who will argue in public that it's all nonsense.
Policy makers and the media are some combination of corrupt and clueless, so they're happy to go with the false equivalence this creates.
One way to depoliticise science would be to have an international science foundation, which was funded independently of any individual government.
Of course there would be squeals of disapproval from vested interests, but that would simply highlight the problem - the vested interests don't want independent criticism or oversight. Their entire MO is based on regulatory capture which gives them the freedom (for themselves only) to operate as they want with no personal or financial consequences.
Scientific accountability would set them on the path to democratic accountability, which is the last thing they'll accept.
I think scale/proportion is also a problem. Humans seem to place a lot of value in narratives/stories but we aren't so good with quantities (e.g. https://en.wikipedia.org/wiki/Conjunction_fallacy ). Pretty much everything (economics, climate, etc.) has factors pushing it in different directions, so we can always find a counterargument to any position (e.g. we can rebuff climate change by pointing to solar cycles, CO2 causing extra plant growth, etc.); that's fine, but some factors are overwhelmingly more important than others, whilst we seem to cling on to these stories/narratives and give them more equal weighting than we should.
As a concrete example, a family member used to leave their lights on overnight, claiming that "they use more energy than normal when they're first switched on". Whilst true, the saving is cancelled out after seconds ( e.g. https://www.energy.gov/energysaver/when-turn-your-lights )
Practically no systemic analysis is done within government. At least in Chicago. Government systems are compartmentalized in ways that makes interfacing with them impossible for any worthwhile analysis. Example: the only analysis that Chicago's finance department had done on its parking tickets is a single very high level spreadsheet.
Some analysis is better than no analysis.
This is so far from being true, it undermines your point and your post.
The Federal Reserve does no systematic analysis? The U.S. Treasury does no systematic analysis? The Bureau of Labor Statistics does no systematic analysis? The Congressional Budget Office does no systematic analysis? The Centers for Disease Control do no systematic analysis?
They're actually setting other people up for disaster, hoping that they will already be gone when it hits.
I don't agree with that. Doing some analysis on a limited and possibly skewed data set can lull you into a false sense of understanding. It makes your ideas seems objective when they can in fact be completely baseless.
It's hard to believe that the percentage of successes with no analysis is GTE the percentage of successes with some analysis.
And additionally, you now have the pride/confidence thing in your even worse results because you did "analysis"...
The problem is that most in government simply are not systems thinkers. They are focused on values and fairness, and believe that once you’ve identified those values, you can directly legislate those into outcomes. This thinking leads to spectacular failures (the war on drugs, the war on poverty, tough on crime, etc).
I don't think the main problem is that "people in the government are stupid". Real life and societies are much more complex than a jet engine. There are thousands of value goals, too many to be able to put numerical targets on each one.
What is the optimal ratio of potholes to unsolved murders?
And when you forget to include any of the value goals in your model, then you get a paperclip AI scenario with terrible effects somewhere else.
Sure, but once you decide, then you should use data to optimize it. Seems pretty straightforward, and I don't think anything in this article would disagree with that.
You are also assuming we can agree on what to optimize for. In fact we do not, and will not.
You can't use data to optimize anything? Google and Facebook must be wasting their time collecting all that user data, then.
> You are also assuming we can agree on what to optimize for. In fact we do not, and will not.
We agree on lots of things to optimize for. There are cases of disagreement, but very very broad agreement as well.
No, I can use data, the problem is I don't know what all the effects of that optimization will be. So I constantly have to change what I'm optimizing for.
> We agree on lots of things to optimize for.
Broadly, but there are limited resources and each thing effects the other. So even though we agree that A and B are worth optimizing for, we will disagree on which is more important. Worse in many cases we will agree on A and B, but the data shows you cannot optimize for one without pessimism the other.
That is the broad agreement isn't really enough to do anything with, we need the details and there we disagree.
What are you trying to say here? Optimizing things with data is hard?
> Broadly, but there are limited resources and each thing effects the other. So even though we agree that A and B are worth optimizing for, we will disagree on which is more important. Worse in many cases we will agree on A and B, but the data shows you cannot optimize for one without pessimism the other.
I think we even agree broadly enough on the relative weights of many things to optimize for them. And I think we at the very least agree enough on things to partially optimize them, or pareto optimize them. In many cases there are low-hanging fruit to be picked that can optimize a metric that we all agree is good without sacrificing another.
The academy is the wrong place to direct politics, because politics already direct the academy.
From my perspective, a more likely overall improvement in the ongoing quality of policy would be a requirement that all policies sunset reasonably soon by default, even if they don't seem divisive at the time they're passed. As it regards controlled substances, this would lower the bar for repeal to "nobody particularly cares to renew it" from "nobody particularly cares to do the work to repeal it".
A lot of the most important debates of the last 100+ years has been on exactly this debate.
Rationalists like Ayn Rand (sort of), Rosa Luxemburg (red Rosa) and others on one side, and the like of Karl Popper & Hayek on the other.
When people talk about "testability" of a theory to decide if it's a scientific theory... they are borrowing from Popper's criticism of Freudianism, Marxism & the concept of metric based "government science."