
Statistics have lost their power (2017) - prostoalex
https://www.theguardian.com/politics/2017/jan/19/crisis-of-statistics-big-data-democracy
======
Majromax
Ultimately, I feel this headline ("Statistics have lost their power," in case
the Guardian changes it later) buries the lede. The dispute is not really
about statistics as a way of understanding the world, it is about the
epistemology of it, and at its core the idea that a national aggregate is a
valid proxy for well-being.

It's easy to decry:

> Presented with official estimates of how many immigrants are in the country
> illegally, a common response is to scoff. Far from increasing support for
> immigration, British Future found, pointing to its positive effect on GDP
> can actually make people more hostile to it.

... but the underlying belief appears to be closer to:

> If you live in one of the towns in the Welsh valleys that was once dependent
> on steel manufacturing or mining for jobs, politicians talking of how “the
> economy” is “doing well” are likely to breed additional resentment. From
> that standpoint, the term “GDP” fails to capture anything meaningful or
> credible.

The first statement makes no sense in and of itself, and it would seemingly
suggest a knee-jerk reaction against the very idea of data. However, in the
context provided by the second statement we see a different view: a core
distrust that "average well-being" (represented by the statistic of choice) in
fact represents the _common_ good, suitably defined[1].

As a result, I feel that the article's later musings -- that we're merely
collecting the wrong kind of statistics -- misses the core point. If we start
reporting on underemployment, then the right circumstances might come along to
discredit _that_ indicator, and so on. The article begins to grapple with this
issue in its digression into big data (and how there are no national agencies
for it in the manner of the ONS), but even there the author deals with a
practical problem of said data being more useful in the short term than long
term than the core problem:

Ultimately, if society as a whole does not have a shared view of the 'common
good', then _no quantity at all_ can measure it. The question of "is this
good?" becomes fundamentally unanswerable in a democracy with an identity
crisis.

[1] -- That could mean "median well-being," "well-being of people I know / in
my region / like me," or "personal, individual well-being of the speaker."

~~~
chongli
I feel like all of this is overcomplicating the issue. Now, at the risk of
oversimplifying, I offer this:

If I am not doing well and some journalist or some economist tells me the GDP
has grown a lot and everything is great, I am going to feel resentment toward
that person. Doubly so if they see my less-than-enthusiasm and start talking
about the “common good.” Why? Because the implication is either that there’s
something wrong with me (for not keeping up with the Joneses) or that I am an
acceptable sacrifice for the benefit of the common good.

Okay, it’s fine if you (the journalist or economist) think I’m an acceptable
sacrifice, but don’t expect me to be happy about it, let alone agree with you.
Don’t expect me to support the status quo, come Election Day. DO expect me to
try to throw a spanner into the gears. DO expect me to support a strongman who
says he’ll champion my cause over the cause of those who think I’m the
acceptable sacrifice.

------
prvc
Journalism has been one of the worst offenders with regard to the misuse of
statistics, not only in "soft" topics, but also in science reporting. Perhaps
statistical literacy ought to be normative in that profession the same way
that checking sources and using correct grammar (supposedly) is.

~~~
mikelyons
Even if statistical literacy was normative wouldn't people's selfishness still
cause them to use/abuse statistics to influence those who are less literate in
them?

~~~
lostmyoldone
Knowledge can be abused, but lack of knowledge will almost certainly lead to,
unintentional, abuse. Especially in the case of statistics.

In many cases, reporters want to be dramatic, but not liars. However, the lack
of deeper statistical knowledge often lead to them becoming liars, even though
they only wanted to be dramatic.

------
cjslep
As usual, the title's description is wrong, and the real news is buried and
overlooked in the story:

"People assume that the numbers are manipulated and dislike the elitism of
resorting to quantitative evidence."

If people don't believe that something is useful, it doesn't matter what that
"something" is, it will start to have a diminishing affect our society. It
goes for statistics and liberal democracies just as much as ancient gods and
ancient languages of yore.

~~~
traderjane
That's like saying if people don't like military drones, then military drones
will start to have less relevance; but some technologies are sufficiently
powerful in their own right that it needs only a few people to recognize their
power. Then the stress of competition can do the rest.

I doubt statistics or mathematics gained its influence because various
cultures were fond of symbolic play. We study these things because we expect
concrete _payoff_ , not because of cultural affection.

So is social science lame? Or is Google insight scary? At some point a
deliberate posture of mis-devaluation will come to terms with reality.

~~~
cjslep
> That's like saying if people don't like military drones, then military
> drones will start to have less relevance;

Sure I'll back that up. Thought experiment: if everyone, including all the
drone pilots, suddenly magically stand up all at once, walk away, and never
talk about it again, then yeah it won't impact humanity.

Is this particular scenario realistic? No. Has mathematic or chemical
knowledge been collectively lost before in human history, and been eliminated
from society? Yes.

> but some technologies are sufficiently powerful in their own right that it
> needs only a few people to recognize their power. Then the stress of
> competition can do the rest.

Sure, those few power can recognize its power, but if it's anything short of
immortality, never underestimate the power of every other human being being a
Luddite wielding a pitchfork ready to kill over progress, because it violates
a cultural norm. Doesn't matter if your statistics can find a cure, societies
and collective humans' irrationalities are expert suppressors of technology as
both a mean and an end. Look no further than the US stance on nuclear waste
reprocessing. The science and power are there. The will of people, is not. Or,
look at South Africa and Ukraine giving up nuclear weapons. Power? Yes.
Popular will to hold onto this power? No.

> We study these things because we expect concrete payoff, not because of
> cultural affection.

Exactly, because society values concrete payoff, things like "derivations from
first principles" and "following the scientific method". But if society as a
whole pressures people to never pursue ideologies along these lines, or turn
actively hostile against it (book burning, HDD erasing).

I take this article as an alarming warning. Scientific progress is itself not
an end. And it is not an inevitability. I like scientific progress and
statistical tools, this article just goes to show how careful -- ethically
responsible -- those of us wielding them must be when living in our society.
And call out bad actors.

This is why engineering professions have ethics boards. If engineers build
bridges and bad actors build bad bridges, people won't trust bridges anymore.
And if people don't trust bridges, they don't get used, so more don't get
built.

The greatest failure of the "modern" education has precisely been the removal
of religious ethics _without_ an equivalent amount of time working to help
students develop their sense of personal ethics.

------
threwawasy1228
This is an objectively good thing, most statistics don't actually tell most
people anything interesting about the world. They don't aid the average
untrained guy in the street in gaining a better understanding of the world
around them. They are bombarded with opinions that are based on contradictory
studies and statistics all day, this is bad in politics and sociology but
medical statistics are probably the worst offenders.

Wine is good for you, chocolate is bad for you, cholesterol is good for you,
wine is bad for you, red meat good for you, Fluoride bad for you! I could find
you a statistically significant study supporting any conclusion I want right
this minute. Since that is the case, why should the average Joe believe any of
them? This is made even worse by news articles and popsci blogs not even
linking to the source papers, so avg Jane can't even investigate further if
she wants to.

~~~
lotsofpulp
Statistics aren’t bad. Extrapolating unwarranted conclusions is bad.

~~~
threwawasy1228
I am not a trained medical professional, and I don't have time to be
scrutinizing and reading background literature to gain a full understanding of
the context each specific study has within its sub-field. I don't have access
to whether these conclusions are generally accepted or are commonly
disregarded for some technicality in the methodology that I didn't understand.
Me personally, I can't be an expert in everything all the time, and I can't
waste my life gaining a full breadth of understanding of every topic. I rely
on extrapolations to do this for me, if those extrapolations are commonly
faulty, I won't pay any mind to them.

~~~
lotsofpulp
Unfortunately, this is the best option for modeling the world around us that
humanity has come up with. Throwing out the only tool we have to evaluate and
re-evaluate what we consider to be knowledge is not the best course of action.

Solution is to support proper scientific procedures, support civic
organizations carrying out the important tasks of vetting claims, and being an
example of someone who values the correct protocols when evaluating data and
making conclusions (or not making them).

It's a very slow process, and we're lucky we're in a timeline where the work
of various people throughout history has gotten us here. We'll have to work to
keep it going further, and literacy in math, statistics, and the scientific
process is crucial.

~~~
glofish
I disagree here strongly. It is not the best option. Statistics has become the
tool of obfuscation and making exaggerated claims.

Compare the two statements: "10 out of 80 people that took the drug lived a
month longer but were extremely miserable."

Versus: "A new cancer drug doubles the average survival rates scientists say".

Which one do you understand better? Here I just made up some examples, but as
a scientist I'll tell you just about all the papers I read are in the second
category, it takes an unreasonable amount of effort to get the first type of
infomration out of the publications.

~~~
lotsofpulp
I don't understand the chain of reasoning in your claim regarding the 1st and
2nd statements, and statistics in general.

When I use the word statistics in this conversation, I mean the use of math to
analyze and compare data. This can be done inappropriately, of course, but the
problem doesn't lie with the principles of math (including statistics), but
the agent using them inappropriately.

And medical (or biology related) statistics are used most inappropriately, due
to various cognitive biases and financial incentives. But the problem is not
statistics, and it's still the best tool we have unless we can find a Magic
School Bus to shrink us down and go observe the chemical reactions in real
time. But I know that stats are abused, so anytime I see a fantastical claim,
especially in medicine, I approach it much skepticism.

~~~
glofish
There are fields of science that have banned certain statistical measures:

Psychology journal bans P values

[https://www.nature.com/news/psychology-journal-bans-p-
values...](https://www.nature.com/news/psychology-journal-bans-p-
values-1.17001)

I happen to agree with the sentiment expressed above that the entire concept
of hypothesis testing is fundamentally flawed. This is what you call the "best
tool". It is not, it is the tool that is used because it allows for the most
fudging of results.

------
motohagiography
The quote, "using statistics like a drunk uses a lamppost, for support and not
for illumination" comes to mind.

Data is great for illuminating things, but for arguments, it's usually cherry
picked. I do think that writers for the guardian vastly underestimate the
intelligence of people who don't agree with them. I generally mistrust
statistical arguments because someone forearmed with statistical talking
points has usually already signalled their intransigence. Data is nice, but
alignment of interests and incentives is better.

It reduces to, "according to criteria and data I have selected, your argument
does not bear out." It is a passive appeal to the authority of whoever
provided the number, and not meaningful or logical information in itself.

Logical arguments tend not to persuade anyway, but to say someones argument is
illogical because it does not resist a coded illogical premise (appeal to
authority fallacy), is certainly a way to make them give up on arguing with
you.

------
PaulKeeble
It is more about declining trust in the source of that information and the
institutions that spread it around. People have been given plenty of reasons
to distrust the various sources of information over the past decades.

------
caseysoftware
Even if the underlying statistics accurately represent the situation
correctly, bad analysis on top of it creates the same results. From last week:

 _" Paul Krugman and other mainstream trade experts are now admitting that
they were wrong about globalization: It hurt American workers far more than
they thought it would."_

Ref:
[https://news.ycombinator.com/item?id=21357096](https://news.ycombinator.com/item?id=21357096)

~~~
QuesnayJr
That Foreign Policy article illustrates exactly why the influence of
statistics is on the decline. If you read Krugman's original article
([https://www.bloomberg.com/opinion/articles/2019-10-10/inequa...](https://www.bloomberg.com/opinion/articles/2019-10-10/inequality-
globalization-and-the-missteps-of-1990s-economics)), you can see Krugman make
several points, all using statistics. One point is that the effect of
globalization in the 90s was small, but it had an unexpectedly large impact on
manufacturing in the 2000s. The second point is that the overall effect
disguised the fact that certain industries were particularly hard hit. He then
cites the papers where you can find the details. These papers only exist
because of the statistics they do. Krugman is a clear writer, and he is taking
the unusual step of admitting he was wrong, so his article should be
interesting.

But nobody cares about that, because nobody cares about statistics anymore.
They want that dopamine hit of outrage, so they read the FP article, that
deliberately misrepresents the argument (notice how "manufacturing" suddenly
becomes "workers", despite the fact that most workers are not in
manufacturing), and doesn't give any evidence. It just quotes a bunch of
people settling scores with Krugman. Even there, it mischaracterizes, for
example, Krugman's review of Greider's book to heighten the outrage.

Statistics is boring, and rage is interesting, so the Internet era optimizes
for rage.

------
08-15
This single sentence explains the problem very well:

> The thinktank British Future has studied how best to win arguments in favour
> of immigration and multiculturalism.

BF didn't set out to figure out whether immigration and multiculturalism is
favorable. They only tried to figure out how to use statistics (or anything
else) in support of their foregone conclusion.

Maybe the headline should be "Statistics have lost their usefulness for
propaganda."

------
Gimpei
I think the solution to this problem is more statistics. In particular, we
need a mandatory statistics course at the high school level. The problem is
that people are bombarded with "studies" right now claiming to prove stuff and
many of them do no such thing. If people had a better sense of what is
provable and what isn't, what makes for a good study, how much you should
trust the results of a single study etc, I'd hope that they'd be far more
informed and effective citizens.

~~~
glofish
that is the cliche definition of insanity:

"doing the same thing over and over and expecting a different result."

probably the most common mistake students everywhere make. They keep taking
more courses - it never helps just makes you more confused.

------
deogeo
I can't help but laugh. Make the slightest mention of statistics broken down
by race, and the same publication that is despairing of how people no longer
trust statistics, would be finding innumerable reasons why _those_ statistics
are invalid.

~~~
jayd16
Why does that make you laugh? Your take away should not be "well clearly
everything someone calls stats should be supported". The article is about how
stats are becoming dismissed before examination, not that they shouldn't be
scrutinized.

~~~
deogeo
And that all the examples given of uneducated masses distrusting statistics
just happen to align with the Guardian's pet causes is of course pure
coincidence.

Really, that's the perfect example of _why_ statistics are distrusted - only
those statistics that align with someone's agenda get presented. And
sometimes, they're not even gathered in the first place:
[https://www.thelocal.se/20180508/why-sweden-doesnt-keep-
stat...](https://www.thelocal.se/20180508/why-sweden-doesnt-keep-stats-on-
ethnic-background-and-crime)

~~~
jayd16
You're describing a separate phenomenon from what the article is about. Poking
at statistics with biased/politically motivated effort is not the same thing
as dismissing the field of statistics entirely.

The article is not suggesting that statistics should not be scrutinized.

------
jammygit
Too many science projects reporting statistical results that do not hold up in
repeat experiments; contradictory media coverage of the same topic repeatedly
as a result.

Reduces trust in science, statistics, and the media all at once.

~~~
lotsofpulp
The problem is verifying experiments gets no funding.

~~~
munk-a
I think the bigger problem is that the initial experiments are usually funded
by interests with a desired outcome in the study - instead of being funded by
grant money or educational funds.

------
glofish
It is a 17th-century concept applied to 21st-century data collection.

Plus it gets misused all the time - even by statisticians that should know
better.

------
freejulian85
At one point does policy targeting a statistical measurement render that
statistic useless? I’m looking at you, CPI.

~~~
Majromax
> At one point does policy targeting a statistical measurement render that
> statistic useless?

When the statistic is a proxy for a quantity of interest, and when policy
exploits ways of manipulating that proxy without commensurate change in the
underlying quantity of interest. My driving policy of targeting my speedometer
does a fine job of controlling my vehicle's speed even though the former is
just a potentially-inaccurate measurement.

In the case of the CPI (actually the PCE for the US Fed [1]), the quantity of
interest is the "[nominal] cost of living." To the extent the CPI doesn't in
fact measure the cost of living, the easy solution is to change to a price
index that _does_ measure the cost of living. Since living tends to involve
buying and consuming things, a price index, suitably defined, can be a fine
measure of this.

To the extent you are concerned about the exact ways in which the CPI is
computed, independent measurements [2] generally reproduce the broad trends in
US data, which in turn suggests there is no significant manipulation by the
BLS.

[1] --
[https://en.wikipedia.org/wiki/Personal_consumption_expenditu...](https://en.wikipedia.org/wiki/Personal_consumption_expenditures_price_index)

[2] --
[http://www.thebillionpricesproject.com/](http://www.thebillionpricesproject.com/)

------
MR4D
90% of statisticians agree with the conclusion.

The other 90% challenge the sources of this article.

(With apologies you Yogi Berra)

------
al2o3cr
Describing this entirely as "the declining authority of statistics" is like
saying the WTC towers suffered "declining structural integrity" before they
collapsed - there are _people_ behind both...

~~~
Google234
God damnit, you should know better than bringing up WTC towers here. It
attracts conspiracy theorist like how moth are drawn to a flame.

