
The declining authority of statistics and the experts who analyse them - prmph
https://www.theguardian.com/politics/2017/jan/19/crisis-of-statistics-big-data-democracy
======
mjfl
Statistics is not trusted because it shouldn't be. It is incredibly easy to
lie with statistics. There's even a tutorial:

[https://www.amazon.com/How-Lie-Statistics-Darrell-
Huff/dp/03...](https://www.amazon.com/How-Lie-Statistics-Darrell-
Huff/dp/0393310728)

Statistics is ultimately counting, and therefore is incredibly vulnerable to
discretion in choosing "what counts". Take the unemployment rate as one
example. When people realize that its not equivalent to "people who do not
have a job", how can you complain that they trust statistics less?

Expert authority is in decline because it should be, as there is an increasing
body of evidence that experts, from politics to medicine, have almost no
advantage in forecasting power than the average person.

[https://www.amazon.com/Expert-Political-Judgment-Good-
Know/d...](https://www.amazon.com/Expert-Political-Judgment-Good-
Know/dp/0691128715/ref=sr_1_1?s=books&ie=UTF8&qid=1485036840&sr=1-1&keywords=expert+political+judgment)

Why should "experts" (often just pundits) have any authority when they have
consistently demonstrated they deserve very little?

Finally, the political slant of this article, going along with the decried
"fake news", blaming the election results on these declines in authority, is
pathetic. It's basically an extension of "the other side is filled with
stupids" and has no credibility, no matter how you dress it in professional
journalistic veneer.

~~~
psyklic
Implying that most experts have agendas that supersede the truth is outrageous
and frankly dangerous. Saying we should just give up on using data to make
predictions is ludicrous -- even if it hasn't worked before in some
situations.

We should hopefully be in agreement that using data to make decisions is
preferred to winging it. We should also hopefully be in agreement that we
should trust people who devote their life to studying an issue over someone
new who merely has gut feelings about it -- and hence is even more biased in
many additional ways.

The thrust of your argument is valid -- summarizing data inherently leaves
things out. However, the solution to this is not to give up and throw
statistics out the window. And the solution is definitely not to unfairly
discredit people who devote their lives to these issues.

Rather, if you see statistics, wonder how they were generated and why some
data was included and others thrown out. By being open and candid about the
decisions made, the statistics are meaningful. Just giving numbers without
providing details of how they were achieved is not valid statistical work.

From my experience, most scientists (and myself in my work) care about the
truth over everything else. I use statistics to better understand my data. My
objective is always to choose the most sensible path in analyzing it that
allows me to most accurately answer the questions I pose.

~~~
engi_nerd
> Implying that most experts have agendas that supersede the truth is
> outrageous and frankly dangerous.

I think that "experts with agendas that supersede the truth" claim is true if
you consider the pool of pundits that regularly appear in mainstream media.
Unfortunately they are not representative of scientists like yourself.

~~~
psyklic
This attitude of not trusting experts and not trusting statistics
unfortunately permeates from the political arena into science.

The belief that all statistics are not to be trusted is a clear danger to our
society. Not trusting climate scientists. Not even trusting simple statistics
on vaccinations. As another poster noted, the only alternative is ignorance.

~~~
vanattab
Exactly. So what we need to do is reduce the misuse and cherry picking in the
statistics that make it to the main stream media (both sides) so that public
confidence in science does not degrade further.

------
lordnacho
Is the problem that statistics are losing authority, or that everything
connected with the elite is losing authority? From what I see, just about
every class of experts is less trusted now than a few years ago. Economists,
scientists (climate change), doctors (vaccines), and so on. Statistics are
used to back up every quantitative thing that's ever been studied, so of
course that quantitative element sometimes comes under scrutiny as well.

As for statistics as a national health monitoring system, it is a victim of
regime shift.

You collect the statistics that seem relevant when you are setting up the
collection. You look at things like money supply, unemployment, GDP and
manufacturing jobs because they seem to make sense at the time. Whatever
models people are working on seem to want those numbers, so those are the
numbers you collect.

Unfortunately, it's hard to model things for which you have no data. Social
science is especially messy, and so there are many ways things might interact,
but haven't yet, and some of those things will be a surprise to you, with the
added problem that the confounding variable is one that you haven't collected,
because you didn't think it was important.

This regime shift, which causes a new way for the numbers to be generated, is
to be expected. Why would the economy grow for hundreds of years, like it has,
in the same way? Looking at history, it hasn't. The structure of the economy
changes, as you would expect from any ecology that isn't in equilibrium. And
"The World Economy" has been growing for hundreds of years.

One example of numbers changing meanings was in the Economist the other day.
Manufacturing is not what it was. A lot of builders of equipment now make
money maintaining equipment or training people to use it. For instance I was
surprised to find out a professional coffee making machine is actually not
just a thing a coffee shop buys. They lease it, and along with the lease comes
training for baristas. Something that would skew figures.

------
lindig
The article doesn't mention the problem that we often try to predict the
outcome of a singular event when statistics can only tell us something about
it when repeating it. The prediction could still be correct even when the
actual event is not what was predicted.

~~~
omginternets
There's also something to be said for the replication crisis in certain
highly-politicized fields of sociology and psychology.

People (rightly, IMHO) realize that political activists sometimes work under
the guise of scientific researchers. It shouldn't come as a surprise that this
kind of conduct erodes confidence in the field as a whole.

Sadly, the baby has a nasty habit of being thrown out with the bath water...

~~~
kem
Not disagreeing with you, but the replication crisis is happening everywhere,
especially in the biomedical sciences. In fact, analyses have suggested it may
be worse in certain fields, like cognitive neuroscience (whether or not you
consider that a branch of psychology or biology is maybe debatable).

~~~
omginternets
Indeed, but I mention sociology and (social) psychology in particular because
of their clear potential for political slant.

------
coldcode
People often fail to trust what they don't understand, and instead believe
those who offer easy to understand but often false or inaccurate information.
People also look locally but are told things that apply globally and they
can't connect the two. This of course means they are likely to be manipulated
by people who know how to tailor the message. I don't know if our education
system is up creating a population that can understand what they are being
told and when they are being lied to. Life today is filled with so many
decisions and concepts and information and connections and maybe it is too
complex for most people to comprehend. This may just lead people to give up
and never get beyond easy answers.

------
Mendenhall
Statistics is a useful metric, but how they are used and what they are trying
to say while using them can often be very subjective.

Example USA-someone is collecting unemployment, in December their unemployment
money is up. They have no job, they want one and are looking.

By government standards in January they are no longer unemployed, they are
"not part of the work force" and ding! guess what unemployment rates just
dropped because that person is no longer unemployed, awesome!

Anyone interested in the reality of who is unemployed knows that statistic is
full of bs. In December that person is looking and wants a job and is
unemployed. In January that person is looking and wants a job and is
unemployed. Stats changed but reality didnt.

~~~
zigzigzag
Inflation statistics have the same problem. They typically don't include
house, stock or bond prices. At best, they include some convoluted and highly
questionable derivative of house prices like estimated rents. At worst houses
don't affect the indexes at all.

This leads to a situation where central bankers are publicly saying "hmmm,
we've done massive QE but there's no inflation, what a mystery" whilst the
average punter is seeing housing and stock market bubbles inflate right in
front of his eyes. It causes people to tune out: headlines like "inflation
statistics hit new low" are placed right next to "house prices reach record
highs".

------
jordache
Statistical analysis of poor data doesn't mean statistics is a poor method.

------
milesf
It's not that the authority of statistics is lost on me. What I don't trust is
the reporting of statistical information without transparent access to the
data and actual published results from knowledgeable statisticians.

I think the solution is to push calculus off the K-12 agenda and up to the
college/university level and replace it with probability and statistics
instead. Most people understand averages, but they have no clue what standard
deviation is. In fact (sad to say) many software developers don't even know
what SD is!

Until the general public are able to understand the power of numbers and their
pragmatic use, people who know how to manipulate the masses will continue to
do so.

------
basicplus2
Not sure statistics ever "accurately" represented the world

[https://en.m.wikipedia.org/wiki/Lies,_damned_lies,_and_stati...](https://en.m.wikipedia.org/wiki/Lies,_damned_lies,_and_statistics)

~~~
HillaryBriss
Yeah. The article makes a case that people have come to a newfound mistrust
for statistics.

But, that mistrust has been around for a very long time.

A big part of the mistrust is not in the numbers themselves, but in the people
who trot out the statistics and the way they carefully cherry pick numbers
that support their cause, ignoring other numbers.

~~~
nitrogen
_the way they carefully cherry pick numbers that support their cause, ignoring
other numbers_

Sometimes not so carefully. In a previous election, I remember seeing a
purported screenshot from an evening news show that showed poll results, but
skipped the #2 candidate entirely. Presumably they favored #1, and didn't want
people to think there was a viable contender.

This screenshot itself could have been faked, but we've seen other serious
blunders by TV news, like the fabricated names of officers on a downed flight.

~~~
zigzigzag
A recent example I saw was a friend of mine on Facebook, who posted a snapshot
from an MSNBC program. It was an attack on Rex Tillerson, in the form of a bar
graph of profit potential for ExxonMobil by country.

The countries seemed pretty random (that well known oil giant Germany was in
there) but at the top was Russia with a bar far far larger than any other
country on the chart. The graph was an obvious attempt to continue the theme
of "Trump administration is owned by Russia". The problem was that the numbers
next to each bar weren't labelled: the graph was totally lacking in context.

I did some googling and found that the graph appeared to be measuring profit
potential in millions of acres of drilling rights. Thus it was comparing an
acre of frozen Arctic ocean with an acre of Germany as if they were the same
thing. There are industry standard ways to compare oil fields. I've never once
seen oil fields measured in this way because it ignores extraction costs, but
sure, if you want to imply that Russia will be wildly more profitable than
anywhere else on earth then that's one way to do it. But it's an abuse of
statistics.

I pointed this out to the friend in question who simply said that she had a
policy of not taking part in political debates on Facebook.

------
afrancis
A problem I found with article is the talk of opacity of statistics techniques
in the guise of "big data" used in social media corporations. Statistics used
by individuals and companies is nothing new. As a counter-example, I feel the
public has greatly benefited from statistics in the form of quality management
embedded in the proprietary processes of manufacturing and large service
corporations. Stuff like TQM allow us to take for granted so many things. I
think the issues revolve around who owns and gets access to large data sets,
mathematical literacy and the morality/ethics of various players.

~~~
jessaustin
Perhaps from a moral perspective, statistics about inanimate matter like TQM
are better than statistics about people? It's somewhat reminiscent of the
Second Formulation of the Categorical Imperative, anyway.

------
DanielBMarkham
_"...Is there a way out of this polarisation? Must we simply choose between a
politics of facts and one of emotions, or is there another way of looking at
this situation?..."_

I feel extremely uncomfortable being put in the position of supporting
populist, right-wing causes simply because the essayist is doing such a bad
job, but here we are.

This author did a terrible job of understanding and explaining whatever the
hell it was that they were trying to understand and explain. Perhaps
"Populists hate science! How can we live with them?"

Heck with it. I'm not going to take the other side. There are a half-dozen
problems with how the argument is set up. Anybody with a passing
familiarization with "There are lies, damned lies, and statistics" should at
least be able to set up a counter-argument or two.

Statistical _science_ is great. It's the foundation of much of the western
economies. "Statistics", as that kind of numerical smorgasbord that's thrown
around constantly in the media and politics where correlation and causation
are the same? Not so much. If you can't see the difference, you really
shouldn't be writing essays supposing to inform us on anything.

------
general_ai
The "statistics" is only in decline because it was blatantly used to try and
sway the election. As in, oversampling demographics to get the necessary
conclusion, phrasing the questions such that you're "literally Hitler" if you
answer them "wrong" and so on. I quoted "statistics" because it's not
statistics, it's pretty obvious propaganda, using the most shameless tricks in
the book. Editors of Soviet Pravda would be proud. We know much of this thanks
to Wikileaks now, as well as because it was so blindingly obvious that only a
complete moron would think the polls were impartial.

I'm not afraid of "what's next" because as far as political polling is
concerned (calling it "statistics" is way too generous), it just can't really
get any worse than it already is.

~~~
gipp
The polls were no better or worse than past years. _Reporting_ on the polls is
a different story.

~~~
general_ai
Even the polls were cooked. E.g. to improve the odds of polls fitting the
narrative, democratic leaning pollsters would disproportionately sample
minorities known to vote predominantly democratic. The patron saint of
pollsters, Nate Silver, was consistently wrong about everything all the way
through the campaign. Now, if his main goal wasn't to peddle a particular
narrative, one would expect him to make corrections to his models after
getting things wrong, but no, he sacrificed his professional integrity
instead.

I'm sure this is not the first time this happened, but this year it was
particularly obvious, and I don't really trust the polls at all anymore except
when it comes to completely non-political topics.

~~~
mikeash
Nate Silver gave the eventual winner a 30% chance of victory, and his
prediction for the popular vote was off by only 1% for Trump and 0.5% for
Clinton. That's a damned good job if you ask me.

I can't fathom people who look at the results of this election and conclude
that the polls are all horrible. They got very close to what happened. Many
places took those poll results and interpreted them far beyond what they
should have, saying Clinton was a lock when she wasn't, but that's a different
matter. (It seems like they assumed that errors in swing state polls would be
uncorrelated, thus vastly underestimating the chances that a lot of them would
flip together.)

If you were trying to sway the election for Clinton, talking about how her
victory is assured would be a bad way to go about it. That just encourages her
voters to stay home. _If_ the polls were biased that way, and _if_ it was a
deliberate attempt to sway the election, then if anything it would be an
attempt to sway it for Trump. Which really doesn't seem likely at all.

~~~
tomjen3
They came out with the wrong result and that is _all that matters_. I don't
care how close a surgeon is to cut out my tumour, either it is gone or not.

~~~
mikeash
How is giving a 30% chance of winning to the person who won "the wrong
result"? Do you think that anything over 50% odds means a guaranteed win or
something?

------
conistonwater
The weird thing is that there are statistics that are based basically on
surveys, whose collection and analysis is essentially model- and assumption-
free. Like the unemployment rate, or inflation, for example. And yet there are
enough politicians who reject even those statistics, and that seems like a
totally different category of stupidity. When that happens, it's hard to
believe that the fault lies with the statisticians.

~~~
defen
One problem is that "unemployment rate" is a technical term that does not
match the colloquial / folk understanding. "There is a 5% unemployment rate"
and "25% of working-age adults do not have a job" are not mutually
inconsistent statements. So if you "know" that more than 5% of people are not
working but you keep hearing about the low unemployment rate (and multiply
that by 10s or hundreds of other little things like that), it's not hard to
make the jump to "the government is lying to us"

~~~
TheCoelacanth
Your example seems more like an example of people not having a consistent
mental model of what words means. A typical person would probably say that "a
working-age person without a job is unemployed" and "a full-time student or a
stay-at-home mom or someone who retired early are not unemployed" are both
true statements, but the two statements are contradictory. A statistician
cannot possibly come up with a definition of unemployment that matches the
colloquial understanding of what unemployment means.

~~~
zigzigzag
Of course they can. They can report the unemployment rate as "percentage of
the population physically capable of working who do not", and then provide
other terms to refer to "people who are looking for a job but can't find one",
maybe call it the jobseekers rate or something.

------
eva1984
Statistics never changes, what changes is the people who interprets them. So
if there is anything to blame, elites should blame themselves because people
don't want to believe their interpretation anymore.

------
briantakita
There's a 98.328% chance that statistics are used to deceive the consumers of
the lossy compression of information.

------
apathy
Reminder: the terms "statistics" and "statisticians" come from the common root
word "statist".

I am a trained, practicing statistician (clinical and medical, not
governmental) and this always gives me pause. "Data science" is a shitty term,
because science without data isn't science at all. Moreover, these days it's
just long for "IT guy". But I'm not sure what's better as a term for the real
thing, where experimental design and inference are as important as predictive
accuracy in one specific context.

Anyways. Statistics was always built around statism.

~~~
severine
I'm not sure I get your point, care to elaborate?

~~~
chris_wot
He's wrong, that's not correct. In fact it comes from "state", statism is a
term referring to the actions of a highly centralised government who handle
economic planning and controls.

"Statism" is a term that has been around since the 1850s, but "statistics" is
a term coined by Gottfried Aschenwall in the late 1770s.

~~~
apathy
Well, I didn't claim to be a trained historian, which is probably the correct
training for this type of work. Based on your temporal evidence, I must
concede the point.

However, I did not actually pull this straight out of my ass:
[http://www.thefreedictionary.com/statist](http://www.thefreedictionary.com/statist)

Thanks for the history lesson (seriously). I learned from it.

