
Lessons in statistics: How not to analyze time series - xtacy
http://wmbriggs.com/blog/?p=14718
======
signalsmith
As a mathematician, that article was a little embarrassing to wade through.

While there are valid criticisms of the way statistics are sometimes misused
in science, pretty much every one of them comes from lack of understanding
about how statistical models work - scientists reaching for a familiar test
and following a formula they were taught. I can't blame them too much -
understanding the true purpose and nature of statistical models is HARD (my
recommended step one: become a Bayesian). What we need is for more people to
recognise when they don't have that understanding and work with somebody who
does.

What Briggs seems to have done, though, is decided that because _HE_ doesn't
understand statistical inference and modelling, that statistics are bunk.
Taking a simplistic definition of "trend" like "the second-half average is
higher than the first" and turning that into a boolean yes/no answer is the
statistical equivalent of being an anti-vaccer.

The most frustrating thing, though, is that all the alternative definitions of
"trend" he defines can actually be expressed as statistical models! The issue
is that when you express these definitions/tests for "trend" as models, you
see that the statements each model makes about the underlying system are very
problematic.

TL;DR - Briggs doesn't understand statistical modelling, and has therefore
concluded that his home-rolled tests are just as good.

~~~
nkurz
(responding both to you and 'duckingtest')

No, the 'answer' is not that the person you disagree with "believes in magic"
or "doesn't understand" the field they work in as respected experts. Appell
goes on to try to explain why climate modelling is concerned with 'projection'
rather than 'prediction', and has a very clear view of the difference between
science and magic.

Briggs has been a professor of statistics at Cornell, written several book on
statistics, and published dozens of peer reviewed papers. Claiming that he is
the "statistical equivalent of an anti-vaccer" might be a useful rhetorical
strategy but unless you can point to some prominent mainstream career vaccine
researchers that you'd also lump into this category, it has no basis in truth.

Playing to a home crowd is easy, but has left us where we are with two locked
down groups each trying to persuade the undecided middle that their opponent
is the devil incarnate. This doesn't lead anywhere desirable, just to further
entrenchment of positions. The "way out" is trying to formulate criticism that
can be understood by the party being criticized and influences them and their
followers to reevaluate their beliefs and approaches.

I think it has to be something stylistic that we as readers associate with a
world-view (a dog whistle) rather than the actual content of the statements.
Take the paragraph in "Lesson 1":

    
    
      If you say the mixed marriage of splicing the disjoint 
      series does not matter, you are making a judgment. Is it 
      true? How can you prove it? It doesn’t seem true on its 
      face. Significance tests are circular arguments here. After 
      the marriage, you are left with unquantifiable uncertainty.
    

I'd assert that this paragraph can only appear on the 'denier' side of things.
Is this because it embraces uncertainty, and implies we should not act until
we have properly quantified it? One of the most interesting exchanges in the
comments is between 'Sheri' and 'Brandon Gates', on whether it's better to
accept a known-flawed model or to throw it out and have no model at all.

My request to 'signalsmith', concentrating on Briggs because his positions are
more self-contained: Re-reread Briggs article, ignoring the set-up paragraphs
that explain what he's criticizing and why. That is, blind yourself as best as
you can to the side that he is taking. Stop at the point where your pulse
starts to rise and report back. Then let's see how 'duckingtest' responds to
the same passage.

------
nkurz
This is a very interesting piece to read. The amazing thing to me is how smart
people can look at the same evidence and come to very different conclusions.
To get the feel of this piece, you really need to read it along with the
comments.

The author of the piece is a very talented statistician, with a PhD in
Mathematical Statistics (I hesitate to think of what other kind of statistics
someone might study?), a Masters in Atmospheric physics, and a BA in
Meteorology. Without question, he is very extremely well qualified on paper to
make write the post he did:
[http://wmbriggs.com/blog/?page_id=1085](http://wmbriggs.com/blog/?page_id=1085)

The main critical commentator is David Appell. He's a freelance writer who's
been concentrating and reporting on global warming issues for many years. He's
got a PhD, Masters, and BA in Physics, has interviewed most of the main
scientists in the field multiple times, and understands the issues as well as
or better than anyone else writing about it in the mainstream press:
[http://davidappell.com/](http://davidappell.com/)

And it appears that the two completely disagree about practically every line
in the article!

Without doubting the credentials or understanding of either, I can't shake the
feeling that one is following the data wherever it leads, and the other could
rationalize any result under the sun and not be knocked off a single one of
his talking points. And I'm sure that others can read the piece and the
comments and have exactly the opposite reaction.

What is it about their prose that can produce generate in me this amount of
certainty and trust? Somehow, one is able to signal to me that they share my
worldview, and thus I'm willing the trust them on the details I'm unfamiliar
with. The other loses my trust within the first paragraph. At some level I
know that neither of these responses can be fully trusted, and yet I can't
shake the feeling that one can be trusted and the other cannot. How can this
be?

~~~
duckingtest
>At some level I know that neither of these responses can be fully trusted,
and yet I can't shake the feeling that one can be trusted and the other
cannot. How can this be?

David Appell said it himself that what he believes isn't based in fact and
can't be disproven in any way, in other words, a religious belief. Seems you
just prefer science to religion.

"In climate science you can’t “predict” anything — climate models aren’t
capable of predicting, because no one can read the future."

------
phrixus
Wow, the extensive comment section is eye-opening to me, in how little
consensus there exists about the validity of models, line-fitting, and
prediction.

~~~
dxbydt
I am amazed at the level of erudition in the comments section as well. Shows
what a highly literate intellectual audience that part of the world must be.
Had this article been printed in the Times, you'd have the usual republicans
vs Krugman name calling and none of this scientific minutiae.

