
Eric Raymond on "Hiding the Decline, Part 1" - skmurphy
http://esr.ibiblio.org/?p=1447
======
martincmartin
As The Economist points out:

"The 'hiding' concerned the decision to leave out a set of tree-ring-growth
data that had stopped reflecting local temperature changes. That alteration in
growth pattern is strange, and unexplained, but eliminating it is not
sinister."

and:

" 'I’ve just completed Mike’s Nature trick of adding in the real temps to each
series for the last 20 years (ie from 1981 onwards) and from 1961 for Keith’s
to hide the decline.' Trickery associated with Dr Mann was catnip to the
sceptics. But Dr Jones has clarified that 'The word trick was used here
colloquially as in a clever thing to do. It is ludicrous to suggest that it
refers to anything untoward.' "

[http://www.economist.com/opinion/displaystory.cfm?story_id=1...](http://www.economist.com/opinion/displaystory.cfm?story_id=14960149)

~~~
Perceval
My concern about the tree-ring data is that, on face, it seems like
tautological reasoning. Please correct me if I'm misrepresenting what's going
on.

The tree ring data, after a certain point in history, is no longer a reliable
indicator of local temperature changes. There is no explanation for why trees
no longer reflect local temperature changes. We know that they no longer
reflect local temperatures, presumably, because we have other more reliable
measurements available now (measurements that are not available further back
in history).

This leaves us with a conundrum: is there any period in which alternative
measurements and tree-ring measurements correspond with one another? Is the
divergence between our measurements and the tree-rings something we can trace?
Is it sudden or gradual? If there is no period in which there is a
correspondence between our measurements and the tree-rings, what does this say
about our assumption that tree-rings have ever accurately captured local
temperatures?

It seems that the implication is that changes in the environment have rendered
tree-rings unreliable. But if we're arguing that tree-rings are unreliable
because of environmental change (i.e. the chemical changes to the composition
of the atmosphere and water cycle that contribute to global warming), and
we're seeking to explain environmental change, we're begging the question
(assuming the explanandum in the premise of our argument).

Until a causal mechanism for the divergence between tree-rings and other
measured temperatures can be specified, how can they legitimately be discarded
as evidence, without invoking tautological reasoning?

~~~
btilly
And this is why people who don't know science should ask questions first.

 _The tree ring data, after a certain point in history, is no longer a
reliable indicator of local temperature changes. There is no explanation for
why trees no longer reflect local temperature changes. We know that they no
longer reflect local temperatures, presumably, because we have other more
reliable measurements available now (measurements that are not available
further back in history)._

There is NO SINGLE EXPLANATION for the discrepancy. This is NOT the same as
there being no explanation.

What happens instead is that if you look in specific localities you can find
specific explanations. Increased acidity here. Drought conditions there.
Insect infestations in another. And in yet another the correlation with
temperatures remains.

What happened is that after 1960 due to large scale changes in environment in
many localities the correlation broke down, so the overall correlation broke
down. But there is no single explanation because in different areas the
specific reasons differ.

 _This leaves us with a conundrum: is there any period in which alternative
measurements and tree-ring measurements correspond with one another? Is the
divergence between our measurements and the tree-rings something we can trace?
Is it sudden or gradual? If there is no period in which there is a
correspondence between our measurements and the tree-rings, what does this say
about our assumption that tree-rings have ever accurately captured local
temperatures?_

Actually there is about a century of comparative data before that which shows
a good correlation. That is why we are willing to project temperatures back
farther using that record.

Now there is the possibility that there were previous periods which saw a
similar breaking of the correlation. This question is a question of scientific
debate.

 _It seems that the implication is that changes in the environment have
rendered tree-rings unreliable. But if we're arguing that tree-rings are
unreliable because of environmental change (i.e. the chemical changes to the
composition of the atmosphere and water cycle that contribute to global
warming), and we're seeking to explain environmental change, we're begging the
question (assuming the explanandum in the premise of our argument)._

There is no begging the question when we are also studying the mechanisms
involved.

 _Until a causal mechanism for the divergence between tree-rings and other
measured temperatures can be specified, how can they legitimately be discarded
as evidence, without invoking tautological reasoning?_

You are making two large assumptions here. The first is that there is a single
causal mechanism. The second is the assumption that we don't actually
understand the mechanisms involved. Both assumptions are wrong.

Look. Scientists aren't exactly stupid people. When they spend a lifetime
studying something, you should assume that they've asked the obvious questions
and try to find out the answers they have found rather than assuming that they
are stupid and you're the first to notice the glaring issues.

Unfortunately climate scientists are also humans who are under constant
assault from people who can't and won't be bothered to actually learn the
record. People who feel they are under constant assault engage in defensive
behavior. And people who are being defensive tend to circle the wagons and
have difficulty in remaining open with outside groups.

When you encounter such defensive behavior I strongly recommend trying to
understand what the defensive person's position is rather than immediately
piling on and validating their defensiveness. If they are wrong, this approach
leaves you in a better position to figure that out. But if they have a point,
you'll be more likely to figure that out as well.

~~~
DanielBMarkham
Do not confuse groupthink with analysis.

Simply because a heckuva lot of smart people have studied an issue for decades
does not mean -- at all -- that somehow they have made more progress than some
schmuck reading about all of this for the first time in Newsweek. You do not
get a free pass by being smart and asking a lot of questions.

(A side note: I find among academia the meme that somehow intelligence, hard
work, and admiration of peers equates to progress. These are all great traits
to have, but in themselves they don't automatically translate into solid value
for posterity. In fact, in some fields there may actually be an inverse
relationship)

Having said that, you are absolutely correct in saying that you should ask
questions first. There's no point in getting everybody on the defensive.

But if the reply is "Look -- it's all very complicated, and we've thought
about it a lot and we're way ahead of you here and this is the best plausible
way to settle several internal arguments... etc, etc" then you have not
received an adequate answer.

I applaud your appeal to being inquisitive and polite. I find your defense of
this particular case, however, sorely lacking.

------
ieure
I don't know what ESR's politics looked like before, but he's been in deep
teabagger territory for a while now. I would take anything he says on the
subject of politics with… Well, I'd just not take it. He can pretend to be
objective, but the use of winger slang (“AGW” and “MSM” are right-wing dog
whistles in the same way as “Democrat party” or “socialism”) is a giveaway.

Here are some of his prognostications from the election last year. They're
delightfully wingnutty, with a crunchy glibertarian shell:

“The Obama campaign smells of defeat”: <http://esr.ibiblio.org/?p=449>
“Democrats’ polling league in a generic contest has collapsed. Control of
Congress is in play. Wow. Just wow. Considering how hard the Republicans have
been fucking up and alienating their base, this is astonishing. EPIC FAIL.”

“Sarah Palin, American Centrist”: <http://esr.ibiblio.org/?p=481> “So when I
tell you that I think Sarah Palin’s religious position is pretty near dead
center in the American spectrum, you can be pretty sure I’m not fudging to
make that position look good.”

“The Obama campaign smells of defeat: II”: <http://esr.ibiblio.org/?p=490>
“The attempt to smear, discredit and delegitimize Palin has steadily become
more intense and more damaging — to the Democrats.”

When it comes to politics, dude simply does not know what the fuck.

~~~
javert
You're just attacking ESR's character. It's fallacious to discount his blog
post just because you don't like his character. Please, can't we have a
rational discussion on the topic?

~~~
trominos
He is certainly attacking ESR's character, but I think that the old mantra
that we should always address an argument and not the person delivering it
isn't entirely fair. The strongest reason for this, in my opinion, is that
most complex systems can't be thoroughly analyzed with any objectivity,
because it's impossible to consciously process all the available data; there's
just too much going on. So whenever somebody argues about a complex system,
there's a very implicit part of their argument that involves _which data they
choose to present_. A bystander who's trying to follow (and validate or
invalidate) a logical argument can't always tell if the choice of data is
unbiased, and of course looking at _all_ available data takes the bystander as
much time as it took the arguer to formulate the argument in the first place.

So USUALLY what an unbiased observer will do is proceed with caution: see what
the arguer's point is, see how it fits with the observer's intuition (because
this is the same as appealing to the observer's subconscious, which in most
cases has processed far more data relevant to some issue than has their
conscious mind), and see whether the data seems overwhelming enough that its
conclusion _couldn't_ be the result of biased data selection on the part of
the arguer. But this method often fails. Without formulating your own opinion
by immersing yourself in data, you just can't be sure whether you're looking
at a legitimate picture of an issue. That's why whenever I hear a convincing
argument about something complex, I only provisionally believe whatever I've
just been convinced of. And that's why an argument's author actually matters
if the argument relates to a sufficiently complex system.

Now, arguments about simple (by which I really mean "completely specified")
systems aren't victim to this weakness -- mathematical proof, for instance,
can actually be really verified by an observer without the observer doing the
same work that the discover did, as can many logical arguments in the hard
sciences.

But what ESR is trying to show here is that some particular set of scientists
committed scientific fraud. This is a) a value judgment and b) depends
substantially on context.

So I think that it is not only reasonable but very much a _good_ thing that
ieure pointed out that ESR is a nutjob. Now I know that even if I find his
post compelling, he has no intellectual honesty and the data he's showing me
have been carefully selected from the unimaginably vast amounts of data in the
world because they happen to support his worldview. Maybe he's still right --
but ieure's post is still very much relevant.

~~~
jimbokun
"So I think that it is not only reasonable but very much a good thing that
ieure pointed out that ESR is a nutjob."

That was a very sophisticated formulation of "I don't listen to people whose
politics I disagree with."

That is exactly the problem of the entire Republican party, and you seem to be
saying that the principle is sound (just as long as it is directed at "them"
and not "you").

It saddens and surprises me that you and your up-voters seem to be endorsing
ad hominem as sound debating practice. Of all places, I would hate to see
Hacker News descend to this level.

~~~
trominos
> That was a very sophisticated formulation of "I don't listen to people whose
> politics I disagree with."

It wasn't. It was an _argument for believing_ that "in many cases it matters
where the arguments that I hear come from."

Let me be clear: I don't _like_ the fact that I think this policy is right. I
just think it's right.

And let me also be clear that in many or most cases it makes sense to
disregard the source of an argument. In most exchanges on HN, the people
involved all implicitly agree on the data that's being argued from, and in
those cases I tend to think that we don't ever need to think about people's
biases.

But the fundamental belief underlying the "we can always disregard an
argument's source" movement is that sufficiently logical and dispassionate
people can validate or invalidate an argument very easily (compared to the
work it took to formulate the argument in the first place). I really really
would like this to be true. I just don't think it is, IN GENERAL.

To reiterate why: I think it's perfectly possible to follow a logical train of
thought without actually re-deriving that train of thought -- and so I think
that we can untangle argument and arguer when arguments are purely logical --
but if an argument is based on data from a complex system I _don't_ think it's
possible to determine if that data was chosen in an unbiased manner without
actually looking at all the available data and essentially re-deriving the
arguer's conclusion.

Combine that with two hopefully-obvious (and hopefully-true) premises: first,
that a large data set on any sufficiently complex system contains individual
data points that support any given belief about that system,

and second, that most really complex systems can't be considered completely
logically, because there's so much going on, and must instead be approached by
looking at data, and I think it follows that _you need to think about the
source of any argument about a complex system when you're deciding whether you
(provisionally) believe it_.

Incidentally, I find it genuinely ironic that your post is actually a not-
particularly-sophisticated example of "I don't listen to people whose politics
I disagree with," since you ignored my argument based on my dissenting
conclusion.

~~~
jimbokun
"...but if an argument is based on data from a complex system I don't think
it's possible to determine if that data was chosen in an unbiased manner
without actually looking at all the available data and essentially re-deriving
the arguer's conclusion."

But this is _exactly_ what I come to Hacker News for. In this case, there are
definitely HN readers who can understand the code ESR is talking about, and
pretty quickly figure out if there are flaws in ESR's reasoning regarding the
code in question.

Once we have the actual flaws in his reasoning, his tea-baggery or whatever
become completely irrelevant, because we have much more solid reasons for
rejecting his conclusions. If this were a different forum, with no
participants well versed in technology, mathematics, and the scientific
method, maybe you need to consider the motivation of the arguer when
considering the validity of the argument. HN is not that forum.

------
skmurphy
opening paragraph comments on a code fragment:

From the CRU code file osborn-tree6/briffa_sep98_d.pro , used to prepare a
graph purported to be of Northern Hemisphere temperatures and reconstructions.

    
    
      ;
      ; Apply a VERY ARTIFICAL correction for decline!!
      ;
      yrloc=[1400,findgen(19)*5.+1904]
      valadj= [0.,0.,0.,0.,0.,-0.1,-0.25,-0.3,0.,-0.1,0.3,0.8,1.2,1.7,$
          2.5,2.6,2.6,2.6,2.6,2.6]*0.75 ; fudge factor
      if n_elements(yrloc) ne n_elements(valadj) then message,’Oooops!’
      ;
      yearlyadj=interpol(valadj,yrloc,timey)
    

This, people, is blatant data-cooking, with no pretense otherwise. It flattens
a period of warm temperatures in the 1930s--see those negative coefficients?
Then, later on, it applies a positive multiplier so you get a nice dramatic
hockey stick at the end of the century.

~~~
indy
jgrahamc mentioned in an earlier comment (and also on his blog:
<http://www.jgc.org/blog/2009/11/about-that-cru-hack.html>) that these
variables aren't used.

~~~
waterlesscloud
Great, so now all we need is to run this in the released data and verify if
the graph was produced with the code commented in or out.

If there was released data, that is.

The point being we only know how the code looks in that file, we don't know
when the commented out sections were commented out.

So nothing is proven here either way.

------
hop
How do all the "best" climate prediction models have squiggly lines after 10
years from now and not a broad range or something realistic. Their algorithims
smell of BS and way too many variables go into it to make them have anywhere
near their plotted accuracy for 100 year projections. From wikpedia:

[http://en.wikipedia.org/wiki/File:Global_Warming_Predictions...](http://en.wikipedia.org/wiki/File:Global_Warming_Predictions.png)

~~~
azgolfer
Anyone who says they have a good computer model of the climate is either
ignorant, stupid, corrupt or some combination thereof.

------
jacoblyles
While I'm glad people are digging through the leaked source code, I doubt they
find anything so blatant as outright fraud.

------
yters
So maybe we haven't found a smoking gun yet, but the general gist of the
documents and emails in that package is not very encouraging.

You can find reviews of IPCC's book that are full of very strident critiques.
You can find emails discussing the inherent inaccuracy of their work. You can
find internal dismissal of the tree ring data, stating how useless it is. You
can find a scientist openly glad that he's not being held accountable so he
can push his climate interests.

After spending a couple hours pouring through the repository, I haven't found
much that makes this work seem stable enough to base global lifechanging
legislation on.

------
alex_c
The code is pretty damning, but before I get really outraged, a bit of context
is always useful.

Has anyone figured out where this graph has been used?

~~~
jgrahamc
No. And, also, the interpolated value yearlyadj isn't used. It's only
referenced in commented out code.

<http://www.jgc.org/blog/2009/11/about-that-cru-hack.html>

------
Joeboy
Can anybody tell me what, if any, published work this data has made it into?

------
j_baker
I'm not necessarily attempting to justify _any_ data-cooking whatsoever, but
why is it such a big deal? I mean, the climate "skeptics" do it all the time
and it hardly makes the news.

We're never going to have a decent public discourse on this issue until _both_
sides are honest with the public. Maybe I'm just a pessimist, but I don't see
this happening any time soon.

~~~
DomesticMouse
Science that relies on cooked data isn't science. It's snake oil sales.

