
How thermometer and satellite data is adjusted and why it must be done - Tomte
http://arstechnica.com/science/2016/01/thorough-not-thoroughly-fabricated-the-truth-about-global-temperature-data
======
chrissnell
It wasn't really mentioned but there is a program where weather enthusiasts
can submit readings from their automated weather stations to a service, which
provides them (in bulk) to various data consumers. Some of the consumers
produce consumer weather sites (Weather Underground, for example) and others
are scientific users who aggregate the data and do statistical analysis. In
most towns, there's at least one enthusiast submitting data, so when you
average all of these thousands of collectors, it can cancel out the effects of
the methodology changes described in the article above.

On a side note, I have a weather station in my back yard and I wrote my own
software in Go to collect and graph data from it. The station is a Davis
Instruments Vantage Pro 2, which is a decent "pro-sumer" level station like
you might find on the roof of a fire department or ski patrol building.

My software is called gopherwx. It doesn't have CWOP support yet or any
documentation but you can check it out here:

[https://github.com/chrissnell/gopherwx](https://github.com/chrissnell/gopherwx)

I have the data feeding into InfluxDB and graphing via Rickshaw on my site,
which updates the readings in real-time every three seconds:

[https://mhkweather.com/](https://mhkweather.com/)

~~~
abraae
Off topic but it sounds like you know your weather stations, are there any
cheaper than the Davis you would recommend?

~~~
chrissnell
My dad has the Accurite that's sold at Costco and Amazon. He loves it. It's
decently accurate--enough for a home user--and not expensive at all.

He was supposedly going to send me one and I was going to extend gopherwx to
support it but he hasn't yet. I can't imagine that the protocol will be
difficult. Davis's protocol was designed in the early 90s for serial lines and
is binary to optimize for space and speed on early PCs. It was kind of a pain
in the ass to implement a driver for it.

~~~
abraae
Awesome, thank you.

------
m3rc
The real fight isn't proving that the data is there, it's proving that people
should even look at the data. Climate Change Denial doesn't come from a
scientific place and it doesn't have a scientific answer really.

~~~
cool_look
and what about skeptics who think the current explanations are deeply flawed
and being propped up for continued funding ?

this happens outside climate science, why should it be different because
Climate ?

See the physics lady critical of the Higgs boson project:
[http://backreaction.blogspot.co.uk/2016/08/the-lhc-
nightmare...](http://backreaction.blogspot.co.uk/2016/08/the-lhc-nightmare-
scenario-has-come-true.html?m=1)

"I hope that this latest null result will send a clear message that you can’t
trust the judgement of scientists whose future funding depends on their
continued optimism."

in this case translate optimisim for Climate Pessimism

~~~
JoeAltmaier
Factor in the cost of being wrong. That's the chilling part. "We're all going
to die!" "Ha ha, I doubt it, lets just continue on down the same path because
maybe we won't die."

------
yread
Wow this graph (from the comments) is really scary

[http://www.realclimate.org/images//Marcott.png](http://www.realclimate.org/images//Marcott.png)

~~~
pedrocr
An even scarier way of looking at the same reality:

[https://xkcd.com/1732/](https://xkcd.com/1732/)

------
nanis
See also GHCN v3 Great Dying of Thermometers
[https://www.youtube.com/watch?v=h95uvT67bNg](https://www.youtube.com/watch?v=h95uvT67bNg)

Historically, humans have not measured and recorded temperature according to
some over-arching research plan with well designed criteria.

------
exabrial
I think it's fine to make adjustments, but the destroying, or making the raw
data harder to obtain is unscientific and unethical.

I know we've moved on from Climate Gate but I didn't realize that actually
happened IRL, and I thought that was one of the more important revelations.

The real problem with getting skeptics to believe is that agw is tarnished
science. You have people saying things like "it didn't snow in Kansas in
October, clearly global warming is real". The silence from the scientific
community when people make statements like this tarnishes there validity of
what they've actually proven.

~~~
inimino
And people say it snowed in the summer and therefore global warming is false.
Is it the scientific community's responsibility to step up every time John Q.
Public makes an entirely uninformed and irrelevant statement?

~~~
exabrial
Yes of course. Do you want an educated public or are you just trying to get
them to believe in a doctrine?

~~~
inimino
If climate scientists shot down every mistaken claim people make about their
field, they wouldn't have any time left over for climate science.

The science is no more "tarnished" by people's misunderstanding of it than
quantum mechanics is by people using it as a justification for pseudoscience.

~~~
exabrial
I see. The end justifies the means? It's ok for people to lack true
understanding because it's helpful for your, er, greater, cause?

~~~
inimino
> end justifies the means

What? Are you replying to what I wrote or just spewing random nonsense? Is
there such a thing as an astroturfing bot? You aren't really demonstrating
human-level reading comprehension here.

Is this always what it feels like to engage climate deniers?

------
cool_look
Long time HN user using a throwaway account because of the lynch mob ( both
sides )

The article page 3 mentions a crucial get-out-of-jail card that HN is taking
completely hypocritically at face value.

Namely that NOAA adjustements are in response to Met Office/Hadley/UEA
adjustements.

this should be setting off "tampered" alarm bells

Lets step away to make parallels:

UK Copyright extended to harmonise with US.

US copyright then extended to harmonise with UK copyright

HN response: The copyright holders are colluding to extend the term globally

======

NOAA adjustments: We're harmonising because of Met Office / Hadley / UEA
adjustments

HN: Yay science, nothing fishy could be going on here because Scientists.

~~~
EdwardDiego
You're comparing an abstract set of laws used to control ownership of
intellectual property, which are often abused, to scientists trying to collect
the most accurate data they can?

You don't see a difference?

~~~
cool_look
difference yes.

you dont see the similarities?

------
jrm415
> So in the US, accounting for non-climatic factors ends up increasing the
> warming trend over the raw data—which we know is wrong.

I wish the author had been a bit more careful when writing this sentence. I
think the intended meaning is "we know the raw data is wrong," but I read it
at first as "we know [the increase] is wrong."

------
pjc50
You know how we were discussing the "fake news" problem? In the comments here
we have the reason why it's so popular: people are not interested in facts,
especially if they're inconvenient for their lifestyle. And now HN is
uprooting climate change denial.

------
kapitza
This is actually an excellent illustration of how organizational incentives
seep into data.

Imagine you have a historical temperature series record. You have an
bureaucratic incentive to find and justify an argument as to why past
temperatures should be adjusted downward, making the curve of global warming
look steeper. You have no bureaucratic incentive to look for the opposite.

The result is that, overall, modern corrections adjust past temperatures
downward -- what we'd expect if these biases exist. But this doesn't mean any
specific correction is based on "fabricated" research. It's probably perfectly
valid. It's just that downward corrections are more attractive to research,
because they produce research that has a social impact that the field finds
desirable.

(Edit: here's a picture of the net adjustments:
[https://judithcurry.com/2014/07/07/understanding-
adjustments...](https://judithcurry.com/2014/07/07/understanding-adjustments-
to-temperature-data/))

~~~
conistonwater
This is a pretty weak argument, because (a) arguments from scientific validity
always trump arguments from incentives, (b) you seem to assume that the
incentives you list are all the incentives there are. Do note, for example,
that scientists also face the incentive to _do good science_ that can stand up
to the scrutiny of their peers. It's not enough to point out that the science
has some impact and call it a day (after all, almost all good science has
impact).

For this to be an "illustration", as you put it, you'd have to _first_
demonstrate that what they did was scientifically dubious, _then_ conclude
that they must have done the dubious thing because of incentives. You can't
just point at the sign of the adjustment and invoke incentives. I'd say you'd
be much more convincing if you raised actual scientific issues, instead of
making vague assertions about their incentives.

~~~
kapitza
I think you're missing the point.

The negative adjustments of past temperatures are neither the work of one
scientist, nor even one scientific organization. Let's say for simplicity that
they're the result of ten entirely separate confirmed hypotheses about
systematic errors in old data.

Each of these was produced by an honest researcher who followed the scientific
method: formulate a hypothesis, test it, etc. Let's say their technical work
is perfect. All the hypotheses are correct.

But all ten researchers, faced with the choice of what hypothesis to formulate
and/or investigate, naturally chose high-impact hypotheses -- ones that, if
confirmed, adjust past temperatures downward.

We would naturally expect historical errors in temperature measurements to be
evenly distributed. Like errors in cheap thermometers. If they are evenly
distributed, we would expect there to be ten other such hypotheses, involving
_upward_ corrections in past measurements, which were not studied because of
their low impact if confirmed.

This hypothesis explains the data that we ourselves see: that past
temperatures are continuously being lowered as the data evolves. And it
requires no one to be a bad, incompetent, or dishonest researcher.

~~~
xenadu02
Your posts seem like classic climate change denialism. The same tactics were
used to argue that lead in gasoline wasn't a problem and that cigarette smoke
doesn't cause cancer.

CO2 causes atmospheric warming. The underlying principles are simple and
sound. We are emitting tons and tons of CO2 by burning vast quantities of
fossil fuels.

The only reason to argue against the science is if you have an ulterior motive
or you've been unwittingly duped by those that do.

>But all ten researchers, faced with the choice of what hypothesis to
formulate and/or investigate, naturally chose high-impact hypotheses

That isn't how science works. It doesn't matter what they hypothesize, what
matters is what the evidence shows.

>We would naturally expect historical errors in temperature measurements to be
evenly distributed.

No we wouldn't and you refuse to offer any evidence to back up your claims.

>If they are evenly distributed, we would expect there to be ten other such
hypotheses, involving upward corrections in past measurements, which were not
studied because of their low impact if confirmed.

I'm going to be direct and honest here: This is why I think you're just lying
to push a point. This is not how the scientific method works. None of what
you've postulated is correct. And for the record there are various groups
(including some oil companies) pouring millions into scientific studies trying
to prove global warming isn't happening. None of their supposed "evidence" has
withstood peer review or scrutiny.

Evidence is what ultimately matters, a point you seem to be trying really hard
to avoid addressing.

~~~
hueving
Your post is also lacking many concrete details so it's not adding to the
conversation.

>Your posts seem like classic climate change denialism.

Any skeptical post would qualify for this.

>The same tactics were used to argue that lead in gasoline wasn't a problem
and that cigarette smoke doesn't cause cancer.

The same tactics (attacking data sources) are used in any critical analysis of
a given scientific field (see sociology).

>CO2 causes atmospheric warming.

Repeating the conclusion is not an argument.

>The underlying principles are simple and sound.

Right, but the underlying principle isn't being disputed. The underlying
principles behind lots of research can be fine while the research itself is
garbage (e.g. bad data collection methodology).

>The only reason to argue against the science is if you have an ulterior
motive or you've been unwittingly duped by those that do.

This is so idiotic I don't even know where to start. You're saying that there
is no reason to question scientific findings if you think a whole method of
research is fundamentally flawed?

"The only reason you have to argue against numerology is if you have an
ulterior motive or you've been unwittingly duped by those that do."

Sounds silly, doesn't it?

>That isn't how science works. It doesn't matter what they hypothesize, what
matters is what the evidence shows.

Cute, I take it you've never worked in academia? People completely throw away
experiments that fail to support their hypothesis all of the time. Especially
if the result would bring down a fire of threats to their career.
([https://xkcd.com/1478/](https://xkcd.com/1478/)
[https://xkcd.com/882/](https://xkcd.com/882/))

Bias in the researchers is a very big problem in all scientific fields. When
you're trained to be a hammer and all you work with are hammers, and you get
your funding from the hammer use-case fund, you certainly aren't out looking
for screwdriver use-cases.

Please keep the discussion to substantive counterpoints other than ad
hominems, strawman arguments, and appeals to authority.

~~~
magicalist
> _Your post is also lacking many concrete details so it 's not adding to the
> conversation._

Your post is nonsense in context. The top post provided no concrete details,
so what details could the GP use but to quote it directly?

It was a non-falsifiable theory devoid of evidence; by their logic, we can't
listen to experts _because_ they're experts. All research is subject to
systemic and _directed_ bias, discernible without actually looking at that
research.

It's dumb and not remotely concrete. And it's certainly a time honored
diversionary and denialist tactic.

> _Bias in the researchers is a very big problem in all scientific fields.
> When you 're trained to be a hammer and all you work with are hammers, and
> you get your funding from the hammer use-case fund, you certainly aren't out
> looking for screwdriver use-cases. Please keep the discussion to substantive
> counterpoints other than ad hominems._

I'd suggest maybe doing a little self reflection before posting? This entire
thread is fundamentally an ad hominem argument, and this quote just doubles
down.

~~~
madaxe_again
Hacker news agrees with him, not you. Take your elitist ration and logic
elsewhere, it's not welcome here. I've just about given up on this place - I
think Thiel has been astroturfing for years and it's getting worse.

~~~
hueving
>Take your elitist ration and logic elsewhere

Did you read my post? It was pointing out that that the parent's post was
lacking ration and logic and was resorting to ad homs, strawmen, and appeals
to authority.

------
asmithmd1
If only there were something that wasn't man-made that would integrate the
temperature reading continuously over 100 years and display if the temperature
is increasing or decreasing. Anything that did that would probably be subtle
and take data processing that the deniers would question.

Oh Look: [http://www.nbcnews.com/id/36224913/ns/us_news-
environment/t/...](http://www.nbcnews.com/id/36224913/ns/us_news-
environment/t/glacier-national-park-loses-two-more)

~~~
Shivetya
We also have rain fall studies that show no change in sixty odd years, we have
studies which can indicate or disprove about any point anyone can raise.
Glaciers are interesting because they don't all melt at once and some increase
while others decrease. Based on some skeletal and other findings with some
retreating glaciers its pretty obvious it has happened before, long before we
were having an effect.

all articles like the one we have today is simply explain that its not a
decided science and anyone claiming otherwise is just as bad as those who
refuse to believe any changes occur.

the question that must be answered is, are the estimates for change in line
with what has occurred and is occurring and if not, why.

~~~
erikpukinskis
> we have studies which can indicate or disprove about any point anyone can
> raise.

What are you basing that belief on? How many studies did you spot check to
reach this "studies can disprove about any point" determination?

That's an exceedingly broad claim, and I'm skeptical you've done more than
hand pick studies that support your worldview, but I would be incredibly
relieved to be proven wrong.

------
theparanoid
I read the NOAA papers, they're enlightening.

------
privilegedImp
z

~~~
jackmott
> how do statisticians prove other factors are not responsible?

satellite observations over ~30 years have tracked the amount of outgoing
radiation in various regions of the electromagnetic spectrum over time. The
amount of outgoing radiation has decreased, only in the spectra that co2 and
methane block. The amount of reduction is as expected given the amount of co2
and methane we have added to the atmosphere:

chart:
[http://static.skepticalscience.com/images/harries_radiation....](http://static.skepticalscience.com/images/harries_radiation.gif)

This is just one of many ways we can confirm the attribution of climate
change, it is IMHO the most clear, and obvious one, and the one for which I
have never seen a refutation that even sounds serious, so I like to use it.
There are many other ways to confirm this though.

~~~
monkmartinez
30 years! How old is the climate and Earth again? How many volcanoes erupt
daily, weekly, monthly... and how much do they contribute to "other factors?"

I worked at AFWA in 2004, in the Global Event office. I tracked Volcanoes,
Hurricanes, Haboobs, and the like... The scientists there loved to have these
conversations.

------
boneheadmed
So in other words, you really have no idea what you're doing do you?

------
mirekrusin
What about urbanisation? Near where I live 20 years ago there was huge green
space. Now it's all concrete, buildings and roads. Urbanised area is much more
warm. Aren't those stations getting closer (without moving) to urban areas
over time - just because urban areas tend to expand? What about new stations?
Are they being setup in remote areas only?

~~~
rconti
Mentioned in the article.

