
Interpreting Noise - siddg
http://robjhyndman.com/hyndsight/interpreting-noise/
======
hawkharris
On a related note, we could take another simple step toward improving the way
people communicate about statistics: ban point estimates in news reports.
Using a point estimate, I can accurately claim that there's a 50% chance
heaven and hell exist. This is technically true because my confidence ranges
from 0% to 100%.

I get that people often need so-called "killer facts" — quick statistics that
make an impression in press releases — but confidence intervals are more
honest and they don't take much longer to communicate. Instead of allowing
people to say that "x% of users prefer...", let's say that "x-y% of users
prefer."

~~~
yummyfajitas
I get the point you are making, but confidence intervals are _completely the
wrong solution_. They do NOT mean what you probably think they mean (where
"you" is a generalized "you the reader").

[http://www.ejwagenmakers.com/inpress/HoekstraEtAlPBR.pdf](http://www.ejwagenmakers.com/inpress/HoekstraEtAlPBR.pdf)

Credible intervals are probably what you want. A lot of the time people
misunderstand confidence intervals and treat them like credible intervals.

[http://en.wikipedia.org/wiki/Credible_interval](http://en.wikipedia.org/wiki/Credible_interval)

------
monochr
"What is going on here is that the com­men­ta­tors are assum­ing we live in a
noise-​​free world. They imag­ine that every­thing is explic­a­ble, you just
have to find the expla­na­tion."

They know full well that there isn't anything behind it. The problem isn't the
news organization, it's the media consumers. People want stories to understand
the world in easy bite sized pieces. Unfortunately what affects us doesn't
come with a simple story line, it's just one damned thing after another. But
no one would sell any news with "Stuff happens, we have no idea why. More
stuff to happen later."

~~~
batoure
>The problem isn't the news organization, it's the media consumers.

I've always been somewhat disheartened by comments like this. People will
blame consumers for how media is but then say this about technology:

"If I had asked people what they wanted, they would have said faster horses."

At some point someone sold the idea of sound bite consumable news to customers
and convinced them that it was good. the way that we know that is because news
had never looked that way until cnn and networks like it started.

consumers either know what they want or they don't know what they want... but
I hardly think that issue is cut and dry

~~~
ZoF
>At some point someone sold the idea of sound bite consumable news to
customers and convinced them that it was good.

What makes you say this? This seems pretty false in my opinion.

It seems much more likely that sound bite consumables are the outcome of
testing/feedback and iteration. It seems entirely possible to me(if not
probable) that consumers(for the most part) don't know what they truly 'want'
in terms of news.

We live in a world of(largely) instant gratification; the fact that 'news'
would have to adapt to fit that lifestyle makes perfect sense to me.

> People will blame consumers for how media is but then say this about
> technology: "If I had asked people what they wanted, they would have said
> faster horses."

What point are you trying to make here? I'm having difficulty parsing your
meaning. Humans, when they aren't versed in the underlying operations of a
system, WILL make broad assumptions(about said operations) and WILL have
irrational expectations and 'wants' based on those assumptions. What would my
grandmother(or even mother for that matter) ask me to improve about her
spyware-ridden current-gen laptop? "Make it faster." If I asked her how to
improve the news on the other hand I guarantee "make it faster" would not
enter the conversation.

Soooo, what's your point here? That consumers do want 'faster' news? And why
would a consumer's desire for more succinct news imply that they are okay with
less accurate news...?

Obviously the issue isn't cut and dry, everyone is a consumer of the news and
we all have our own opinions. That said, I have a hard time disagreeing with
the parent comment. Have you watched the news recently? 'News' stories are NOT
given time based on their importance to the world as a whole; and for good
reason. When a media company(whose purpose is to provide fiscal return) is met
with positive feedback for stories pertaining to popular culture, or pandering
to popular opinion, they will invariably continue to publish in that vertical.
The result is a 'news' which consists mainly of stories which are either
racially/politically charged or major events(read: actual news) which are
strung along for weeks(natural disasters, terrorists actions, etc...)

~~~
batoure
I guess I just don't like the way people talk about consumers as single
decidable entity. It ends up being a wiping boy to distract from core
problems. Example:

"Our technology succeeded that's because consumers are idiots, but we sure
showed them a better way!"

or

"Our technology produced an offensive cultural phenomenon, that's because
consumers are vultures who force us to bend to their will"

One of these statements can true but not the other... but frankly both of them
are false as markets are infinitely more complex than boiling down to a mass
generalization about consumption...

~~~
ZoF
Agreed(that grouping consumers into a single entity is bad), sorry for the
(aggressive? condescending?) tone of my comment.

------
ISL
Errorbars are largely absent from public dialog. Can you imagine a world in
which the weather report said:

"Tomorrow's high will be 30±3 degrees, and the low will be 25±2 degrees. We
expect light precipitation, but our confidence is low."

That world is coming, I hope.

~~~
thaumasiotes
That world seems pretty pointless to me. What should the public do when
tomorrow's high is forecast at 30±3 degrees but it's actually 35 degrees? Why
was the hedge an improvement?

~~~
dredmorbius
30±3 is different than, say, 30±6 or 30±1.

If the temperature range is, say, close to freezing, or some maximum safe
temperature for an activity or crop, then the error bound tells you how
certain the forecast is, and whether or not an adverse condition might be
experienced, for which you might want to take precautions (cover plants to
protect them from freezing, ensure that they're watered for heat, say).

~~~
thaumasiotes
That's not true at all. Look at the example I asked about -- "what should we
do when the forecast says the high temperature will be 27-33, but the high
temperature is, contrary to forecast, 35?"

A minor amount of imagination can transform this question into "what should we
do when the forecast says the low temperature overnight will be 2-6 degrees,
but it's actually -1? If you think the forecast will tell you "whether or not
an adverse condition _might_ be experienced", you're likely to do something
stupid. A more sensible approach would be "if it's frost season, cover the
plants".

Weather forecasts already treat the odds differently depending on (a) what
they are, and (b) what events are being predicted. A 70% chance of rain
tomorrow might be reported as a 70% chance of rain, but a 10% chance of rain
is more likely to be "overreported", say as a 40% chance. They do that because
people don't want accurate numbers -- instead, they get mad when the forecast
suggests that they don't need to prepare for unfavorable weather, and
unfavorable weather inconsiderately happens anyway.

~~~
dredmorbius
_A more sensible approach would be "if it's frost season, cover the plants"._

If you're a homeowner and you've got a small garden or some potted plants,
covering them in straw or bringing them indoors isn't a major concern.

If you're a farmer with a crop that may need to be harvested, smudged, or
sprayed with water (ice coating protects some crops), with a nontrivial
investment of time and materials, the distinction is meaningful.

~~~
thaumasiotes
If you're a homeowner with a small garden, losing the plants to frost isn't a
major concern.

If you're a farmer with a crop that may need to be harvested, smudged, or
sprayed with water, losing everything to frost is the kind of thing that might
drive you to bankruptcy. Say there are 30 days over the course of winter where
the overnight low temperature is forecast near but above zero. By your
assumption, the forecast is using 95% confidence intervals. What are the odds
of your crops dying if you rely on the weather forecast?

------
vijucat
I have a principle to explain this, and much more : The universe sides with
stupid.

There, there, take a look at Wild Poppies, near Argenteuil :
[http://i.imgur.com/1bSp0TJ.jpg](http://i.imgur.com/1bSp0TJ.jpg)

------
batoure
I think that there are two things at work here.

1\. Who is the designed consumer of this data.

In the US the CEW (census of employment and wage) is a data set that is
primarily aimed at economists or modelers. The startling thing is how these
data-sets became mainstream. I work in economics and for years no one was
particularly conscious of when these figures are released(non quantitative
people i mean) but now I am often surprised how many regular people know that
the first Friday of the month is unemployment data day. The danger here is
that modelers planning to work with the data are well aware of the signal to
noise ratio and take it into account with everything they do. Unemployment
data in the US at a granular level is also heavily redacted for a variety of
reasons which makes this problem even worse.

2\. The second issue that I would have loved to have seen this article
elaborate on is what are our core metrics as a society. This is something that
we still haven't nailed yet.

I have friends who work at the government institutions the produce this data
and they can tell you that most of this work is still done with a very mid
20th century mindset.

On the other hand I know people who work with large business data-sets such as
supply chain data from Amazon Walmart and others. This data is far more robust
and could be used to give a much truer picture of how the economy is doing but
it is currently locked up under the veil of corporate secrecy. if gov't
statistical agencies could figure out how to work with this data we could do
significantly better.

------
mbillie1
This is particularly evident during presidential election season, when the
mainstream political news will do things like show a _real-time opinion poll_
accompanying speeches or debates. The article hits the nail on the head at the
end: this phenomenon isn't going anywhere unless people start getting a whole
lot less gullible.

------
highfreq
While I think the articles most point is reasonable. The choice of stocks as
an example is a poorly one. Because prices are reacting in large part to news.
To the degree that the news presents something material and unsuspected it
influenced peoples trading. Of course good news doesn't always lead a stock to
increase, because sometimes most investors are expecting better news.
Certainly there is a lot of randomness, but for heavily traded stocks the
impact of news is real and substantial. Unexpected good news most often leads
to an increase in stock price, and unexpected bad new most often leads to a
decrease.

~~~
SixSigma
Here's an example:
[http://m.bbc.co.uk/news/business-27058146](http://m.bbc.co.uk/news/business-27058146)

Google's share price changes the same day as an announcement. The BBC
attributes the change to the news as though they are monotonic. But who is to
say that the news didn't stop the share price falling twice was far?

Also, every trade requires two parties, so one side thought they were getting
a good deal.

I'm also amused that the share traders are called "investors" rather than
"gamblers".

------
plg
An example: a phenomenon that I think has come into play gradually over the
past 3-5 years: national news broadcasts lead stories are based on the
weather.

"3rd highest temperature seen in 5 yrs"

"4th coldest winter since 2001"

"2nd highest rainfall in March in the past 12 years"

etc etc

PS on a secondary note: shame on the news services for leading with weather so
often. I guess it's laziness. No need to spend time, resources, expertise on
investigating actual news.

------
MaggieL
Warmists take note.

------
mtimjones
Does anyone remember the unemployment rate dropping just before the 2012
Presidential election (and then rising again immediately afterwards)? These
numbers can't be trusted, and are nothing more than a tool for party control
of government.

~~~
sd8f9iu
Which statistic are you interpreting this from? The Bureau of Labor
Statistics' publications? In glancing at their unemployment rate estimates, it
looks like the overall trend throughout 2012 and continuing into 2014 is
negative, with some noise in between. Though the methods used to calculate
unemployment statistics are frequently discussed (such as their inclusion or
exclusion of workers who give up searching for a job), it seems like you are
interpreting the noise for some mischievous activity (ironically, in a way
similar to what this post discusses), for which you provide no evidence.

