
Want Better Forecasting? Silence the Noise - yarapavan
https://knowledge.wharton.upenn.edu/article/want-better-forecasting-silence-the-noise/
======
milesvp
Skimming the transcript, I can’t help but feel like this field needs to study
some signal processing ideas. They use a word like noise, but never use a word
like filter, or attentuation. Even their use of the acronym BIN is
unfortunate, since binning is a very important concept when dealing with the
frequency domain.

~~~
wenc
Can you say more about what type of attenuation would be possible?

The noise models in physical systems are fairly simple/characterizable (y =
f(x) + e, where e ~ P and P is some stable distribution, or e = a z-transform
model), whereas in social sciences, the "noise" component is actually a catch-
all/residual for whatever is unknown (e is unknown or unstable). It seems to
me that it would difficult to apply any kind of signal processing techniques
but I could be wrong.

~~~
milesvp
And this might be the very problem. Noise is a very well defined term in
information theory. If political forecasting is throwing around the term
without applying it correctly they’re gonna have a bad time.

At the end of the day though, any time you have time series data you can apply
filters to smooth and shape your data. I don’t understanding how they’re
modeling their data. There’s a good chance they’re doing some kind of
frequency modeling where they’re counting correct predictions. It definitely
sounds like they’re doing some stochastic modeling when they start talking
about percentage predictions. You can definitely shape frequency domain as
well with filters, though I havn’t quite thought through how the stochastic
aspects might interact.

Keep in mind, filters are very basic, and even something as common as
averaging data is a low-pass filter. As is fitting to a curve. This all acts
to attenuate the signal we care about without also attenuating the noise.
Though, again, if someone isn’t being rigorous about what constitutes noise,
then no amount of filtering will actually help...

I’m also sorry if this thread isn’t very insightful. I’ve been having my nose
rubbed in signal processing at work for the last 2 months, and it’s all I can
see everywhere I look. I see parrallels everywhere that may not be there.

You’re also very correct about the simplicity of physical models versus social
sciences. It may just be that trying too hard to apply basic information
theory at models that are almost impossible to create in the first place is a
fools errand.

~~~
topologistics
> Noise is a very well defined term in information theory. If political
> forecasting is throwing around the term without applying it correctly
> they’re gonna have a bad time.

Class is a very well defined concept in software development. Are political
science people going to have a bad time because they "throw around" the word
class and use it in an economic sense?

There are only so many possible finite words that can be created from
vocalizable letter combinations, there is bound to be overlap across
disciplines.

~~~
j88439h84
I'm not sure they're using "noise" any differently

------
chroem-
Protip: an EWMA is a smoothing convolution whose deconvolution can be computed
in closed form with a little algebra. To reduce the effects of noise, apply
the EWMA, then use a forecasting method of your choice to predict the smoothed
series, and finally apply the deconvolution to recover the original series.
This technique can be useful for series with strong seasonality, where some of
what may appear to be noise is actually useful signal, but signal which
arrived slightly ahead or behind schedule within the season.

------
cwyers
> How would you feel if an algorithm decided whether or not you should be
> charged with a crime? Whether an algorithm decided whether or not you had
> cancer? I think for many of these cases, it’s well known that the algorithms
> do much better than people. I was just reading Malcolm Gladwell’s new book,
> Talking to Strangers. He tells the story of a judge in Chicago who decided
> whether to keep detainees or release them on bail. He liked to look into the
> eyes of the detainee to decide whether he would skip bail. It turned out
> that information wasn’t nearly as valuable as other information that you can
> derive from machine learning, algorithms and so forth. Accuracy increased
> greatly with the algorithm.

I feel like this glosses over a lot of evidence that shows that using
algorithms to determine guilt and innocence in the criminal justice system is
incredibly fraught.

[https://www.wired.com/2017/04/courts-using-ai-sentence-
crimi...](https://www.wired.com/2017/04/courts-using-ai-sentence-criminals-
must-stop-now/)

[https://www.propublica.org/article/how-we-analyzed-the-
compa...](https://www.propublica.org/article/how-we-analyzed-the-compas-
recidivism-algorithm)

[https://www.washingtonpost.com/business/2019/11/19/algorithm...](https://www.washingtonpost.com/business/2019/11/19/algorithms-
were-supposed-make-virginia-judges-more-fair-what-actually-happened-was-far-
more-complicated/)

------
AznHisoka
Read the paper and still confused as to what the author means by “noise”?

And don’t you know whether something is noise or not after the fact? You may
think some signal is useful when you first encounter it and you may not know
it’s noise until after it produces a false prediction. So silencing it isn’t
actually possible.

~~~
AznHisoka
As an example, before 9/11, there was some chatter about a possible terrorist
attack in the USA. After 9/11 we know now this wasn’t noise. But if it hadn’t
happen, we would classify it as noise. Telling the CIA to silence the noise is
useless advice because we don’t know beforehand what is noise!

------
pagutierrezn
This "silence the noice" concept sounds like the Deming "common causes of
variance". And "better forecasting" like "reduce variability to improve
predictability"

------
deepnotderp
Soo... signal processing?

