
ASAP: Automatic Smoothing for Attention Prioritization in Time Series - SEJeff
http://futuredata.stanford.edu/asap/
======
jnordwick
The examples they give are periodic so removing the periodicity is the big
win, not removing the noise.

How did ASAP perform better than ARIMA and other similar algos should be the
main question, not how does ASAP compare to nothing.

~~~
aashu_dwivedi
Isn't Arima used for forecasting? Pardon me I'm not very informed about it but
how is Arima used for smoothing?

~~~
jnordwick
Various ARIMA methods are used to deseason data for forecasting, but yes a
better more directly comparable example might have been methods like STAMP and
STL.

------
SEJeff
Blog post with additional info on why this is useful:

[http://dawn.cs.stanford.edu/2017/08/07/asap/](http://dawn.cs.stanford.edu/2017/08/07/asap/)

------
polskibus
In short - is this a moving average with window range automatically tuned for
optimum detail to smoothness ratio?

~~~
hcrisp
A moving window average is a poor digital filter in the frequency domain [0].
You can get better results with more control on which frequencies are removed
by implementing a FIR or IIR (especially Butterworth) digital filter.

[0]
[https://ptolemy.eecs.berkeley.edu/eecs20/week12/freqResponse...](https://ptolemy.eecs.berkeley.edu/eecs20/week12/freqResponseRA.html)

------
aashu_dwivedi
Is there a python implementation of it as well?

~~~
SEJeff
[https://github.com/stanford-
futuredata/ASAP/blob/master/ASAP...](https://github.com/stanford-
futuredata/ASAP/blob/master/ASAP.py)

~~~
aashu_dwivedi
Just tried it. I was using the seasonal_decompose from statsmodel in python
for a project. I have to say ASAP looks great!

~~~
SEJeff
It really is a gem for analysis of time series data. I was lucky enough to
watch Kexin's talk live @ Monitorama on ASAP. Really great stuff.

~~~
jnordwick
Does he explain how this is better than any other methods?

~~~
SEJeff
I don't believe she did. She investigated the specific problem of helping
users to identify deviations in time series by applying smoothing.

Here is a link to her presentation:
[https://vimeo.com/221062931](https://vimeo.com/221062931)

It is pretty fascinating, I do recommend giving it a watch.

~~~
srean
I upvoted the authors comment linking the Arxiv paper, that said, I think the
presentation oversold the claim that smoothing is always a good thing to do.
In particular, showing that excel, grafana, graphite etc, do not smooth by
default, (apparently because it is a hard) looked like a low point of the
talk, sorry. Smoothing has its uses but its hardly the panacea.

What is an appropriate smoothing is very context sensitive and I would like
the consumer of the plots be in charge of that. Sometimes smoothing away the
high frequencies or transients are exactly the wrong thing to do (for example,
when you care about transient spikes)

I have seen more than enough damage done by plotting tools sneaking in a
smoothing without informing the consumer. Especially so in the monitoring
space. When the user expects raw plots it better be raw, and raw is a good
default. Smoothening should be invoked explicitly.

~~~
krong
Thanks for the comment. I strongly agree that smoothing is not always
beneficial and that the users should be informed and have control of the
amount of the smoothing that is being applied to their plots.

We developed ASAP with the intend to complement, instead of replacing the raw
plots. We view ASAP more as a "smart default" \- if you were to expose a slide
bar of different moving average window sizes that users could apply to their
visualizations, where should a reasonable default be on that slide bar?

In addition, ASAP does not always choose to smooth the visualizations. In some
cases (e.g. Figure C.1 in the arXiv paper) where the raw plots already have a
few well-defined outlying regions and smoothing would only decrease the extent
of such deviations, ASAP leaves the visualization unsmoothed.

~~~
srean
That sounds very sensible.

