
Corrupt, sparse, irregular and ugly: Deep learning on time series - dsalaj
https://www.notion.so/Corrupt-sparse-irregular-and-ugly-Deep-learning-on-time-series-887b823df439417bb8428a3474d939b3
======
BadInformatics
Was prepared for an introduction to some novel never-before-seen method near
the end, but this was a nice and well-structured summary regardless! Wanted to
tack on a couple of questions/thoughts:

1\. Is the author familiar with all of the recent work around neural
differential equations? Latent (Neural) ODEs[1] and Neural Controlled
Differential Equations[2] are already able to match or exceed the performance
of GRU-D when working with certain sparse and irregularly-sampled time series.

2\. More of a nitpick, but most of the models covered as working with EHR data
are using clinical data instead of raw physiological signals. For example,
measurements from a high-frequency physiological waveform such as an ECG are
usually synthesized into something like a per-minute heart rate in a medical
record. Most research working with physiological signals directly is using
either traditional signal processing approaches or some form of CNN (this
includes 1-D resnets and wavenet-like architectures). RNNs do pop up
occasionally when dealing with dramatically downsampled signals, but seem to
suffer pretty badly from catastrophic forgetting and other issues when run on
longer, higher-frequency data.

[1] [http://papers.nips.cc/paper/8773-latent-ordinary-
differentia...](http://papers.nips.cc/paper/8773-latent-ordinary-differential-
equations-for-irregularly-sampled-time-series) [2]
[https://arxiv.org/abs/2005.08926](https://arxiv.org/abs/2005.08926)

~~~
blt
Yeah, irregularly spaced time series data seemed to be the killer app of
Neural ODEs according to my understanding.

------
agakshat
Didn’t realise Notion was being used to write blog posts too. It does make
quite a bit of sense to use the same interface to take notes and then
occasionally combine and refine those into a blog post.

~~~
MauranKilom
That use case has been the topic of a recent, well-received Show HN thread:
[https://news.ycombinator.com/item?id=23514682](https://news.ycombinator.com/item?id=23514682)

~~~
agakshat
Thanks for sharing the link, that looks fantastic!

