
A Survey of Deep Learning for Scientific Discovery - alokrai
https://arxiv.org/abs/2003.11755
======
antipaul
In a survey on "scientific discovery", I would have expected more examples
than face and image recognition and natural language processing, which are so
stale at this point.

Healthcare? Physics? Chemistry? Biology? Sociology?

~~~
anthony_doan
The rest you listed require inference and causality.

Deep learning does not do this.

Data with less noises are what most deep learning and non statistical models
does well. Meaning that image, nlp, etc.. deep learning does well. But data
with lots noises/uncertainty/variance or even data that isn't large enough,
such as time series, currently statistical models are still king
([https://en.wikipedia.org/wiki/Makridakis_Competitions](https://en.wikipedia.org/wiki/Makridakis_Competitions)).

Even with healthcare you're answering a question/ hypothesis. This is where
statistical models strength lies because all statistical models are hypothesis
tests and vice versa. There are very little opportunity in healthcare where
you would use deep learning compare to statistic. I've seen NLP can be of use
but the majority of work in healthcare are inference/casuality base (this is
why they use propensity model so much). I'm in this space public healthcare.

~~~
p1esk
Interesting you mentioned Makridakis competitions. There's one going on right
now on Kaggle, and the current leader believes a NN will be the winning model:
[https://www.kaggle.com/c/m5-forecasting-
accuracy/discussion/...](https://www.kaggle.com/c/m5-forecasting-
accuracy/discussion/138881)

More generally, it seems that time series forecasting so far has mostly
attracted statisticians with little DL experience [1]. Now that there is $50k
prize, this will be a good test of whether statistical methods are "still
king". If I were to enter this field, I'd probably look into latest
transformer based models, especially the ones used to model raw audio data,
e.g. [2].

There's also a real possibility that whenever any strong forecasting method is
developed (DL based or otherwise) it's not published as the developers simply
use it to make money (betting, stock market, etc).

[1]
[https://journals.plos.org/plosone/article?id=10.1371/journal...](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0194889)

[2] [https://arxiv.org/abs/1904.10509](https://arxiv.org/abs/1904.10509)

~~~
anthony_doan
I'll wait to see the result at the end of the competition.

This is just one of the two competitions for m5. The other one is uncertainty.

------
jefft255
Eric Schmidt, as in Google's ex-CEO, is the second author of this paper! I
didn't know he did any scientific research.

~~~
hervature
He has a PhD, unlike Brin and Page.

~~~
jefft255
Right, but they both were Ph.D. students and Brin I think published quite a
bit of scientific papers before dropping out.

------
wswin
for the moment I thought it was from 2003

------
throwqwerty
Looks like a good summary. Will read. But at the rate the discipline moves I
feel like we need one of these every couple of months for everyone (not just
"lay" scientists). Anyone know a good journal or something that produces a
similar sort of survey frequently? Like once a quarter?

~~~
ssivark
“Rate at which the discipline moves” is mostly churn, not progress. Important
insights come at a slower rate — at the speed of human understanding, not at
the speed of conference papers. Good papers from even decades ago are likely
to still be useful — in fact, they will have the key ideas presented simply
and clearly, without much jargon or hype. Yes, deep learning practice moves
quite fast these days, but that’s just the veneer on top of those deeper
ideas, trying out tweaks and variations. That’s not completely an indictment
of deep learning, rather, any nascent field has a lot of confusing bustle.

------
biomodel
Always wonder who these kinds of reviews / surveys are for? Nobody is going to
learn machine learning by reading a 50 page pdf. Meanwhile, people that have
experience will have a hard time finding the info they don't already know.

Opinionated & narrow >> Shallow & comprehensive

~~~
mistrial9
I will read it, to defend my non-DeepLearning choices for supervised ML .. so
many on the bandwagon for unsupervised CNN with their GPUs

~~~
mistrial9
I am misunderstood here.. it means, for the purposes that are appropriate, use
a disciplined, supervised model.. and know the strengths and weakness' of the
CNN models.. yes, some reaction to the hype of CNN..

