
Up to 2B times acceleration of scientific simulations with deep neural search - Anon84
https://arxiv.org/abs/2001.08055
======
jofer
I get more than a little annoyed with the advertising-like tone in this and
several other articles around this general class of method/trick for speeding
up complex simulations.

It's not that it's not a useful method (it is). It's that it misrepresents the
utility of the general "build an exact simulation, then train a regressor on
it to make fast approximations" approach. It's very useful in certain
situations (repeated calculations on similar parameters) and completely
useless in others.

The key issue is that most of the slow models you'd want to use this on are
highly non-linear. In certain regions of the parameter space, very small
changes in input result in very large changes in output. This is fine, so long
as you know where all of these regions are and can capture them in your
training data. That's easier for some problems than others. Even assuming you
do know how/where to collect dense training data, this approach is only useful
within the bounds of the training data you collect. It's relatively uncommon
(but not super rare) that you want to repeatedly run a complex simulation
within the same parameter space. This method is great when you do want/need to
do that, and useless otherwise.

You have to understand that what you've trained is little more than a look up
table. Anything that claims it can actually learn highly non-linear and
irregular behavior well outside of the training dataset's bounds is snake oil.

Understand where this general class of technique is useful and where it isn't
and ignore overblown claims. The entire abstract here is overblown hogwash.
The actual paper is relatively interesting, but I really wish folks would drop
the absurd advertising language and focus on what distinguishes this from the
hundreds of very similar studies/methods on this topic.

~~~
jofer
Sorry for the rant... I just feel like abstracts are the most important part
of a paper and get annoyed with poor ones. A better abstract would arguably be
something they have a few paragraphs into their introduction:

"To construct high fidelity emulators with limited training data, the machine
learning models need to have a good prior on the simulation models. Most work
to date in building emulators using random forests, Gaussian Processes,or
other machine learning models, do not fully capture the correlation among the
output points,limiting their accuracy in emulating simulations with one, two,
or three-dimensional output signals. On the other hand, convolutional neural
network (CNN) have shown to have a good prior on natural signals, making them
suitable for processing natural n-dimensional signals. However, as the CNN
priors inherently rely on their architectures,8one has to find an architecture
that gives the suitable prior of a given problem. Manually searching for the
right architecture can take a significant amount of time and domain-specific
expertise and often produces sub-optimal results.

Here we propose to solve this problem by employing efficient neural
architecture search to simultaneously find the neural network architecture
that is best suited for a given case and train it. With the efficient neural
architecture search and a novel super-architecture presented in this work, the
algorithm can find and train fast emulators for a wide range of applications
while offering major improvements in terms of accuracy compared with other
techniques, even when the training data is limited."

------
dmarchand90
Basically what jofer said...

I work in a field of condensed matter simulation, and the claim seems bizarre,
empty, meaningless. Basically it's not that original, or difficult, to train a
linearly-scaling with system size machine learning model on an underlying
physics simulation that scales very slowly.

Don't get me wrong, it's still very cool and satisfying to get reasonably
accurate results on my laptop in a few seconds what would take hours, or days,
on supercomputer.

------
_bxg1
Hopefully this only ever gets used for exploration, not verification.

~~~
jdkee
Why not?

[https://en.wikipedia.org/wiki/Four_color_theorem](https://en.wikipedia.org/wiki/Four_color_theorem)

~~~
Gormisdomai
Could you explain the link you're making here? As far as I understand four
colour theorem was computer assisted proof in the sense that mathematicians
wrote a very specific and analytical definition for it in a logical theorem
prover.

The linked article is very different right? Because it's using a neural net to
do some kind of probabilistic inference as a heuristic for imperfectly
stimulating physical events.

~~~
_bxg1
Exactly; it's about generating (much faster) heuristics for "guessing" the
outcome given certain parameters, instead of running the real simulation every
time. Useful, for sure, but the real thing should be used to verify once
you've found something interesting.

