
A neural net solves the three-body problem 100M times faster - EndXA
https://www.technologyreview.com/s/614597/a-neural-net-solves-the-three-body-problem-100-million-times-faster/
======
conistonwater
If I understood correctly, on page 2 they say they generated their training
and test cases using random initial positions and velocities for the 3-body
problem. But this is exactly the sort of case that a non-sophisticated solver
should be able to handle easily as well. Brutus is special in the sense that
it tries hard to give some sort of a guarantee about the accuracy of its
solution to a chaotic problem. But the neural network solver provides no such
guarantee, and has not been trained or evaluated on "hard" examples. I'm not
sure the comparison is fair or particularly meaningful. I wonder if exactly
the same sort of "positive" result could be achieved with a standard
symplectic integrator, it's not at all obvious from the paper that they have
ruled that out.

~~~
pixelpoet
Argh, just looked up Brutus and it's pretty much _exactly_ what I wanted to do
on my own after quite a lot of reading on n-body simulations and finding
nothing like it out there. I was (very naively, I realise) hoping to do some
original research.

Oh well, time to study this intensively. Here's a link for the curious:
[https://arxiv.org/abs/1411.6671](https://arxiv.org/abs/1411.6671)

~~~
aaronblohowiak
That’s total validation of your thinking!!!

~~~
jszymborski
You know, as a PhD student, I sometimes feel like the moment I come up with a
good idea, it's guaranteed to show up as a result on Google Scholar the next
day, which can be rather discouraging.

I've never really thought about it in terms of your comment, however, which is
actually very motivating. Thanks for the perspective random internet person.

(Apologies for the OT comment)

~~~
mehrdadn
Reminds me of when I asked a professor once "why don't they try <alternative
approach> instead?". He replied, "I tried that when I was a grad student. It
turns out not to work that well because <reasons>." I never figured out if
that validated or invalidated my thinking, but I found it funny and kind of
eye-opening either way.

~~~
madhadron
A lot of the value of being in a department is being able to have these
shortcuts. I remember as an undergrad in physics asking why we didn't use
imaginary time to formulate special relativity? The grad students around me by
and large looked puzzled and said they had never thought of it. A very, very
experienced nuclear/mathematical physicist said, "Oh, yeah, folks did try that
back in the early 20th century. It works fine, but you have to go to a full
metric for general relativity, so why carry around two formalisms?" I
contemplated that for about thirty seconds, said, "That makes sense," and
happily put aside the question.

It vindicates your thinking in that it indicates that you are picking up the
techniques that professionals use to generate ideas. You just have to accept
that as you're learning, most of the ideas will have been generated and
addressed.

What gets awkward is when you're doing interdisciplinary research and you're
using the generative mechanisms from one discipline in the other. By and large
both sides stare at you in confusion, or it requires multiple hour
conversations to lead the other discipline's practitioners into a mental space
where they can say, "Oh, yeah, that wouldn't be useful because of X." It makes
the process of tuning your idea generation much slower. On the flip side, you
hit the unknown way faster, sometimes great vistas of unknown where everyone
goes, "Umm...wow. Yeah. Never heard of that continent. Let us know what you
find."

------
EndXA
The original paper can be found here:
[https://arxiv.org/abs/1910.07291](https://arxiv.org/abs/1910.07291)

Abstract:

> Since its formulation by Sir Isaac Newton, the problem of solving the
> equations of motion for three bodies under their own gravitational force has
> remained practically unsolved. Currently, the solution for a given
> initialization can only be found by performing laborious iterative
> calculations that have unpredictable and potentially infinite computational
> cost, due to the system's chaotic nature. We show that an ensemble of
> solutions obtained using an arbitrarily precise numerical integrator can be
> used to train a deep artificial neural network (ANN) that, over a bounded
> time interval, provides accurate solutions at fixed computational cost and
> up to 100 million times faster than a state-of-the-art solver. Our results
> provide evidence that, for computationally challenging regions of phase-
> space, a trained ANN can replace existing numerical solvers, enabling fast
> and scalable simulations of many-body systems to shed light on outstanding
> phenomena such as the formation of black-hole binary systems or the origin
> of the core collapse in dense star clusters.

~~~
xgbi
I expect KSP will finally get a real N-body simulator! My mun orbit insertion
will be even harder!

~~~
perl4ever
When I read that KSP _didn 't_ use "real" N-body physics, I lost interest in
trying it. Maybe it doesn't make a big difference, but I don't understand how
it would save any noticeable amount of cpu cycles. I mean, there aren't that
many objects in the game, are there? It's not like it involves galaxies or
star clusters.

~~~
wcoenen
That's because N-body physics implies unstable orbits.

For the planet scale objects, the game designers would have to make sure that
the orbits don't go crazy (over times that can be explored at maximum time
warp). But then if the goal is to keep those orbits stable, why not just save
all the dev effort and just keep them on Keplerian rails?

For the smaller objects launched by the player, unstable orbits aren't
necessarily desirable either. Imagine going on a multi-decade mission and
returning home to find your favorite space station missing. Game devs aren't
going to spend effort on complicated features that will just piss off most
players.

------
spodek
> _one of the classic problems of applied mathematics._

Maybe I'm biased with my physics PhD, but I'd call it a physics problem.
"Solving" it to me means a closed-form solution, not a particular prediction,
but I guess the applied math problem is a different problem with a different
solution.

~~~
oefrha
Physics PhD here. I'd call it both. A lot of what applied math departments
study are problems of physical nature; and a lot of physics research boils
down to applied math. (Broadly speaking, all theoretical physics is applied
math.)

Guess I'm biased too because I'm a theorist and started out from a math
background.

~~~
ska
I went the other direction, physics to math, but agree with your broad
characterization. Not all applied math is physics, obviously. I do find
mathematicians and physicists often have different characteristic ways of
thinking about similar things, but the lines are blurry.

------
tiborsaas
It's not a surprise that after brute forcing a problem with neural networks
and generating a black box model it becomes much faster than on-the-fly
computations.

The post made me remember this article about predicting chaotic equations with
machine learning with great accuracy:

[https://www.quantamagazine.org/machine-learnings-amazing-
abi...](https://www.quantamagazine.org/machine-learnings-amazing-ability-to-
predict-chaos-20180418/)

------
vladimirralev
I've been wondering if neural nets will able able to find solutions to PDEs
with sparse inputs such as this one. Intuitively it makes sense that fast
solvers would exist for any PDE where the input data is either very sparse or
highly symmetrical. A lot of the PDE domain can be "meshed" advantageously
with portions of the domain either reusing computation or being wiped out as
insignificant contributions. A neural net can learn to represent a PDE domain
in terms such regions. I am really curious if this is what happens here and if
there is something vastly beyond what current PDE solvers are able to do.

But I also imagine as the input size grows within the orders of the hidden
layers it will lose the advantage.

~~~
kuiooullkre
But how are you going to trust it?

If the neural net gives you a solution, is there any cheap way to confirm it?

~~~
TheEzEzz
Depends on the problems, but in many situations yes. For PDE's you can plug in
the proposed solution and calculate the error. Usually you'd do this in a
discretized version of the PDE, but since neural nets are symbolically
differentiable you could instead approximate (by sampling) the average error
from the real PDE itself.

~~~
xiphias2
Sounds like a good way to train an adversial network

------
tompccs
Not had a chance to read the paper yet but the ability to find computational
shortcuts in a epistemically minimalist way (ie, essentially admitting that we
have no idea how to solve the problem and let a computer iteratively work it
out) for solving analytically irreducible PDEs makes me _very_ bearish on
quantum computing. A quantum system can be described by Schrodinger's
equation, which is no more esoteric than any other wave equation. If it's hard
to solve now, maybe it's just because the computationally efficient methods
can't be reached by mathematical analysis. But it doesn't mean those
computational methods don't exist.

------
hibijibies
I like all the amazing comments and critique there but have we considered that
somehow, the neural network was able to formulate the concept of gravity,
emulate a chaotic system and do all of it using a constant computational cost?
It is a bunch of interacting units that reconfigured itself using gradient
descent to behave like a chaotic three body system. I find that quite
fascinating. Although I would agree that the hype created is not justified but
this neural network approach bypasses concepts of gravity, newtonian
mechanics, chaos, ODE integration and gives a simple function to calculate
position at arbitrary times. Quite interesting.

------
badrabbit
Can someone explain: When they say solve,do they mean it can show state after
N iterations accurately after sufficient computation? Or do they mean it can
find a formula for the bodies where state can be predicted at constant time
regardless of iteration?

~~~
antepodius
The first. For N>=3, there is no such formula (it's been proven).

~~~
badrabbit
Thanks, didn't know lack of formula was prove. I thought it was simply not
found yet.

------
bigred100
This may be news for reasons I’m not aware of (it got an MIT writeup), but as
someone who worked for a bit in scientific computing, using a neural network
to build a reduced model has basically no novelty to me.

------
amelius
Possibly unrelated, but did anyone ever attempt to accelerate the solution of
linear systems of different kinds using neural networks?

------
hcarvalhoalves
Basically, memoization, trading off accuracy for space instead of time?

It’s also faster since you never factor in the training time for the NN,
alright.

