These papers are such unutterable horse shit it makes my head spin. You know what: Kriging on the data set (or any other kind of functional approximator) would do the same thing a hell of a lot computationally faster on the 3-4 shitty solutions they demonstrated. As a bonus; it's more likely to conserve angular momentum and energy.
The fact that HN weebs gobble this horse shit up as if it were pate de fois gras is also depressing.
TLDR: physics nerds do an unimpressive thing with neural nets. You want to look at something impressive and still mysterious involving physics and neural doohickeys: why do echo state networks (reservoir computers that are effectively projections onto a random hyperplane) reproduce chaotic time series, most famously Mackey-Glass, so well?
In the paper: "The physical setup allows the initial conditions to be described by 2 parameters and the evolution of the system by 3 parameters"
Let's see, 3 input parameters, relying on a ton of pre-computed data, no ability to extrapolate beyond the precomputed numbers...they've invented the world's shittiest lookup table.
If your integrator isn't built to respect the symmetries of symplectic space it is going to give qualitatively wrong answers as t -> infinity. A non-symplectic geometry might do a better job if you want to track the Earth around the sun for 1000 years, but if you use a non-symplectic integrator who knows where the Earth ends up in a billion years.
"Deep networks" are the new hypnotic induction phase to induce willing suspensions of disbelief. Maybe there is some "emperor's new clothes" going on. Usually if somebody makes a bogus claim people will think "the person making the claim is stupid", but if a person makes a bogus claim today involving a deep network, the reader tends to assume that they are too stupid to understand it.
Neural doohickeys find real uses in computational chemistry. Accurate ab initio solvers (many-body Schrödinger equation) are very slow and everything else is just heuristic-, quasi- approximation mixed with semi-empirical tricks.
When someone makes even marginally better heuristic improvement, it's more likely that they sell it or start a company than post paper in arxiv. There is serious money to be made.
HN may provide a good reality check for AI/NN (I'm not equipped to say). But HN is not a physics site, and it's really not such a good place to go for a reality check on physics. (Not really a criticism. HN doesn't claim to be a physics site.)
For the small minority who are going to say "I'm really smart and I've taken some physics classes" ... please consider how far "I'm really smart and I've taken some programming classes" would get you on HN.
Not that that's a real physics paper, but HN has many former physicists; for example, my Ph.D. thesis had the words "three body problem" in it, and it wasn't about Chinese science fiction.
Yeah we were doing numerical linear algebra before everyone though it was the key to replacing call centers (well in my case after, but the field generally)
For sure there are physicists reading HN. Absolutely. My claim, which I probably expressed clumsily, is that the majority of commenters here are not physicists.
I think there is quite a large overlap in audience - quite a few of the readers here will have PhDs in physics in a way that few internets forums will.
There are physics people on HN. I'll stick by my claim that the crowd here is not a physics crowd. I'm confident that among web sites HN scores well above average on physics ... but discussions of physics on an average web site aren't something I really want to even think about.
I think a lot of machine learning applied to computational science problems is basically function approximation. However a lot of computational science problems actually are => here's a good basis function to approximate functions, now use it to solve an ODE/PDE. It seems machine learning is eternally stuck in the former.
Not sure if I agree. In physics you typically have the analytical solution, but it's expensive to evaluate (integral over many dimensions) so you use machine learning as a cheap way to approximate it.
In most other cases of machine learning there is no "objective" solution and hence no "target function" to approximate.
Well, all of supervised learning is basically approximating an unknown function from a finite list of samples.
But it's still an approximation, with things like e.g. backpropagation 'simply' (in the abstract mathematical sense) tweaking weights in the direction of the derivatives to get closer to expected values.
The vast majority of machine learning just builds on that by going deep (more layers), automatically generating inputs (e.g. in game AIs playing against themselves), etc.
One might argue that's even worse than function optimisation as you can only vaguely guess at the target and thus all your validation is suspect and you have to prove it using humans by, for instance, beating them at Starcraft.
> In most other cases of machine learning there is no "objective" solution and hence no "target function" to approximate.
A crucial step of any AI/ML project is to define this objective solution/target function. For example, a task like "classify photos into cats and dogs" cannot be solved by an ML system: it's too ambiguous and ill-defined. We can define a specific, unambiguous task, which we feel is somehow similar to "classify photos into cats and dogs", but it wouldn't actually be the same task.
For example, "minimise the average L2 loss across these million example inputs" is a specific task, which we can hence use ML approaches to solve. This task has an objective solution: return 100% cat for all the inputs labelled cat, and 100% dog for those labelled dog. Interestingly, a perfect approximation of this target function would probably be considered a poor solution to the original, fuzzy problem (i.e. it will over-fit); although again that would be an ambiguous, ill-defined statement to make.
There are many ML problems which aren't of the 'fit these examples' type, but they still have some explicit or implicit target function; e.g. genetic algorithms have an explicit fitness function to maximise/minimise.
Even attempts to 'avoid' this "blindly optimise" approach (e.g. regularisation, optimistic exploration, intrinsic rewards, etc.) are usually presented as an augmented target function, e.g. "minimise the average L2 loss ... plus the regularisation term XYZ"
Isn't it even in theory not possible for any algorithm (be it an ANN or otherwise) provide a solution to an infinitely variable chaotic problem based on solutions from a less-than-infinite set of other chaotic problems?
Isn't essentially they key concept of chaotic problems that they aren't predictable, so there is no real "pattern" to train on so to speak?
If there weren't a real underlying system how would the universe function operationally? How do the bodies "know" how to move even though we can't predict them past a certain level of inaccuracy?
Suppose we know the system's state and rules exactly. This can't be true in the real universe, but we can construct classical, deterministic systems in pure math that we can say that about, and even those systems will exhibit this characteristic we're talking about.
You can look at it from the point of view of, if we watch the system evolve, can we tell whether the rules were violated at some point, by some arbitrarily small amount? As the chaotic systems evolve, it becomes harder and harder to tell if that is the case. There isn't a discrete transition from knowing to not knowing; our level of knowing goes down over time.
In information theory, we can see that as a loss of bits of precision on the system, requiring more and more bits initially to make up for it. Since we can't compute with real numbers, but only approximations given increasingly more bits over time, even in the pure mathematical case where everything is perfect and specified, we still lose this knowledge as the simulation progresses. It's that much worse in the real world, where we don't even start with all that many bits of precision.
It's not quite the question you asked, but... it's like the shadow of the question you asked, and it's a bit easier to explain. (And reasonably mathematically valid. You can characterize chaotic systems by how many bits they lose per time unit.)
This is such a great response that I’m not even going to voice my metaphysical objections because I’m absolutely certain you already know them and perhaps even explore them yourself. Have you got any recommendations for further reading?
It depends on the direction you want to take, but there is a mathematical basis to what I'm saying, not just a philosophical one. Unfortunately I've never scared up the name of the concept, or at least not in any form I can conveniently Google for. I'm not sure there is a non-textbook version of what I'm talking about; I'd like to see it myself.
It's related to the ability to read a Lyapunov exponent as a measure of bit loss. Lyapunov exponent is easy to Google up, and if you understand that and information theory it's not a difficult leap to make, but I can't find any nice explanation for people who don't already have those things.
The universe functions because all the rules are in place. But predicting the outcome of those rules without actually "replaying" them may be impossible.
Example 1: Conway's game of life.
Example 2: Collate conjecture.
If the number is even, divide it by two.
If the number is odd, triple it and add one.
The rules are deterministic. But you can't do any predictions other than running the simulation.
It's not magic, so is it safe to assume the ANN is quickly narrowing down the search space by eliminating the possibilities that were not showing up as patterns in training set therefore limiting the search to a finite subset of infinite set of possible behaviors of the system?
If the chaos produces a new kind of behavior the result of the ANN may be totally wrong. In other words - it works well, often.
i need to go out, so this is just off the top of my head, but isn't this really just some kind of efficient approximation? in which case (1) can it be extended to other problems trivially (so that we have an optimization library that we can 'plug in' to arbitrary numerical problems) and (2) how good is it at extrapolating?
The fact that HN weebs gobble this horse shit up as if it were pate de fois gras is also depressing.
TLDR: physics nerds do an unimpressive thing with neural nets. You want to look at something impressive and still mysterious involving physics and neural doohickeys: why do echo state networks (reservoir computers that are effectively projections onto a random hyperplane) reproduce chaotic time series, most famously Mackey-Glass, so well?