Quick intro for those who don't know Chaos theory or what a strange attractor is.
In the most general abstract sense suppose you have a system which is in some state and there is a deterministic rule which updates it to the next state. For now we can just represent the states and transitions as a bunch of circles and arrows connecting them. Draw it on a whiteboard if you must. Since transitions are deterministic, each state only transitions to a single other state. There might be two states that lead to one, but never one that leads to two. In general then, no matter which state you start in, it must either eventually lead to some closed loop of the same states over and over again or it runs off the page in an infinite succession of states. The first option is what we would call an "attractor" and the latter which could be aptly called a "repulsor". There's a third possibility I didn't mention (which is hard to conceptualize drawing things as a 2d diagram of states and transitions). You can have the state run off in an infinite succession of states, but rather than running off the page to infinity, it goes around and around visiting similar states over and over but never technically visiting the same state twice. Its a strange attractor.
Now to be less general. If your system states are some continuous parameter, instead of circles and arrows it makes sense to represent the nth state as x_n and to represent the transition rule to the next state as a difference equation x_{n+1} = f(x_n). If your system has more than one defining parameter, do the same thing but with vector valued quantities. Lastly, if your transitions themselves are continuous rather than discrete, replace the difference equation with a differential equation dx/dt = f(x). No matter which of these you are using to represent system states and define how that system updates, the same basic patterns as before are still the only possibility. The solution to your equation either converges to some fixed point, diverges to infinity (usually exponential growth), or the curve stays bounded within some region of state space getting arbitrarily close to forming a loop but always just missing itself.
>>There might be two states that lead to one, but never one that leads to two
There is constraint of uniqueness in deterministic differential equation solutions or chaotic maps so even if two flow trajectories appear to converge, for instance orbiting around the Lorenz equation butterfly solution, they are distinct and trace back to a given set of initial conditions.
That’s the cool thing about dynamic systems, it’s just a set of parameters that cause something akin to numerical instability but is a property of the algebraic roots in that parameter range.
Yes this is true you caught me. That's the one thing that doesn't quite fit the story when you move from my abstract graph / node story to diff eqs.
That said, I'm pretty sure you still get "value collisions" in the case of iterated difference equations (at least for some maps). Uniqueness is unique to the continuous time case.
I had to think about this for a bit I've studied dynamical systems with chaos but am nowhere near proficient in the math.
Around system fixed points there are stable and unstable manifolds. There can also be apparently (linearized) center maifolds where there is no growth or decay but that simply isn't true because it is so hard to for an eigenvalue to be exactly 0.000....0 (smooth time) or 1.000...0 for discrete maps. So you have to perform a center manifold reduction to inspect the fixed point behavior in higher order terms x^{1,2,3,..} y^{1,2,3...}, xy^{1,2,3,...}, x^{1,2,3,...}y, x^2*y^{1,2,3,...} etc. all linear combinations for the basis, in R^2 that gives so many combinations.
Around these fixed points in unstable maps the manifolds themselves will intersect in other places than the fixed point which is confusing to me (homoclinic tangle: https://en.wikipedia.org/wiki/Homoclinic_connection). But the chaotic map itself holds uniqueness of solution such that f(x_n): x_n->x_{n+1} and f^-1(x_{n+1}): x_{n+1}->x_n. This is key: deterministic systems follow uniqueness of solutions. You can trace time or steps backwards to a distinct t(0) or x_0. Without this the analysis does not work, then maybe dealing with probabilistic systems but these strange attractors arise from deterministic systems. I don't know much about probability.
Anyway I think we're not necessarily disagreeing with each other just talking past certain really interesting points.
What I think is the most conceptually simple way to visualize chaotic behavior that leads to strange attractors in R^n is the logistic map https://en.wikipedia.org/wiki/Logistic_map to visualize a R^1 system with one parameter that can cause bifurcation, phase doubling, then chaotic regions. The R^n analog to this would be a parameter set that arises in a fixed set {} instead of a fixed point x_o that the system will eventually evolve to. See double hinged pendulum as a simple chaotic system that evolves to a chaotic state. I think the system bifurcation is what you are alluding to where the roots split from one to two states and so forth.
Anyway, dynamical systems is interesting and it would be amazing to see what further is applied from it in 200 years. Its a wild field because it quickly devolves into algebra, set theory, fractals and topology. To pick up on the front you need to understand so much deep theory that is over my head.
Anyway cool stuff I love seeing these type articles here way more than the next best css framework or whatever.
I've been fascinated by NLDS and always wondered how could we use theory of chaos in, for example, psychology, treating states of mind as (strange) attractors. Would that ever be useful to use NLDS thinking in discussing mind/consciousness?
This reminds me of Chaoscope^. It reached EOL a long time ago and the binaries might have succumbed to bitrot by now, but it was a lot of fun back in the day and produced some beautiful visualizations based on strange attractors.
What kind of ODE solvers are used to simulate chaotic systems? They must be very accurate if even a small error can result in a completely different result.
Respectfully disagree; Euler's method is absolutely terrible because it's unconditionally unstable; far better is to use something like Leapfrog or velocity Verlet (both of which have 2nd order accuracy and better stability, for exactly the same number of derivative evaluations). Euler integration is essentially always the wrong tool for the job.
If you want to see the folly of RK4, give it something like a ball bouncing and watch as it bounces slightly higher each time. Subtle at first, but trust me its there. The comment above you is right. If you have conservation of energy and want to keep it that way, use verlet.
Edit: before anyone calls me out on this, the same trick also works when there's no discontinuity in the force function vis a vis collision with the floor. Planetary motion also drifts out of simple orbits. I just picked the ball bouncing because its a more amusing visual.
RK4 is about as unstable. The backwards Euler method is baby's first stable ODE solver. If you want to make RK4 stable, you have to change it into the implicit RK4 method.
However, the Lorenz system isn't stiff, which is usually what you use stable solvers for, nor any of the other systems here, I believe. RK4 should be fine, or even normal Euler with a reasonable step size. The chaos you see in these systems is not due to numerical inaccuracy. They're inherently chaotic. That's what makes chaos a subject worth studying.
James Gleick's "Chaos" remains, in my view, a great popular book on the subject, and manages to convey many of the ideas at a conceptual level without getting technical. And if you enjoy that, you might also like Strogatz's "Sync."
For slightly more technical treatment, Strogatz's Nonlinear Dynamics and Chaos is now a standard text. It isn't terribly technical and is quite well written and (in my view) easy to read for anyone with a background in vector calculus, diff eqs, & perhaps a little bit of linear algebra. (There are now other good options to, some more mathematical and some more application-oriented, but I still think Strogatz is a good place to start.)
Second Strogatz, does an amazing job linking the math to the visual intuition--probably less rigorous than some texts, but the perfect balance for an intro. Took an amazing course based on that book with Charlie Doering, who has sadly passed. He could draw multicolor, qualitatively-correct 3D attractors freehand on a chalkboard while helpfully narrating the system dynamics.
Can you expand on the last part? I mean the good other technical books for someone already a bit familiar who wants a deeper dive? Thanks for the pointers
It is a fun problem because it stems from continuous diff eq but the answers come from topological analysis around the fixed sets, and the ways fixed sets can change as continuous parameters change.
Maybe the most famous is the Lorenz attractor, arising from the Lorenz System (this is one of the simulations shown in the website). In Wikipedia you find a description of the system
In the most general abstract sense suppose you have a system which is in some state and there is a deterministic rule which updates it to the next state. For now we can just represent the states and transitions as a bunch of circles and arrows connecting them. Draw it on a whiteboard if you must. Since transitions are deterministic, each state only transitions to a single other state. There might be two states that lead to one, but never one that leads to two. In general then, no matter which state you start in, it must either eventually lead to some closed loop of the same states over and over again or it runs off the page in an infinite succession of states. The first option is what we would call an "attractor" and the latter which could be aptly called a "repulsor". There's a third possibility I didn't mention (which is hard to conceptualize drawing things as a 2d diagram of states and transitions). You can have the state run off in an infinite succession of states, but rather than running off the page to infinity, it goes around and around visiting similar states over and over but never technically visiting the same state twice. Its a strange attractor.
Now to be less general. If your system states are some continuous parameter, instead of circles and arrows it makes sense to represent the nth state as x_n and to represent the transition rule to the next state as a difference equation x_{n+1} = f(x_n). If your system has more than one defining parameter, do the same thing but with vector valued quantities. Lastly, if your transitions themselves are continuous rather than discrete, replace the difference equation with a differential equation dx/dt = f(x). No matter which of these you are using to represent system states and define how that system updates, the same basic patterns as before are still the only possibility. The solution to your equation either converges to some fixed point, diverges to infinity (usually exponential growth), or the curve stays bounded within some region of state space getting arbitrarily close to forming a loop but always just missing itself.