Second, (and the author may be leading up to this), there's Solomonoff's theory of inductive inference, which he has proven complete: when we apply Occam's razor (where the prior probability of each possible theory drops exponentially with its size), the amount of error a perfect Bayesian makes as they observe event and bet on the next one, ad infinitum, is finite. Roughly proportional to the complexity of the simplest theory that correctly predict the whole sequence of events. It's one of the most convincing proofs that Bayesian reasoning works. https://en.wikipedia.org/wiki/Solomonoff's_theory_of_inducti...
There's just a little snag. Perfect Bayesian reasoning is impossible to compute, so us mortals have to resort to approximations. Just as perfect certainty isn't possible, perfect reasoning is not attainable. Oh well.
> the amount of error a perfect Bayesian makes as they observe event and bet on the next one, ad infinitum, is finite.
There's a little assumption that you're leaving out, namely that the Kolmogorov complexity of the data generating process is finite. From Wikipedia:
> expected cumulative errors made by the predictions based on Solomonoff's induction are upper-bounded by the Kolmogorov complexity of the (stochastic) data generating process
Whether the universe (or our observations of the universe) have finite complexity is very much an unresolved philosophical question.
> Whether the universe (or our observations of the universe) have finite complexity is very much an unresolved philosophical question.
Every time there was a significant advance in physics, it tended to go towards simplification and unification. Geocentrism required epicycles. Then Keppler came up with his ellipses. Then Newton unified celestial and terrestrial laws. Maxwell & Einstein allowed us to view time as less special dimension than we thought it was…
I won't presume about the initial state of the universe, to the extent such a notion is even meaningful. But the fact that it is governed by mathematics, and relatively simple maths at that, sounds likelier and likelier every quarter-century.
And I'm not even talking about everyday life, where we can observe in practice that the simplest theories about who ate the last cookie (little Mike, who lives in the house) are more often true than the more outlandish ones (magical imps, which we never witnessed).
> Every time there was a significant advance in physics, it tended to go towards simplification and unification. Geocentrism required epicycles. Then Keppler came up with his ellipses. Then Newton unified celestial and terrestrial laws. Maxwell & Einstein allowed us to view time as less special dimension than we thought it was…
Unification, maybe. Simplification, no. That's evident if you just scroll down the list of Nobels in physics. You even mentioned Einstein, but I don't know how you could claim general or special relativity are simpler than Newtonian physics.
> I don't know how you could claim general or special relativity are simpler than Newtonian physics
Careful there! You cannot compare both theories in isolation from observation. Newtonian theory fails to match observation if high velocities or big masses are involved.
In order to "fix" that using just Newtonian physics, we're back to figurative epicycles.
Taking observations into account, SR is simpler than Newtonian physics in that it has a greater predictive power.
Remember that if you come up with something simpler than SR it also has to match observation at least as well as SR.
Under the many-world interpretation, when you send a photon through a half sieved mirror, the universe splits in one version where the photon goes through, and one universe where the photon doesn't. This is all very deterministic.
What the researcher subjectively observe however is another matter. If the universe splits, so does the researcher. The problem of observing outcomes turns into an anthropy problem: if I split myself in two copies, in which copy am I likeliest to find myself into? I'm not sure making bets about that even makes sense: which copy I find myself into has no bearing in the final state of the universe.
Most bitstrings are not compressible, so if we look at all the observers across the splitting universes watching a continuous quantum coin flip, most will observe a coin flip sequence that is not compressible, so will observe a constant increase in Kolmogorov complexity.
No, just because a process is probabilistic, in does not immediately follow that it has infinite Kolmogorov complexity. For example the probabilistic effects could cancel out.
Vitanyi -- one of the authors cited in the linked piece -- has a paper where he and his colleagues show that even though ideal compression isn't knowable/computable, you can show that two encodings approach the ideal differentially with some computable probability (I'm being vague and handwavy because it's been awhile since I've read this literature carefully). That is, if you have two encodings A and B, you can compute some probability that A is closer to ideal than B.
It's kind of a Kolmogorov/algorithmic complexity analogue of a p-value.
I think this literature/inferential paradigm in general is far less known than it should be.
No mental construct can exist without a cognition engine to run it. No idea can be thought without a mind to think it. Information has to be stored somewhere, and that somewhere is ultimately physical: a nervous system, a hard drive, a piece of paper…
First, probability is real: it's a construct in one's mind, and minds are just as real as dice or coins. https://www.lesswrong.com/posts/f6ZLxEWaankRZ2Crv/probabilit...
Second, (and the author may be leading up to this), there's Solomonoff's theory of inductive inference, which he has proven complete: when we apply Occam's razor (where the prior probability of each possible theory drops exponentially with its size), the amount of error a perfect Bayesian makes as they observe event and bet on the next one, ad infinitum, is finite. Roughly proportional to the complexity of the simplest theory that correctly predict the whole sequence of events. It's one of the most convincing proofs that Bayesian reasoning works. https://en.wikipedia.org/wiki/Solomonoff's_theory_of_inducti...
There's just a little snag. Perfect Bayesian reasoning is impossible to compute, so us mortals have to resort to approximations. Just as perfect certainty isn't possible, perfect reasoning is not attainable. Oh well.