
What is ergodicity? (2016) - lamby
https://larspsyll.wordpress.com/2016/11/23/what-is-ergodicity-2/
======
uniformlyrandom
Interestingly, ergodicity has two meanings ([https://www.merriam-
webster.com/dictionary/ergodic](https://www.merriam-
webster.com/dictionary/ergodic)):

1\. of or relating to a process in which every sequence or sizable sample is
equally representative of the whole (as in regard to a statistical parameter)

2\. involving or relating to the probability that any state will recur
especially : having zero probability that any state will never recur

Nassim Taleb mostly uses ergodicity in the second meaning, while the article
concentrates on the first. I find it utterly confusing.

~~~
pcstl
Honest question: Aren't both equivalent? If the process will eventually
traverse its entire state space, you should be able to infer the behavior of
the process across all time from a finite (sufficiently large) sample of the
process's behavior.

~~~
AstralStorm
Second version is much more loose, e.g. most semi-stationary processes fit the
bill.

In fact article makes a mistake by requiring stationary nature from ergodic
process. You can have an ergodic process of the first kind where ML
expectation is equal to the value of the process function, which nonetheless
is not stationary in absolute sense. (Because definite integral does not match
indefinite.) Just perfectly predictable and having the indefinite integral
identical to integral over space state. Which implies strictly bounded but not
exactly stationary in any defined term.

Likewise you can prove an equivalence class of the process to be identical to
another known ergodic process without actually calculating such difficult to
obtain complete integrals.

And ergodicity is not required for representative sampling either, just
definite error bounds on predictive (statistical) power or likelihood
estimates. Which can be estimated or directly calculated for many kinds of
processes accurately. It is only required for certainty.

------
fasteo
There is a recent paper[1] and an epic twitter thread[2] well worth the
reading

[1]
[https://arxiv.org/pdf/1906.04652.pdf](https://arxiv.org/pdf/1906.04652.pdf)

[2]
[https://twitter.com/hulme_oliver/status/1139148255969906689](https://twitter.com/hulme_oliver/status/1139148255969906689)

~~~
mturmon
The twitter thread gives a nice example of a failure of ergodicity that I was
not aware of:

"Now consider a gamble with multiplicative dynamics.

Win 50% of your current wealth for heads, lose 40% of your current wealth for
tails.

Changes in wealth now are non-ergodic, so calculating the expectation value is
not informative of the time average growth rate of wealth.

This exact gamble has a positive expectation value, but it has a negative time
average growth rate. We call it the _Peters coin game_."

Clearly the expectation of return at a fixed time is > 0, but they give an
example time series with obvious negative drift.

~~~
gowld
How does this work?

By binomial theorem, the expectation at time t is ((1.5 + 0.6)^n)/2^n) =
1.05^n so the "annualized" return per unit time is 1.05, exactly the same as
the expectation.

What is "time average growth rate"? Web search turns up lots of references do
it by Peters, but no definitions.

~~~
yellowstuff
You have the wrong model. You're betting a different amount each time. Say you
start with $1 and get a heads then a tails:

($1 * 1.5) * .6 = $.90

What if you get a tails then a heads?

($1 * .6) * 1.5 = $.90

Doesn't seem like such a great game now, does it?

The Kelly Criterion sets a limit for how much of your total wealth you should
bet when the odds favor you. If you bet more than that limit you increase your
odds of losing everything without improving your expected return.

So imagine a better game where you could bet any amount, and you still got
paid $5 for every $4 you risked and had a 50% chance of winning. The correct
amount to bet is 10% of your bankroll. If you bet more than that in the long
run you will go broke.

~~~
gowld
You are ignoring half of the possible outcomes. If you get HH or TT, that's
$2.25 and 0.36, which averages to $1.25.

50% of ($1.25 + $.90) for a $1 bet seems like a great game to play.

~~~
yellowstuff
This is actually an awesome illustration of why a lot of famous results in
academic finance were wrong in the 70s and 80s. Returns don't average, they
multiply. If you get HHTT or TTHH you're in terrible shape:

2.25 * .36 = .36 * 2.25 = .81

You started with $1 and now have 81 cents. I recently learned that a lot of
early computer-era academic finance actually made the same error of averaging
returns, as described in Finding Alpha:

[http://falkenblog.blogspot.com/2016/08/finding-alpha-
pdf.htm...](http://falkenblog.blogspot.com/2016/08/finding-alpha-pdf.html)

------
cs702
In addition to the paper and Twitter thread mentioned by fasteo elsewhere on
this thread[a], this video of a lecture by the author is well-worth watching
too:

[https://www.youtube.com/watch?v=f1vXAHGIpfc](https://www.youtube.com/watch?v=f1vXAHGIpfc)

Among other things, in this lecture the author shows, step by step, in non-
technical language, an example with coin tosses in which changes in wealth are
non-ergodic (in the sense he explains), such that the expected value of
winnings over time is not informative of the time average growth rate of
wealth.

[a]
[https://news.ycombinator.com/item?id=20285638](https://news.ycombinator.com/item?id=20285638)

------
ska
Getting a good feel for both stationarity and ergodicity in systems is
worthwhile; assuming them is a common way to “intuitively” infer incorrect
behaviour.

------
claudiawerner
The author writes,

>Paul Samuelson once famously claimed that the “ergodic hypothesis” is
essential for advancing economics from the realm of history to the realm of
science.

It's curious that science and history are pitted against each other; not only
is science notoriously difficult to define, but there are several historical
approaches to scientific objects (e.g archaeology for the object of human
history). This also seems to involve a logical model in which the concepts
stay static, but some popular models outside of mainstream economics are used
because the authors argue that an approach divorced from real development
leads to results that don't apply to real situations (e.g. the dialectical
logical method, which is both syntactic _and_ semantic.

Unfortunately the sort of logic Samuelson applies has been misapplied (in neo-
Ricardian lenses) to thinkers which seem to hold strictly temporal (rather
than equilibrium) interpretations of economy.

------
selimthegrim
Ergodic and chaotic are not the same either (the difference being in long time
diverging behavior of two trajectories that start very close together); many
lazy physics science writers and occasionally physicists conflate them.

------
Liquix
Great read. Eat your heart out, Betteridge's Law!

