
Project Cybersyn and the origins of the Big Data nation - blackbagboys
http://www.newyorker.com/magazine/2014/10/13/planning-machine
======
bcoates
If this interests you, you might like the book "Red Plenty".

It alternates between novel and essay about Khrushchev-era USSR, when people
on both sides of the curtain thought there was a pretty good chance that the
Soviets would succeed, if not as a Socialist Utopia at least in economically
overtaking the west in output and quality of life.

Also, this excellent essay about the computational aspects of Gosplan, the
USSR's mostly human-powered predecessor to Cybersyn:
[http://crookedtimber.org/2012/05/30/in-soviet-union-
optimiza...](http://crookedtimber.org/2012/05/30/in-soviet-union-optimization-
problem-solves-you/) (which is where I learned about Red Plenty)

~~~
brendano
Also, see Cosma Shalizi's excellent review of "Red Plenty", which includes a
mini-introduction on the application of linear programming to economic
policymaking.

[http://crookedtimber.org/2012/05/30/in-soviet-union-
optimiza...](http://crookedtimber.org/2012/05/30/in-soviet-union-optimization-
problem-solves-you/)

------
omnibrain
About 10 years ago I intended to read "Towards a new Socialism", after reading
an article about the chileniean experiment that also mentioned this book.
Perhaps it was actually an article about the book. I don't quite remember. I
never read the book and only got reminded about it by this article now. The
topic itself is quite fascinating, especially if you think about what amount
of planning happens in huge companies.

You can, if you are interested in the topic, find the book at
[http://ricardo.ecn.wfu.edu/~cottrell/socialism_book/](http://ricardo.ecn.wfu.edu/~cottrell/socialism_book/)

~~~
trhway
As they say - the bigger the company, the more socialism there (inside the
company).

------
skylan_q
This is relevant:
[http://en.wikipedia.org/wiki/Leonid_Kantorovich](http://en.wikipedia.org/wiki/Leonid_Kantorovich)

Although not dealing so much with big data, it was still a "socialist" attempt
at mathematical optimization being applied in society.

~~~
omnibrain
Leontief's input-output model is also interesting:
[http://en.wikipedia.org/wiki/Input–output_model](http://en.wikipedia.org/wiki/Input–output_model)

------
bluekeybox
Related and a very good BBC documentary on the role (and tragedy) of
engineering and planning in the USSR:
[https://www.youtube.com/watch?v=h3gwyHNo7MI](https://www.youtube.com/watch?v=h3gwyHNo7MI)

------
otakucode
... and then one day someone read a paper about chaos theory and computational
complexity and quickly realized that predicting systems with chaotic behavior
is guaranteed to be impossible.

~~~
quinnchr
It's not impossible at all. In fact there is entire field of study devoted to
the long term evolution of dynamical systems
([http://en.wikipedia.org/wiki/Ergodic_theory](http://en.wikipedia.org/wiki/Ergodic_theory)).

And within a chaotic system there are several types of stability that can be
readily predicted over a long period of time such as Lyapunov stability and
asymptotic stability.

Why do you think this system would be chaotic in the first place?

~~~
otakucode
Chaotic in the sense of sensitive dependence upon initial conditions? Because
human actions depend upon a large number of factors influenced heavily by
their environment and their life experiences. It's far harder to find systems
which are not chaotic than ones which are. Even the classical 3-body system
interacting under gravitation is chaotic.

And while, yes, there is definitely good work being done in studying dynamic
systems, applying those things to human beings is very difficult and
dangerous. We already often face situations where humans are expected to
conform to a limited set of predicted responses rather than the system being
seen as flawed for failing to predict all possible responses. And human beings
react very, very poorly to being restricted in these ways.

~~~
quinnchr
I totally get what you're saying about the unpredictability of human
behaviour. But I don't see the connection to this particular system. It could
totally be chaotic if it was modelled as a dynamical system, I don't disagree
with that, but as far as I can tell they approached it using neural
networks...

Chaos isn't an intrinsic behaviour of humans, it's a property arising from
specific conditions in a dynamical system. I do think there's been interesting
results in trying to model human behaviour
([http://www.wilmott.com/pdfs/090610_mandelbrot.pdf](http://www.wilmott.com/pdfs/090610_mandelbrot.pdf)),
but like you I'm hesitant to apply these mathematical models to us irrational
humans. Anyway, I'm just being a pedantic math nerd.

P.S. Sensitivity to initial conditions alone does not imply a dynamical system
is chaotic.

------
bsbechtel
I only read the beginning of the article and skimmed the rest, but this seems
to be a misleading article. Just because the Chileans had the idea they could
use massive amounts of data and computers to achieve their socialist goals
doesn't mean this was the origin of big data analysis in companies today. It
doesn't take a quantum leap in creativity, imagination, or innovation to
realize you can combine computing power and lots of data to glean insights
into customers, or in socialism's case, societal (i.e., the government's)
goals.

~~~
trhway
It isn't about customer insight, it is about control system in cybernetics
sense (where the more system information reflected in observation vector, the
more chances you have to generate optimal control input). While capitalist
society puts a lot of belief into emergent behavior of the system, socialist
is obsessed with optimal control of the system.

