
Randomness: the Ghost in the Machine? (2014) - probe
http://www.3quarksdaily.com/3quarksdaily/2014/09/randomness-the-ghost-in-the-machine.html
======
visarga
Just want to add that biological neurons are stochastic and that artificial
neural nets rely on noise injection, random connection dropout, random
initialisation and random mini-batch reshuffling to work well. GANs start from
a random number in generating images or other types of data. Alternative
optimisation strategies to gradient descent - such as evolutionary strategies,
use randomness in combining genes for the next generation of agents.

Without randomness we would have no intelligent artificial agents. It's like
intelligence can only happen at the demarcation line between chaos and order.

~~~
thomasahle
> Without randomness we would have no intelligent artificial agents. It's like
> intelligence can only happen at the demarcation line between chaos and
> order.

I doubt most NN's are trained using true randomness. Most likely a fairly
simple deterministic generator works just fine.

------
dahart
> you are not your body, you are not your brain, you are not your genome, you
> are not your connectome, you are not what you eat, you are not your social
> network, you are not your taste in music, you are not your history

This totally reminded me of the Chuck Palahniuk quote from Fight Club: “You
are not your job, you're not how much money you have in the bank. You are not
the car you drive. You're not the contents of your wallet. You are not your
fucking khakis. You are all singing, all dancing crap of the world.”

This article covered a lot of ground, I quite enjoyed it. It made me think
there’s a big irony, or maybe it’s a paradox.

We need the existence of true randomness in physics in order to allow for free
will. And yet willfulness is the exact opposite of randomness.

~~~
kuroguro
I never really understood why randomness would be needed for free will. If
that's the only thing keeping the will "free" would it even matter if our
decisions are made deterministically or by a dice roll?

Guess I have some reading to do.

~~~
pmoriarty
I think the point is that if there's no randomness, people's actions must be
completely predictable, and if they're predictable it's impossible for people
to do other than what's been predicted and so there's no free will.

On the other hand, having randomness opens up the possibility to act in ways
that are not completely predictable and that's where the potential for freedom
of the will comes in. But randomness alone isn't enough for free will, for the
reason you allude to (that one might be "forced" to follow some sort of cosmic
"dice roll", as unpredictable as it might be). It just seems to be one pre-
requisite, in a certain way of viewing the problem.

~~~
syrrim
But the universe is turing complete, so the only way to predict what will
happen is to run a perfect simulation of everything in the universe. This is
clearly impossible within the universe, and so your actions remain
unpredictable.

~~~
mannykannot
This line of thought applies equally well to every individual hydrogen atom in
the universe as it does to a person, yet it does not follow that they also
have free will. Where OP says 'predictable', I think he means 'determined' or
'pre-ordained'.

~~~
dingo_bat
But since you cannot differentiate between predictable and preordained, what
is the difference?

~~~
mannykannot
That something is not predictable does not mean that it is not pre-ordained,
even if one cannot tell the difference. Classical mechanics is a model that is
fully deterministic yet ultimately unpredictable, and similarly, syyrim's
model of the universe as Turing-complete seems to assume it is pre-ordained (a
model that goldenkey challenges), and yet ultimately unpredictable.

On re-reading the thread, I am not sure what point syyrim was making. The
ultimate unpredictability of the universe doesn't seem to settle any issue
with regard to free will. Free will is not a claim that we cannot predict what
we will do; it is a claim that we can make a choice (within some constraints)
about what we will do.

------
placebo
Interesting how the concept of randomness can only exist in contrast to the
recognition of order, but who notices order other than a mind that defines it
as such? But this mind could not have been created in the first place were it
not for some intrinsic order, some consistency in reality. Makes my head
spin...

~~~
visarga
> But this mind could not have been created in the first place were it not for
> some intrinsic order

The mind was created by self replication and competition over multiple
generations (evolution). Self replication requires ability to adapt to
environment, to collect energy and materials, to protect from predators. This
was the playground where minds evolved. We're using similar techniques in
AlphaGo, for example, where a sequence of players are pitted against each
other.

Randomness is essential for exploration, which is a step without which we
couldn't be intelligent. In reinforcement learning, the so called epsilon-
greedy strategy of exploration says to select a random action with probability
epsilon. It is a simple strategy that leads to intelligence. Of course, humans
have smarter exploration strategies which nonetheless rely on randomness.

------
air7
"Free Will" as a concept just makes no sense. I think it's obvious that any
system can be conceptually described as a function of all-inputs-that-affect-
it, plus randomness. No room for free will.

The only thing free will has going for it is the unmistakable, unshakeable
feeling that I _really_ have it.

~~~
Viliam1234
And that feeling probably just means: "I can't reliably predict myself." Which
is more about one's ability to predict, than about some intrinsic
unpredictability.

------
everdev
I'm not familiar with the inner workings of random number generators but is
true randomness even possible or is the randomness just so complex that we
can't reverse engineer it?

My instinct says that if you run the same random number generator in the same
hardware with the same Unix timestamp and same inputs wouldn't the outputs be
the same?

~~~
dahart
If your random number generator is pure software, it’s repeatable.

People sell hardware random number generators that are based on Geiger
counters, since radiation is (as far as we know) truly random.

~~~
martinpw
I believe some (all?) new Intel processors have built-in hardware components
which are a source of truly random numbers.

~~~
dahart
I just read about it, and it looks like a very strong and fast RNG.

[https://spectrum.ieee.org/computing/hardware/behind-
intels-n...](https://spectrum.ieee.org/computing/hardware/behind-intels-new-
randomnumber-generator)

But, fwiw, this isn’t “truly” random in the quantum sense. It’s only difficult
to predict as far as anyone knows, but is still built on deterministic
processes. Radiation, on the other hand, is (as far as anyone knows)
physically “truly” non-deterministic.

------
laretluval
In a maximally efficient interactive system, user input appears as random
noise. This is because such a system will contain a model of the distribution
of its input in order to be able to predict it and react as soon as possible.
The system only needs to react to the component of input which it was not able
to predict using this model. Thus if there is an intelligent entity
manipulating our universe, and if our universe is efficient in this sense,
then the outside entity's input will appear as perfectly random maximum
entropy noise when viewed from within the system.

