
Three questions that keep me up at night - andrewnc
https://blog.evjang.com/2020/04/three-questions-that-keep-me-up-at-night.html
======
mannykannot
> Q1: Can we imitate "thinking" from only observing behavior?

If you believe that the human mind is the result of Darwinian evolution, then
you implicitly believe that thinking can arise from interacting with the
environment. That does not exactly answer the question, but if anything, it
goes beyond it.

On the other hand:

"This could also be addressed via other high-data environments:

1) Observing trader orders on markets and seeing if we can recover the
trader's deductive reasoning and beliefs about the future. See if we can
observe rational thought (if not rational behavior).

2) Recovering intent and emotions and desire from social network activity."

Before you could even begin to get anything from these streams of symbols, you
would have to have some way to give them semantics - to understand what they
signify to the actors. I suspect that these would be among the last things you
would achieve, not a means to the end.

------
TheMechanist
Question 2: This is better left unanswered. We've been building superhuman AIs
for at least 300 years which are exploiting reality to their advantage and
have started out-competing humans for resources (including by changing laws in
their favour). These things we've been building are called corporations, and
they're certainly smarter than any individual, richer than any human can
become and are now working hard to figure out how their humans can be replaced
by more reliable components.

------
ethn
Q1) By the mere copying of behaviors, no. It is clear that there are at least
the a priori intuitions of space, time, and causality necessary for humans to
obtain any knowledge. It is within that a priori framework where concepts and
then decisions are derived from.

Q2) This would require the knowledge of what is sufficient & necessary for
AGI.

Q3) Minsky, Neumann, and most sophisticatedly, Norbert Weiner answered this
question. Read page 33 of Weiner's Cybernetics. The real insight, where in
what I will now explain in brevity is at the expense of the insight's
resolution, is that in the nature of microscopic phenomena there are an
exorbitant quantity of fickle components ("equal particles") whose positions,
accelerations, velocities are all impossible to record because of that sheer
quantity by the nature of this sphere of study. More so, there are then at
least second-order coupling effects between all these fickle components due to
their constant interaction. As a resolve, we have developed terms of
statistical understanding. Weiner cites "cloud, temperature, turbulence, etc."
all being concepts which are specifically defined in a statistical context.

Where as in the macroscopic sphere there are far fewer particles much further
apart from each other with near certain initial masses, positions, and
velocities.

The result? In the microscopic systems, because their properties are precisely
defined statistically the distribution which predicts the future nor past for
a given system isn't unique. That is, the future state (predicted by the
distribution provided by some initial system) could have also been predicted
by other distributions describing many other initial systems unless by "some
miracle" there is a "tightly defined statistical range".

So are microscopic physics condemned to this paradox? Not necessarily, there
may be some way to construct scientific objects from systems phenomena which
don't require a statistical conception because they can be precisely defined
from a microscopic systems point of view temporally & spatially. However, this
would require a revolution in theory and likely mathematics

~~~
lidHanteyk
I dislike the cybernetic approach because I find that it first commits to
epistemic helplessness, saying that we know nothing of particles. Perhaps that
was reasonable when QM was so young, but these days we can use QM for a better
answer. This isn't complete, but it helps a lot.

First, note that CPT-symmetry can be extended up to the macroscopic level.
Einstein's thought experiments about time involve macroscopic entanglement.
For example, given that a red ball and a blue ball are each in their own
boxes, and one possesses one box, then the box is entangled to have either a
red or blue ball inside. We can use linear logic to reason about these
situations; one holds the multiplicative disjunction of red and blue.

Second, note that in order to make MWI's numbers work, we need not just
branching of possibilities, but also coalescing. Imagine that, in empty space,
we suddenly have a single virtual-particle pair; the diagram would look like
() with time vertical. At the beginning, we have one universe; during the
split, two universes, one for each chirality of the pair; afterwards, one
universe. When we split, the probabilities of each possible universe decrease.
Now, imagine doing this with more pairs, and verify that the Binomial Theorem
manifests. This gives the same rise to macrostate-like entropy as Boltzmann's
approach to thermodynamics, but only using QM particles and QM logic; a QM
state is likelier when more of the many worlds in MWI would support its
existence.

Putting these together, we may conclude that Loschmidt's paradox is an
incomplete part of a bigger paradoxical complaint: We can see that there are
time-reversible _and_ time-irreversible behaviors, both in particles and in
macroscopic systems. The time-reversible behaviors seem to govern resources;
the time-irreversible behaviors seem to govern entropy. The two logics for
these two sets of behaviors are distinct; linear logic and intuitionistic
logic can intertwine, but here they don't appear to do so. Why not?

~~~
ethn
> I dislike the cybernetic approach because I find that it first commits to
> epistemic helplessness

The confusion here is that my statement has nothing to do with cybernetics.
Although it’s taken from his book titled Cybernetics, almost 60% of the book
isn’t about cybernetics but instead a philosophy and analysis of Science as it
progresses through history.

Though it’s fair to Wiener to preemptively note there is no failure in
epistemology in one of the most successful theories which holds ground in
neurons, to electrical circuits, traffic engineering, the abstract feedback
loop. There’s no assumption that “we know nothing of particles”. Wiener
originally wrote the book in 1948 and completed the second edition in 1961. He
made significant contributions to quantum mechanics[0]. It’s quite suspect to
make such a defamatory remark about Wiener’s cybernetics given his quantum
knowledge and the field as a whole.

In fact, what would cybernetics possibly be predicting about the interactions
between a particular system of which each component cannot exhibit a property?
Surely some property must be being reinforced by means of itself or another
auxiliary property.

QM & Cybernetics answer completely separate questions.

You’re missing the bigger picture. The irreversibility in any physical
phenomenon is only through scientific paradigms which are probabilistic—such
as thermodynamics, temperature, cloud, turbulence etc. These concepts are
simply more powerful in dynamical systems, which are always due to microscopic
phenomena, where there are no analytical, deterministic solutions for the
otherwise favored deterministic paradigms (e.g. three-body problem).

“while when a great number of atoms (of the order of Avogadro’s number) is
considered, this energy related to irreversibility becomes so large that its
order of magnitude must be taken into account. Consequently, macroscopic
irreversibility results related to microscopic irreversibility by flows of
photons and amount of atoms involved in the processes.”

[0]
[https://www.ams.org/books/psapm/052/psapm052-endmatter.pdf](https://www.ams.org/books/psapm/052/psapm052-endmatter.pdf)
[1]
[https://www.nature.com/articles/srep35796](https://www.nature.com/articles/srep35796)

------
sudoaza
Lame, only true question is will this universe die out and all information in
it be lost or can it be maintained indefinitely/transfer information to the
next universe iteration.

~~~
pwdisswordfish2
> only true question

you mean like, _the last_ question?

~~~
empiricus
I guess, the only question that matters. Unfortunately it does not matter for
us, we are too transient.

------
erostrate
Richard Feynman has a good explanation for 3.

[http://www.informationphilosopher.com/solutions/scientists/f...](http://www.informationphilosopher.com/solutions/scientists/feynman/past_and_future.html)

~~~
ericjang
Thanks! I had a chat with a mathematician friend today, and we came up with a
fairly simple resolution actually that I'm pretty satisfied with: the trick
here is to stop thinking of particles as T-symmetric billiard balls, and just
to assume that there exists microscopic T-asymmetric interactions.

If we assume molecular chaos hypothesis (independent particle velocities prior
to collision) and take into account _probabilistic_ collisions occurring at
the 2-particle microscopic level, then post-collision velocities are now
dependent, thereby imposing an ordering to time.

I have to think more about this though, and I'm still struggling with
understanding the circular dependency between entropy "being time" and entropy
being "caused" by "motion" (with respect to ... time?)

~~~
erostrate
Why can't you assume time symmetric billiard balls? If you take a lot of them,
black and white, put all the black ones on one side and the white on the
other, give them random velocities, and wait a bit, you will end up with them
all mixed together. The entropy has increased even though all interactions are
reversible. The macroscopic transformation is time asymmetric (the balls will
not sort themselves back again) even though the microscopic transformations
are time symmetric (elastic collisions and frictionless movement).

The way I think about this (from Feynman) is that entropy measures "how
special" a macroscopic state is, on average. When you apply many microscopic
independent fluctuations to a given state you are unlikely to end up with a
"more special" state. That's how states are ordered at the macroscopic level
giving the arrow of time a clear direction.

~~~
ericjang
_The macroscopic transformation is time asymmetric (the balls will not sort
themselves back again) even though the microscopic transformations are time
symmetric (elastic collisions and frictionless movement)._

I agree with your observation of what happens to the black and white
billiards. My question is - if the interactions are all time-symmetric
(implies conservation of entropy), where does the entropy come from in the
macroscopic system? My belief is that the microscopic model is flawed; if we
simply introduce some randomness into the collision dynamics of two particles
(which is believable given particle accelerator experiments / QM), then
microscopic interactions are no longer T-symmetric.

It comes down to a philosophical interpretation of whether you can treat a
system of two particles probabilistically (i.e. thermodynamically in
aggregate) or not.

------
Gatsky
Sometimes I wonder if we are the critters related to the second question. Of
course the next question is how far away are we from meeting the simulation’s
spec.

------
thaumasiotes
> If you don't know how to grasp an object, you can't bring food to your
> mouth.

Counterpoint: hummingbirds.

It might sound cheap, but discussions of the "basic necessities" of anything
so often just overlook obvious counterexamples.

~~~
ericjang
Hummingbirds actually proves my point here, on how life and death solve the
symbol grounding problem.

What is a "stable grasp"? An entire body of classical robotics literature is
devoted to answering this problem. I work on robotic grasping research and
defining what it means to grasp something successfully is a little tricky
unless you have a broader context of what the grasping is for.

Nature doesn't care whether hands exist or not - only that the animal can
feed. That is why we see such a spectrum of appendages in the animal kingdom.
Some are hand-like, others not at all, and some in between. In some sense,
"hand" is a mental construct of human language.

------
nojvek
Ask HN: what are the questions that keep you up at night ?

For me: How do viruses really work? How to antiviruses work? I’ve gone into
basics of DNA -> RNA -> Ribosome -> protein -> cell organelles -> Cells. We
know how to sequence DNA and it’s pretty cheap right now. How do we simulate
compiling DNA -> proteins and how they interact with each other? How do you
find weak spots in a virus and figure out what proteins will inactivate them ?

2) what is the best course of action for people who lost their jobs? Taking
<20k from 401k/IRA should be free of taxes and penalties. Everyone has lost
>30% of value since Feb already. Sure most people don’t have 401ks, but those
who do could weather this storm a bit better. Should we stop payments for all
loans for 2 months. No mortgage, auto loans, student loans etc. stop those
foreclosures from happening. No rent.

3) will US have the most deaths? Most likely. A large population + most states
haven’t locked down. This means unless >80% of population gets infected and
recovers to build herd immunity, or we have a vaccine, we’re in deep trouble.
Should we infect everyone >10 and under <30 and in clusters so we build up
herd immunity in weeks? And get most of the young population back to work?
Like if you’re gonna be sitting at home for two weeks, may as well get sick
and build up immunity while we’re at it.

------
jvm___
Do we already posess the hardware required to run full AI?

Deep fakes can be done on older hardware, we just didn't have the code. Does
the same thing apply to AI?

------
mberning
I can imagine many people would not be comfortable answering that question,
particularly depending on the company and their culture. I can easily see this
type of question leading to dramatically different conclusions ranging from
“this person is a deep thinker on varied topics” all the way to “this person
wouldn’t be a good cultural fit”. Don’t think too deeply about the wrong
topics.

------
keiferski
#2 is explored by John Searle's _Chinese Room_ argument, which is
unfortunately completely misunderstood by most technologists working in AI, to
their own peril.

This article is an excellent introduction:

[http://www.dreamsongs.com/Searle.html](http://www.dreamsongs.com/Searle.html)

A relevant excerpt:

 _Searle goes on to contend that intentional behavior must be in response to
real contingencies, not formal simulations of reality. If so, it would seem to
follow that the capacity for intentional behavior is the result of what might
be called phylogenic contingencies, and so would only be found in products of
real evolution. In short, what Searle is saying is that, just like other
biological activities such as digestion and photosynthesis, thought is
intrinsically dependent on the biochemistry of its origin. Just as a formal
simulation of digestion is not really digesting, a formal simulation of
thought is not really thinking. And to tie this to the gender thought
experiment, a formal simulation of gender is not gender, because it lacks
biological structural and functional characteristics._

The "Simulation, Duplication and Evolution" section of the SEP article is also
useful:

[https://plato.stanford.edu/entries/chinese-
room/#SimuDuplEvo...](https://plato.stanford.edu/entries/chinese-
room/#SimuDuplEvol)

~~~
erostrate
See Mike Adler answer to Searle. To use your digestion metaphor, as long as
the input is food and the output is energy and byproducts, AI researchers
don't care whether philosophers would call it "real digestion" or a "formal
simulation of digestion". Same goes for thinking.

[https://philosophynow.org/issues/46/Newtons_Flaming_Laser_Sw...](https://philosophynow.org/issues/46/Newtons_Flaming_Laser_Sword)

~~~
keiferski
This essay is filled with so many falsehoods, assumptions, and lack of
knowledge that I don't know where to begin. It's also written with a snarky,
arrogant attitude which is not helpful. To list a few:

\- "there is every reason to believe that a human brain is a machine"

Given what evidence?

\- Platonism died for mathematicians some centuries ago, and simply looks
silly.

And yet Frege and Gödel, two of the most accomplished mathematicians of the
past century, could be described as Platonists. Clearly this isn't a closed
question.

[https://en.wikipedia.org/wiki/Philosophy_of_mathematics#Plat...](https://en.wikipedia.org/wiki/Philosophy_of_mathematics#Platonism)

[https://plato.stanford.edu/entries/platonism-
mathematics/](https://plato.stanford.edu/entries/platonism-mathematics/)

Regarding digestion: if the Turing Test is determining whether a computer is
thinking like a human being, then focusing only on input and output, and not
the 'being', sort of misses the entire point. Remember that Searle's Chinese
Room is primarily a criticism of the Turing Test.

As always, the truth is: it's more complicated than it seems.

~~~
erostrate
I certainly agree that Adler's provocative tone is not helpful and his strokes
are sometimes too broad to be fully correct as you have rightfully pointed
out, but the general point stands.

To me the Turing test gives a falsifiable experiment that defines what
"thinking like a human being" means.

It does so by focusing only on the input output. To do that, it puts the
system under test out of view of the experimenter and mediates the experiment
with written text.

So yes, the input output is the important part, the Turing test, albeit
flawed, is interesting, and any claim that machines can't think based on a
definition of "think" that does not lend itself to a falsifiable experiment,
such as Searle's, is uninteresting.

I will leave it at that and make a quick exit, as Adler suggests.

------
Zenst
Q1: Can we imitate "thinking" from only observing behavior?

NO as you are measuring actions and not the process that instigated those
actions.

One example - chess - you see the move, but the thinking behind that move may
elude. More so a bad move, as their thinking was that it is a good move when
it is not.

[EDIT format/spelling]

------
6510
> Many AI researchers, myself included, believe that competitive survival of
> "living organisms" is the only true way to implement general intelligence.

My gut translates that idea as: Progress is made though War not Markets.

Which seems dubious to say the least.

~~~
andyjohnson0
Isn't a market a kind of competitive environment?

Given an appropriate reward function, and multiple independent actors seeking
advantage, some might converge on an optimal strategy for attaining AGI.

Anyway, that's the way I read it.

~~~
6510
Sure but exchange is a more important part of it. Even a predator offers
protection services.

------
intended
In this set up, the answer to q1- Intent: No, intent can’t be inferred.

If intent could be inferred- the meaningful and material portion of human
privacy would cease to exist.

------
Red_Leaves_Flyy
Tldr:

>Q1: Can we imitate "thinking" from only observing behavior?

>Q2: What is the computationally cheapest "organic building block" of an
Artificial Life simulation that could lead to human-level AGI?

>Q3: Loschmidt's Paradox: How does T-asymmetry in macroscopic phenomena arise
from T-symmetric (microscopic) physical laws?

------
RickJWagner
1\. What is your name?

2\. What is your quest?

3\. What is your favorite color?

Watch out for that last one.

~~~
noobly
>Watch out for that last one.

This must be a reference I’m not getting?

~~~
umvi
Monty Python and the Holy Grail

[https://youtu.be/0D7hFHfLEyk](https://youtu.be/0D7hFHfLEyk)

------
untog
I know it’s just the moment we’re living in right now but as I read those
questions I just shrug, they couldn’t feel less relevant.

The ones on my mind are “how long will large populations be locked down?” “How
many people are going to die?” and “how long until this all happens again?”

~~~
ceilingcorner
“The greats never sacrifice the important for the urgent. They handle the
immediate problem and still make sure to secure the future.”

The media is incentivized to distract you. Turn off your computer, read a
book, and don’t forget to live.

~~~
untog
What an absurd statement. This crisis matters. Social distancing matters.
Lockdowns matter. “Stick your fingers in your ears” is awful advice.

~~~
m-p-3
But at some point, the medias are rehashing the same statements: Stay home,
and follows good practices.

As an individual, there's only so much you can do and there's no need to keep
yourself distracted from other matters.

~~~
untog
The advice around wearing face masks has changed in the last 48 hours.

You can stay tuned in to the world around you without collapsing on the ground
in hysterics.

