
Incompleteness ex Machina – proving Gödel's theorems in terms of algorithms [pdf] - lisper
https://www.scottaaronson.com/incompleteness.pdf
======
NotAnEconomist
> Godel was programming the integers. That is that. And he didn’t even know it
> at the time; truly impressive.

It seems strange to say that Godel didn't know it at the time -- Godel's work
was part of the effort to mechanize logic, Hilbert's program, and the
axiomatization effort started in response to contradictory calculus theorems,
and followed Frege, Russel, Whitehead, et al.

Or rather, Godel's work showed that Russel's efforts to create a consistent
and complete foundation for mathematics was fundamentally insufficient. Turing
extended this work, by providing an explicit model of a calculating machine,
to show that no algorithm could effectively determine the truth -- even
without providing a proof -- locking the door on Hilbert's program that Godel
had slammed shut.

Regardless, the entire point of Godel's work was exploring the relationship
between our ability to "reason" and our ability "calculate" or "perform rote
tasks".

(Missing details of the narrative aside -- it was a really good read.)

~~~
martinlaz
Any chance OP is referring to "integer programming" (aka discrete
optimization), which came about 20 years later?

~~~
atq2119
No. Integer programming in the operations research sense is "just" solving
linear optimization problems with the constraint that the solution variables
need to be integer.

In particular, IP doesn't really involve the multiplicative structure of the
integers, which is crucial for how Gödel proved his results.

Also, I'd point out that while there's significant overlap between integer
programming and discrete optimization, they're not the same thing. Integer
programming is one tool used in discrete optimization among many.

------
soberhoff
Author here.

I'd like to mention that this article is currently slated for publication in
the Bulletin of the European Association for Theoretical Computer Science. A
revised draft can be found at:
[https://github.com/SOberhoff/incompleteness_ex_machina/relea...](https://github.com/SOberhoff/incompleteness_ex_machina/releases/download/v1.3/incompleteness_ex_machina.pdf)

I haven't handed it over yet, so we'll have to see about further changes.

~~~
triska
That's a very nice writeup, thank you a lot!

One thing that seemed a bit surprising to me, especially in light of the
paper's strong focus on computation, is that the assumption that the
axiomatization is _effective_ is only mentioned in a footnote, and even there
it is not explained but we are told to "don't worry about it" if we don't know
what that means.

Yet, the code pieces in the paper make critical use of this assumption in that
they assume that we can decide (with an algorithm) whether or not a string is
a proof. This only works if the axioms are recursive.

So, maybe you could consider adding a short explanation about this to the
paper?

~~~
soberhoff
Every time I said "either/or" I was making use of the law of the excluded
middle. But I didn't add a detailed disclaimer explicitly highlighting this
fact either. Should I have? I think it's okay to just take some things for
granted.

Though, I admit that I have never spent much thought on non-effective
axiomatizations. So I'm open to be educated.

~~~
triska
The proofs you present are entirely constructive and work without appealing to
the law of the excluded middle! As was also pointed out by Gödel himself about
his proofs, they are obtained in an "intuitionistically unobjectionable
manner".

When you say "either/or" in the paper, you make case distinctions that are
intuitionistically non-contentious, i.e., you say " _either_ F ⊢ G, then
contradiction, so F ⊬ G, _or_ F ⊬ G, then contradiction etc.", but the point
is that G is explicitly constructed, so this is intuitionistically acceptable.

In contrast, the other assumption I mentioned _is_ used in the paper and in
fact essential for its proofs.

~~~
soberhoff
I agree in the cases where I argue "suppose, then..." But there are instances
where I argue "either this is the case, or this is not the case." Unless there
are multiple laws of the excluded middle, that's an application of it.

In any case, that was merely an example. The central point I was making
doesn't depend on this.

~~~
triska
In the paper, it is assumed — per footnote 1 — that the formal systems under
discussion are effectively axiomatized. I assume this means that their
theorems are _recursively enumerable_ , is this correct? If so, then this
means that determining whether a string is a proof may only be semi-decidable,
and may not halt for strings that are _not_ proofs.

However, as I mentioned, in the remainder of the paper, it appears to me that
you assume that the theorems are not only recursively enumerable, but in fact
_recursive_. For example, in the proof of Theorem 4 on page 5, the lines
stating "if s is a proof ..." seem to assume that this check always
terminates, requiring assumptions that the original proof does not and
therefore weakening the obtained result.

In my opinion, adding a short explanation about how this can be salvaged (it
can!), or at least precisely specifying what is assumed, could improve the
presentation.

~~~
soberhoff
Sorry, I was out of town for a while.

I'm only assuming that checking whether "does s prove S?" is a recursive
property. That's not the same as demanding "is S provable?" to be recursive.

Upon reflection I agree that effective axiomatization isn't actually that
difficult to explain. So I've changed the footnote to say: "For the purpose of
this discussion every formal system is effectively axiomatized by definition.
This basically just boils down to the fact that proofs are computer
checkable."

I was initially worried that this might raise the question what non-effective
axiomatizations are all about. That's not a can of worms I want to open. But I
think this should be fine.

~~~
triska
Thank you for adding this! Still, it is not so straight-forward to conclude,
from just the premise that the system is effectively axiomatized (i.e., its
theorems are recursively enumerable), that "does s prove S" is a recursive
property.

In my opinion, if the assumption is that "proofs are computer checkable"
(i.e., "does s prove S" is a recursive property, which is what is required in
the paper), then it would be good to either state _that_ as the key
assumption, or state for example that this follows via Craig's theorem from
the assumption that the system is effectively axiomatized:

[https://en.wikipedia.org/wiki/Craig%27s_theorem](https://en.wikipedia.org/wiki/Craig%27s_theorem)

Interestingly, from this theorem it follows that such a system can be re-
axiomatized so that "S is a theorem" is even a _primitive_ recursive property.
In fact, when Gödel wrote "rekursiv" around 1930, he meant the class of
functions that we now call _primitive recursive_. Because of Craig's theorem,
we know that primitive recursive, recursive, and recursively enumerable can be
used interchangeably in the definition of effective axiomatization, but
without such a theorem, it would be a leap to conclude one from the other.

~~~
soberhoff
I don't see why there's a theorem required here. Let our formal system be
effectively axiomatized. Then by definition its axioms are a recursive set. So
I can just check a proof by starting at the top and verifying that every line
is either an axiom or follows from a line above via an inference rule. Both of
these two checks are recursive. Hence, proof checking is recursive.

Also, the Wikipedia article doesn't say that Craig's Theorem proves
recursively enumerable axiomatizations equivalent to (primitive) recursive
axiomatizations. I would find it very surprising if this was a consequence. It
only says that a recursively enumerable set of formulas (e.g. the provable
sentences in Peano arithmetic, not merely its axioms) can be given a
(primitive) recursive axiomatization. That's a very much weaker claim.

~~~
triska
_" Let our formal system be effectively axiomatized. Then by definition its
axioms are a recursive set."_

"effectively axiomatized" means that the _theorems_ are recursively
enumerable. It does not immediately follow that the _axioms_ are a recursive
set. However, by Craig's theorem, we can then re-axiomatize so that the axioms
are a recursive set, even a primitive recursive set!

You are of course right about the consequences: In the post above, I meant to
say that due to this theorem, it makes no difference whether we require the
axioms to be recursively enumerable, recursive, or primitive recursive. But
still, without this theorem, it would be a leap to conclude from "the system
is effectively axiomatized" that "s is a proof of S" is recursive.

~~~
soberhoff
I don't agree with that definition of "effectively axiomatized". Granted,
that's what Wikipedia says here:
[https://en.wikipedia.org/wiki/G%C3%B6del%27s_incompleteness_...](https://en.wikipedia.org/wiki/G%C3%B6del%27s_incompleteness_theorems#Effective_axiomatization)
But the reference given doesn't check out. Franzén's 2005 book _Gödel 's
Theorem_ doesn't say anything about effective axiomatization on page 112. He
does however give the following definition in his lecture notes at:
[https://books.google.de/books?id=Wr51DwAAQBAJ&pg=PA138#v=one...](https://books.google.de/books?id=Wr51DwAAQBAJ&pg=PA138#v=onepage&q&f=false)

"If T is _effectively axiomatized_ , in the sense that there is an algorithm
for deciding whether or not a given sentence is an axiom of T,..."

In _Computability: Turing, Gödel, Church and Beyond_ Martin Davis even makes
the definition "T is _axiomatizable_ if it has an axiom set that is
computable" on page 42.

And that's also the definition Shoenfield gives on page 125 of _Mathematical
Logic_.

Besides, "effective" has always been a synonym for "computable", "recursive",
or "decidable" whenever I've encountered it in this context.

Also, I must insist that "it makes no difference whether we require the axioms
to be recursively enumerable, recursive, or primitive recursive." is not what
Craig's Theorem says. This is dangerously close to claiming that recursively
enumerable sets are recursive which is plainly untrue. If recursively
enumerable _axioms_ (not just recursively enumerable _theorems_ ) are indeed
interchangeable with recursive axioms, then I'd like to hear more about that.

~~~
triska
Yes, you are right: This is not what Craig's theorem says! It is enough to
assume that the _theorems_ are recursively enumerable. From this, it follows
that there is an axiomatization where the axioms are recursive.

The key point is that I would find it a worthwhile addition to the paper to
somehow make clear that "s proves S" is assumed to be decidable. Since you
have done this: Thank you!

------
imh
I never understood the step about how a system that can do basic arithmetic
can express the "I am not provable in F" sentence. Does anyone have an ELI30
version of that?

~~~
neokantian
It is not about systems that can "do" basic arithmetic but that can "talk
about" basic arithmetic.

The mere execution of basic arithmetic does not require the capability of
manipulating propositions of basic arithmetic.

Doing basic arithmetic:

12 * ( 5 + 8 ) --> 12 * 13 --> 156

Talking about basic arithmetic:

a * ( b + c ) == a * b + a * c

The formal language needed to describe basic arithmetic is much more powerful
than a simple execution engine, such as a stack engine, that can merely carry
out basic arithmetic.

For example, I must be able to express that for all natural numbers x and y,
if x = y, then y = x. The result could look like this:

∀ x,y ∈ N: x=y ⇔ y=x

Hence, this language must be able to express all Dedekind-Peano's axioms as
well as every proposition provable from these axioms, along with their formal
proofs.

In that sense, the Gödel numbering system is the "bytecode" of quite a serious
programming (or at least, specification) language.
([https://en.wikipedia.org/wiki/G%C3%B6del_numbering](https://en.wikipedia.org/wiki/G%C3%B6del_numbering))

~~~
rocqua
It should be noted that the special sauce that makes arithmetic really
difficult is induction.

Induction isn't just reasoning about computation, (i.e simple equations).
Instead it is reasoning about reasoning about equations (i.e. reasoning about
all equations).

Specifically we have: this formula
[https://wikimedia.org/api/rest_v1/media/math/render/svg/67e2...](https://wikimedia.org/api/rest_v1/media/math/render/svg/67e20a414c6f0eb6af309ddbe2bcee1391b3d0ea)

~~~
salthound
But it should be pointed out that induction is essentially unrelated to
Gödel's theorem.

For example, Robinson Arithmetic is a finitely axiomatized theory whose axioms
contain only one existential quantifier, but just like full (Peano)
arithmetic, it is subject to Gödel's incompleteness theorems.

~~~
rocqua
Wow, I thought it was required to have second order logic for godel to apply,
but this seems to be a first order logic that is still undecidable.

~~~
ratmice
From the other side of the fence, another thing to look at would be the
decidability of presburger, and skolem arithmetic, the following paper then
goes over the extensions of these, which render the result undecidable.
[http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.2.21...](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.2.2136)

