
CompCert: A formally verified optimizing C compiler - adamnemecek
http://www.absint.com/compcert/
======
Tomte
Although there seem to have been parts of it that hadn't been formally
verified:

"Using Csmith, we found previously unknown bugs in unproved parts of
CompCert—bugs that cause this compiler to silently produce incorrect code."
([http://www.cs.utah.edu/~regehr/papers/pldi11-preprint.pdf](http://www.cs.utah.edu/~regehr/papers/pldi11-preprint.pdf))

The CompCert homepage tells us that the same authors did a study in the same
year (2011) without finding any wrong-code errors.

But even if they hadn't; I'd still be excited about CompCert demonstrating
that this level of verification is indeed possible. Because then verifying the
unverified parts would probably be just as feasible, given enough resources.

~~~
jbapple
The Csmith paper notes two types of errors found in CompCert: front-end errors
and a misunderstanding of PPC semantics.

The former type of bug was apparently due to a part of the compiler that
wasn't verified at the time. According to the Csmith paper, " This bug and
five others like it were in CompCert’s unverified front-end code. Partly in
response to these bug reports, the main CompCert developer expanded the
verified portion of CompCert to include C’s integer promotions and other
tricky implicit casts."

Additionally, the CompCert team produced a validated parser which they
discussed in their ESOP 2012 paper, "Validating LR(1) Parsers"

The second type of bug (misunderstanding the semantics of the target
architecture) is, as far as I know, impossible to avoid. Any verification
project can end up with a true proof of the wrong theorem!

~~~
DanWaterworth
> The second type of bug (misunderstanding the semantics of the target
> architecture) is, as far as I know, impossible to avoid. Any verification
> project can end up with a true proof of the wrong theorem!

Indeed. However, although it's impossible to avoid, you can still take steps
to make it less likely.

When you formally verify code, you are showing that a program has a particular
specification.

If you want to eliminate errors in your specification, you should aim to make
it as simple as possible, so that people with domain knowledge (in this case,
people with a correct understanding of PPC) would be able to tell whether your
specification is correct.

------
jbapple
See also CakeML, "A verified implementation of a significant subset of
Standard ML in the HOL4 theorem prover".

[https://cakeml.org/](https://cakeml.org/)

Not that this project, unlike CompCert, is Free Software in the Stallman
sense:

[https://github.com/CakeML/cakeml/blob/master/COPYING](https://github.com/CakeML/cakeml/blob/master/COPYING)

------
nathell
The fact that Xavier Leroy has been working on CompCert gives it an extra
level of credibility for me.

(Xavier Leroy of INRIA is the main author of OCaml, among other things.)

~~~
peterfirefly
And the original Linux threading support.

His report on the Zinc Experiment is well worth reading:

[http://gallium.inria.fr/~xleroy/bibrefs/Leroy-
ZINC.html](http://gallium.inria.fr/~xleroy/bibrefs/Leroy-ZINC.html)

------
khaki54
So is this protected from the "Ken Thompson Hack"?
[http://c2.com/cgi/wiki?TheKenThompsonHack](http://c2.com/cgi/wiki?TheKenThompsonHack)

I guess it would have to be cross compiled for your system otherwise your
existing compiler could corrupt it...

~~~
vilhelm_s
It's not self-hosted, so it is not yet at the point where the Thompson hack
would apply.

That is, CompCert is a C compiler, but it's not written in C, it's written in
Coq. So when you use it, you have to trust that the Coq implementation was
written correctly, and that nobody has inserted any backdoors in Coq which
insert backdoors in Compcert which insert backdoors in your program.

The next level would be to also have a verified Coq compiler, which would make
the whole system that much more trustworthy. And at _that_ point you could
worry about self-perpetuating backdoors in that compiler, and ways to mitigate
them.

~~~
more_original
Oh, but they're already building a verified Coq compiler:
[http://www.cs.princeton.edu/~appel/certicoq/](http://www.cs.princeton.edu/~appel/certicoq/)

------
ngrilly
This is probably a Philistine question, but how do we make sure the proof --
that CompCert works as intended -- itself is correct? In mathematics and
science, it happens all the time that someone finds a mistake in a proof years
later after its first publication.

~~~
arcatek
In science it may happen because everything is based on a lot of assumptions,
themselves based on our physical perception of the world, which may be proved
wrong later in time.

Are mathematics subject to this kind of flaws ? There is a very restricted set
of axioms, and almost everything else is based on formal proofs.

~~~
one-more-minute
See, for example:

[https://en.wikipedia.org/wiki/Wiles%27_proof_of_Fermat%27s_L...](https://en.wikipedia.org/wiki/Wiles%27_proof_of_Fermat%27s_Last_Theorem)

[http://rsta.royalsocietypublishing.org/content/roypta/363/18...](http://rsta.royalsocietypublishing.org/content/roypta/363/1835/2401.full.pdf)

~~~
lmm
Wiles' proof was not written in machine-checkable language; if it had been,
the error would almost certainly have been caught.

~~~
ngrilly
But writing Wile's proof in a machine-checkable language wouldn't have been a
problem in itself?

There is no "universal" machine-checkable language which can be used for any
proof. I don't know if Wiles' proof could have been written in Coq, for
example. Let's say it can't, and that no existing machine-checkable language
can be used for this purpose. In such a case, you would have to develop a new
proof language, and make sure the proof checker itself is correct.

I guess that at some point we are hitting the theoretical limits imposed by
the halting problem and Gödel's incompleteness theorems?

It's not my field so I'm just asking questions here. Don't hesitate to lecture
me :-)

~~~
thoran
No, you're wrong. This is not a theoretical issue there, but a practical one.

Almost all mathematics (and this certainly includes Wile's proof) could be
written in Coq, in theory. It is extremely hard to do in practice.

Mathematicians write proof for their peers who have a smart brain. Most of the
trivial and less than trivial details are omitted. Coq cannot not accept this
(because he is very stupid and can't figure the missing steps). It turns out
that it is particularly hard and boring to fill the missing holes in a "human"
proof.

Gödel comes in when you try to prove Coq's correctness within Coq (but this
was partially done, in a sense)

~~~
ngrilly
I want to make sure I understand correctly. :)

I am right on the fact that rewriting Wile's proof in a machine-checkable
language is a problem in itself? But I'm wrong when I suggest it is
theoretical issue, when it's actually practical issue?

I've read that "the theory behind Coq is generally admitted to be consistent
with regard to Zermelo-Fraenkel set theory + inaccessible cardinals". [1] I
also read that some statements are undecidable in Zermelo–Fraenkel set theory
[2]. It follows from this that it must exist some statements that are
undecidable in Coq, correct? But I understand that this is theoretical issue,
not a practical one, because "Zermelo–Fraenkel set theory, combined with
first-order logic, gives a satisfactory and generally accepted formalism for
essentially all current mathematics". Is it the reason why my comment was
wrong?

On this topic, I've read an article titled "Computer verification of Wiles'
proof of Fermat's Last Theorem" which explains that Wile's proof could
probably be verified with a tool like Coq, but that would be a massive
challenge:

> The mathematics used in Wiles' proof of Fermat's Last Theorem is very
> complicated. [...] I do not know the provers Coq and Mizar good enough, but
> I think they are adequate to express the mathematics.

> On the other hand, I do think that the challenge is doable, and that it will
> be done in the coming fifty years. It is definitely not within reach. The
> project will have to cover large parts of current pure mathematics. Its
> scope is comparable with the Bourbaki project, where between 1950 and 1970
> almost all of pure mathematics got a new foundation by the efforts of an
> originally French group of mathematicians who published some 20 books under
> the pseudonym N. Bourbaki.

[1] [https://coq.inria.fr/faq](https://coq.inria.fr/faq) [2]
[http://en.wikipedia.org/wiki/Zermelo%E2%80%93Fraenkel_set_th...](http://en.wikipedia.org/wiki/Zermelo%E2%80%93Fraenkel_set_theory)
[3]
[http://en.wikipedia.org/wiki/Hilbert%27s_program](http://en.wikipedia.org/wiki/Hilbert%27s_program)
[4]
[http://www.cs.rug.nl/~wim/fermat/wilesEnglish.html](http://www.cs.rug.nl/~wim/fermat/wilesEnglish.html)

~~~
thoran
It's my fault, I think I read your answer too quickly and miss a negation
somewhere.

So, yes, a formal rewriting of Wiles' proof is "only" a practical challenge.
But a big one. Just about 15 years ago, we were just able to prove the
Fundamental theorem of algebra in Coq (a 200 years old theorem that is
routinely taught to undergraduates).

~~~
ngrilly
That's okay @thoran! Thanks a lot for having helped me gaining a better
understanding of this topic.

------
Keyframe
Note that it can't be used commercially without a proper license. Also, no 64
bit.

~~~
jk4930
Two thoughts here:

1\. If you use it commercially, you can pay for it.

2\. In most cases where it's needed, 64 bit doesn't matter.

(Though I might be wrong on the second one.)

~~~
Keyframe
1\. Nobody argued that.

2\. What do you mean it doesn't matter? You think embedded and similar systems
don't need/use 64-bit space? True for most I guess.

------
soup10
"CompCert typically runs twice as fast as the code generated by GCC without
optimizations." Lol, That's like saying you run faster than Usain Bolt when
he's walking.

~~~
sanxiyn
Since GCC optimizations can miscompile, target audience ("safety-critical and
mission-critical software written in C and meeting high levels of assurance",
aka Airbus) is currently using GCC without optimizations. So it is a fair
comparison for that case, since they can now enable proven-correct
optimizations.

~~~
craftit
Interesting they are using it without optimisations. The compilers is used far
more with optimisations, so those code paths are actually far better tested. I
have heard of cases where bugs only appear in unoptimised code.

~~~
andrewchambers
I don't think this is really true. Generally all optimization passes are run
upon the unoptimized version, hence ALL code paths first pass through the -O0
machinery.

Besides, they seem to manually inspect the assembly output themselves for the
final production build.

~~~
craftit
But the code path is first pass + optimisation gives correct code. That
doesn't guarantee the first pass generates correct code. I believe there were
bugs along those lines at one stage, though I admit they are rare. I've not
had to track down a compiler bug in quite a few years.

------
whitten
Does anyone know if CompCert can be used to formally verify a commpiler of
some other language than C, such as Assembly Language or Python ?

------
venomsnake
And who verifies the verify-er? (That is generally the problem I have with the
concept of unit tests too)

~~~
vilhelm_s
Ideally, a simpler and therefore more trustworthy verifier. So far, the most
fleshed out implementation of this idea is Milawa, which developed a chain of
11 increasingly sophisticated verifiers, each one checked by the previous.

[http://www.cs.utexas.edu/users/jared/milawa/Web/](http://www.cs.utexas.edu/users/jared/milawa/Web/)

