
The C standard formalized in Coq - janvdberg
http://robbertkrebbers.nl/thesis.html
======
sanxiyn
Note that this formalization is substantially incomplete: check page 195. CH2O
does not formalize variadic function, float, bit field, setjmp/longjmp. All of
them are formalized in KCC.

[https://github.com/kframework/c-semantics](https://github.com/kframework/c-semantics)

~~~
redsymbol
That's true. The immediate-term impact is probably minimal, for the reasons
you mention. However, this is a _major_ accomplishment. As this work is built
upon, its longer-term impact has the potential to be very significant.

~~~
sanxiyn
I agree it is a major accomplishment. I have doubts about its impact, because
I am unsure whether it will be built upon.

CH2O's main innovation is C standard (as opposed to C implementation)
formalized in general purpose prover (as opposed to custom system). KCC
formalized C standard, but not in general purpose prover. CompCert formalized
in general purpose prover, but formalized implementation instead of standard.

Formalizing in general purpose prover is useful because you can use all of
math to prove runtime invariants. But if you want to prove runtime invariants,
you usually want to prove with respect to concrete implementation, not to
prove abstractly to standard. For time when you want to prove abstractly to
standard, this doesn't seem better than KCC.

In short, this feels like Not Invented Here against KCC. Probably unfair
characterization, but that's what it feels to me.

~~~
nickpsecurity
So, you say they did a KCC-like thing in a theorem prover that could do more
than KCC. Then, that it's just NIH syndrome against KCC? Seems contradictory.
If anything, one would just have to tweak it on a per compiler basis to get it
more toward how each implements that behavior. Then it becomes closer to what
CompCert did but more flexible.

That's where I'd go. Extending KCC to do what a theorem prover could seems
more challenging.

~~~
conceit
That seems to be what he says, but after the first part he disregards that as
not useful. Another comment points out following the standard is good for
portabilities sake. Somehow, that all seems obvious to me.

~~~
nickpsecurity
That was the build up. It led to the following statement and conclusion:

"For time when you want to prove abstractly to standard, this doesn't seem
better than KCC. In short, this feels like Not Invented Here against KCC.
Probably unfair characterization, but that's what it feels to me."

The obvious interpretation is commenter thinking KCC is better and (see NIH)
the OP should've maybe done something with it instead...?

" Another comment points out following the standard is good for portabilities
sake. Somehow, that all seems obvious to me."

You can follow the standard and be portable if you keep certain parts as
options subject to interpretation/choice of the implementers. That's the so-
called undefined behavior mostly.

------
alexbock
One issue with this is that the text of the C standard isn't actually enough
to unambiguously determine what the committee (and compiler vendors) intend
for the language to mean in practice.

The official standards mailing lists for C and C++ are constantly filled with
issues regarding inconsistencies in the ISO standards, and for C++ at least
people often resort to trying a code sample in multiple major compilers to
decide what it's "supposed to do" when the standard is unclear.

It might be a bit better with C, but I once tried to start implementing a toy
C++ compiler by following the text of the standard literally and it became
impossible pretty quickly. There are so many pieces where you are just
expected to know what they intend and what inconsistencies can be ignored
because you're already familiar with the language that it would be difficult
to call any one person's interpretation a formalization of the standard
itself.

~~~
thinkpad20
> the text of the C standard isn't actually enough to unambiguously determine
> what the committee (and compiler vendors) intend for the language to mean in
> practice.

Interesting! If the standard is under-specified, then presumably this student
must have had to fill in some of the holes themselves, or found a way to
represent multiple interpretations simultaneously.

> issues regarding inconsistencies in the ISO standards

If the standards are inconsistent (in a logical sense), and this formalization
is sufficiently complete, it should uncover those inconsistencies since they
would amount to type errors in Coq. I wonder if the author had any of those
difficulties.

> trying a code sample in multiple major compilers to decide what it's
> "supposed to do" when the standard is unclear

One of the effects a project like this could have is to make this (at least
theoretically) unnecessary to do.

> implementing a toy C++ compiler by following the text of the standard
> literally and it became impossible pretty quickly

Fortunately C is a much smaller and simpler language than C++. Isn't the C++
standard almost 1000 pages? I'm not sure what the C standard is but it must be
quite a bit smaller.

Regardless, it's a PhD thesis so the difficulty of the task isn't a reason not
to attempt it :)

~~~
nly
> Fortunately C is a much smaller and simpler language than C++. Isn't the C++
> standard almost 1000 pages?

The language specification itself is only about ~450 pages, the rest is
library. It of course inherits all of the insanity of C on top of that (~170
pages of C language spec), which I'd argue brings a disproportionately large
share of the pain.

Alex's point is that the standard doesn't really begin to give you an idea how
hard it is to write an implementation, or make decisions about everything
that's unspecified or ambiguous.

~~~
_kst_
The C standard doesn't exactly "inherit" the 170 pages of the C language spec.
As far as I know, the core C++ language (excluding the library) is defined
entirely by the C++ standard, without reference to the C standard. It was
certainly based on C, but the description doesn't depend on it.

The C standard _library_ is incorporated by reference (with a few tweaks) into
the C++ standard. Of course the C++ standard library also includes a lot of
material that isn't in the C standard library.

------
geon
Heh. CH2O = _formal_ dehyde

------
pron
I would strongly encourage anyone interested in formal methods (and formal
proofs in particular) but may be intimidated by dependent types, which serve
as the basis for Coq (and are foreign not only to software developers but to
most mathematicians as well) to give TLA+[1] a look. It is a decade younger
than Coq (16 vs. 26), but simpler to use, much more gradual, and has had
industry adoption pretty much since its inception.

Instead of dependent types (in fact, TLA+ is untyped[2]) It is based on simple
ZF set theory and logic, and should be immediately familiar to anyone who's
taken even one logic (or discrete math) course in college. In addition to a
proof assistant it has a good model checker, comes bundled in a nice
(relatively speaking) IDE, and is easy to learn (with a thorough tutorial
written by Leslie Lamport himself).

TLA+ is also more widely used in the industry -- both hardware and software --
perhaps most notably in the software industry by Amazon, who use TLA+ to
specify many of their AWS services. TLA+ is a formal specification (and
verification) tool designed for engineers and used by engineers. Although, to
be fair, neither tool is used by a large portion of the industry.

EDIT: Removed potentially incendiary Lamport quote.

EDIT: Changed an incorrect statement about Coq's use in the industry

[1]: [http://research.microsoft.com/en-
us/um/people/lamport/tla/tl...](http://research.microsoft.com/en-
us/um/people/lamport/tla/tla.html)

[2]: See _Should Your Specification Language Be Typed?_ by Lamport and
Paulson: [http://research.microsoft.com/en-
us/um/people/lamport/pubs/l...](http://research.microsoft.com/en-
us/um/people/lamport/pubs/lamport-types.pdf)

~~~
nmrm2
A note for the unitiated: There is a tension between TLA+ and Coq et al.
(typified by statements such as "weird computer-science math") that is similar
and maybe even rooted in to the sorts of culture wars that sprout up around
programming languages (see [2] of parent's post for a perfect example of what
I'm talking about...)

The characterization in pron's post is decidedly biased toward TLA+ in a way
that I think is unnecessary. A few correctives:

* Coq has seen plenty of industry adoption. To say that type-theoretic theorem provers (or the Coq system specifically) are merely academic toys compared to TLA+ is wrong.

* "Normal" mathematicians (not just "weird Computer Science-type" mathematicians, whatever that means...) have used and extolled the virtues of Coq as a tool as well as its theoretical foundations. See e.g., the univalent foundations project. Regardless, I'm not sure I understand why "Computer Science-type math" is an unreasonable foundations for a _program specification tool_ , which is what TLA+ is. And if the goal is formalized mathematics, you're probably better off with Coq (and it's not even clear that the TLA+ community is interested in that sort of thing, independnet of results useful in the course of a program specification/verification task.)

* If you're familiar with (esp. typed) functional programming, you might find Coq even easier to get into than TLA+. On both sides of the type theory vs sets/axioms divide, people should be careful not to confuse one's predispositions based upon educational background with the actual mental load of learning to use a tool.

There is room for both of these tools (and more!) in the world. I don't
understand why they have to be in competition. A formalization of the C
standard is a HUGE undertaking, independent of what tool was used. Let's
celebrate that achievement!

~~~
pron
BTW, what is the justification for using dependent type rather than stating
assumptions/theorems directly in predicate logic over set theory (as in TLA+)?

Lamport has this to say on the matter:

 _The main virtue of these type theories is precisely that they are
constructive. A constructive proof that two arbitrary numbers always have a
greatest common divisor provides an algorithm for computing it [Thompson
1991]. Researchers, using tools such as Coq and Nuprl are investigating
whether this can lead to a practical method of synthesizing programs._

 _You can perform classical reasoning in a constructive type theory by adding
P ∨ ¬P as an axiom. The resulting system will probably be strong enough to
handle any specification problem likely to arise. However, it will be no
stronger than ZF, and it will be much more cumbersome to use._

In the end, he opted for set theory because it is simpler, more familiar, and
more flexible. Are there any advantages to using dependent types? Is the main
advantage indeed synthesizing programs from proofs?

~~~
nmrm2
_> Is the main advantage indeed synthesizing programs from proofs?_

It's one major advantage, yes. And not just for the sake of software
verification. Sometimes when proving a non-CS-type-math theorem you still want
an efficient implementation of an existence proof.

But code synthesis is a lot harder than merely being constructive, and
efficient code synthesis even more so. Use cases for Coq that depend upon code
synthesis don't come for free just by avoiding LEM. So although it's fair to
say that you can do classical reasoning in Coq, I'm not sure saying you can do
synthesis from TLA+ if only you avoid LEM is fair at the moment (I might be
wrong).

 _> Are there any advantages to using dependent types?_

Here are a few other reasons for using Coq:

* Proof terms. There are two major disagreements here. The first grounds out in philosophy and the second basically amounts to "you really want to compose the proofs and make sure they are still a proof instead of just throwing away the proof and using the theorem" which has lots of good systems-y justifications

* Specifically wrt TLA+, Modus Operandi matters. If you want a formal proof in the typical sense of the word (rather than a verification of fact based upon model checker output), then Coq userlands's tactics and theorems are useful.

* Lots of people like to disparage the amount of algebra that FPers talk about. But it turns out that if studying those algebraic structures is your day job, the relationships between types and algebraic structures makes type theory convienant.

 _> simpler, more familiar_

This is almost definitionally subjective. Which isn't meant as a criticism,
in-so-far as one realizes that other people might have different experiences
and so might (just as validly) consider other formalisms more familiar.

 _> and more flexible_

There are (obviously) a lot of people who fundamentally disagree with the
concrete arguments Lamport makes in this paper. I wish someone would annotate
the paper with references to literature from the type theory community that
address some of his concerns.

For the record, I don't disagree with Lamport. I think Lamport is _right_ \--
for some use cases, it certainly makes sense to stick with ZF. But he's also
_wrong_ \-- sometimes, the type theoretic approach really is simpler, more
familiar, and more flexible. E.g., in the case of this dissertation.

~~~
pron
> If you want a formal proof in the typical sense of the word (rather than a
> verification of fact based upon model checker output), then Coq userlands's
> tactics and theorems are useful.

TLA+ has them, too.

> in-so-far as one realizes that other people might have different experiences
> and so might (just as validly) consider other formalisms more familiar.

So you think some people are more familiar with type theory than with
undergrad-level set theory (ZF)?

> sometimes, the type theoretic approach really is simpler, more familiar, and
> more flexible. E.g., in the case of this dissertation

I don't see it (probably because my type theory is very, very basic; I'm an
algorithm's guy and the thought that a data-structure of mine might need type
theory to be specified makes me shudder), as in I fail to see anything that
isn't readily expressible in simple classical logic + temporal logic. Could
you explain?

~~~
nmrm2
_> TLA+ has them, too._

Like I said, modus operandi matters. If I have to perform another model
checking routine (possibly on an infinite state space) to slightly generalize
a theorem, then in many cases that means I never really had a useful proof of
it in the first place.

Also, Coq's mathematics library is extensive and often closely follows the
proofs found in graduate mathematics text books. Last I looked into TLA+, it
was squarely focused on algorithms and hardware (not even CS-related
mathematics more generally -- specifically algorithms and hardware).

 _> So you think some people are more familiar with type theory than with
undergrad-level set theory (ZF)?_

Do I think there exists some population of people who are "more familiar" with
type theory than with ZF? Yes, starting, for example, with every researcher
whose primary area of study is type theory. In fact, I think that anyone who
doesn't honestly believe there are such people must have an extremely low
opinion of quite a number of very smart people.

(Also, set theory qua set theory gets really disgusting and kludgey really
quickly, and is something that almost no undergraduate (or even phd student)
is exposed to these days. The "set theory" you see in a discrete math course
kind of barely scrapes definitions. I think most people who posit that "set
theory" is nice and simple must've never really dove deem into the actual
theory of set theory...)

(Also, most real math these days quickly abstracts away from germanely set
theoretic constructions. Mathematicians don't work in ZF. They do not. Most
will _tell you_ they do not, if they even know what ZF is. Which a lot won't,
because their undergrad discrete math course is probably the last time they
saw a truly formal derivation. And the mathematicians who do know what ZF is
often can't regurgitate an axiomatization of ZF. If you tell a mathematician
that all of math is just set theory, most will chuckle and wink and say "why,
yes, yes it is", and hopefully you're in on the joke...)

 _> I don't see it... I fail to see anything that isn't readily expressible in
simple classical logic + temporal logic. Could you explain?_

Did you read this dissertation's explanation of why separation logic is
important?

To the extent that this observation is remotely reasonable, it's only in the
sense that you can code up one logic in another. Which, well, all the world's
a TM, as they say...

Anyways, I think the argument over whether type theory is a successful
foundations for CS research and practice is already a closed issue. It
demonstrably is. So is set theory.

~~~
pron
> If I have to perform another model checking routine

Why model checking? TLA+ is not a model checker. It's a specification and
proof language. There _is_ a model checker that can check TLA+ specs, just as
there is a proof assistant that can check TLA+ proofs.

> must have an extremely low opinion of quite a number of very smart people.

Why low opinion? Type theory isn't "smarter", it's just more obscure.

> The "set theory" you see in a discrete math course kind of barely scrapes
> definitions. I think most people who posit that "set theory" is nice and
> simple must've never really dove deem into the actual theory of set
> theory...

Thankfully, that basic set theory -- the one that barely scrapes the
definition -- is all that's required to specify real-world programs.

> Did you read this dissertation's explanation of why separation logic is
> important?

Skimmed it. Unless I'm missing something (which is quite possible) it seems
very straightforward to use in TLA+: you just define the operators just as
they're defined in the paper. There's no need to "code" the logic in any
arcane or non-obvious way; just define it. In fact, this short paper describes
doing just that and it seems pretty basic:
[http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.570...](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.570.3798&rep=rep1&type=pdf)

> Anyways, I think the argument over whether type theory is a successful
> foundations for CS research and practice is already a closed issue. It
> demonstrably is.

It never even crossed my mind to make the case that it isn't. That type theory
is an interesting topic for research is obvious. I also never implied that
dependent-type tools can't do the job; as you say, they demonstrably can. I
simply asked if I -- a programmer working in the industry -- had a program I
wished to verify, whether there was a reason why I would reach for a
dependent-type tool vs. a classical logic tool. So far, I don't see why I
should. It seems to basically boil down (at least according to the responses
in this thread) to whether I prefer working with direct, classical logic or
with dependent types.

~~~
nmrm2
_> TLA+ is not a model checker. It's a specification and proof language. There
is a model checker that can check TLA+ specs, just as there is a proof
assistant that can check TLA+ proofs._

For the third (!) time, modus operandi matters.

Coq's huge library of tactics and proofs of facts that TLA+ might just throw
at a model checker are, in my experience, very useful starting points for
things that can't be model checked.

Ostentibly TLA+ could build up an equivalent library of tactics and proofs.

 _> Why low opinion?_

Even supposing type theory is "more obscure", it's pretty hard to believe that
someone could study it for 30 years and not understand it better (in some
sense) than ZF. That's a pretty extraordinary claim.

 _> basic set theory -- the one that barely scrapes the definition -- is all
that's required to specify real-world programs._

Not really -- you often need non-trivial constructions when proving properties
about rewrite systems within set theory. Those constructions are _much_ harder
to understand than induction on algebraic datatypes, which is why type
theoretic approaches have dominated in formalization of programming languages.

 _> Unless I'm missing something..._

I'm disinclined the dismiss separation logic just because you can code up some
of its ideas in another logic (in a way that counts as a novel contribution
worthy of publication, no less).

The clarity with which separation logic presents the idea and use of the
separating conjunction has obviously influenced the paper you cited. I would
think this is reason enough to summarily dismiss these "everything's a
TM"-style arguments you're making here. The authors of the paper you cited are
only not using separation logic in a kind of meaningless sense IMO.

 _> So far, I don't see why I should._

This sentiment kind of reminds me of people who have never typeset mathematics
complaining that LaTeX is useless... which is to say, perhaps you simply
aren't interested in and so don't care about the use cases where Coq shines?
Lots of people -- including me -- have already pointed out a lot of these use-
cases in this comment section.

But really, based on the arc of this conversation, I would be enormously
surprised if you chose to use Coq to prove X, even if there were a proveX
tactic already written in Coq and the TLA+ implementation would take all day
;-)

~~~
pron
> Coq's huge library of tactics and proofs of facts that TLA+ might just throw
> at a model checker are, in my experience, very useful starting points for
> things that can't be model checked.

I don't understand what you're saying. What does TLA+ have to do with the
model checker? TLA+ is a language, while TLAPS (the proof system) and TLC (a
model checker) are two different products (each supporting different subsets
of TLA+, BTW). TLAPS comes with its own library of proofs. I don't know how
extensive they are.

AFAIK, you can't (and certainly you don't) take a fact verified by the model
checker and state it as an axiom of your proof. The interaction between the
two -- if any -- is that Lamport encourages you to model check your spec
before trying to prove it (assuming you want to prove it, or even model-check
it) because proving things that are false may prove hard. However, the result
of the model checker play no part in the proof.

When it comes to proofs, TLA+ is intended to be declarative and independent of
the proof engines used (by default, TLAPS uses SMT (any of several solvers),
Isabelle and another couple of tool, passing every proof step to every tool
unless a tactic specifies a particular one).

This is a nice introduction for the design of the TLA+ proof language:
[http://research.microsoft.com/en-
us/um/people/lamport/pubs/p...](http://research.microsoft.com/en-
us/um/people/lamport/pubs/proof.pdf) which includes (in the appendix) a few
proofs in calculus.

> it's pretty hard to believe that someone could study it for 30 years and not
> understand it better (in some sense) than ZF. That's a pretty extraordinary
> claim.

I did not mean to claim that type theory researchers don't understand it
better than basic college set-theory. I meant that _in general_ , engineers
and academics are more familiar with basic set theory.

> just because you can code up some of its ideas in another logic (in a way
> that counts as a novel contribution worthy of publication, no less).

That was a technical report that mentioned in passing using an approach
similar to separation logic.

> "everything's a TM"-style arguments you're making here

I am not making that argument. My argument is that most engineers and CS
academics (except those in PLT) would be more familiar -- and therefore more
likely to use -- a tool based on basic math they know well. In addition, I
haven't seen anything to suggest that the complexity difference between
specifying a system using dependent types and specifying it in TLA+ is
significant at all in either direction. Learning Coq, however, requires much
more effort, and AFIK, it doesn't support formal verification mechanisms other
than proofs (e.g. model checking)

> perhaps you simply aren't interested in and so don't care about the use
> cases where Coq shines?

Perhaps. I design data structures (mostly concurrent), and my interests lie in
specifying and verifying those as well as distributed systems, and less in
verifying compilers. My current TLA+ spec is too complex to be proven anyway
(regardless of the tool used) in any feasible amount of time, so I rely on
just the spec as well as some model checking.

It is also true that I may be biased. As an "algorithms guy" I feel more at
home working with tools and languages designed by other algorithm people (and
more importantly _for_ algorithm people) rather than PLT people.

Nevertheless, I'm interested in learning the difference between the two.

> But really, based on the arc of this conversation, I would be enormously
> surprised if you chose to use Coq to prove X, even if there were a proveX
> tactic already written in Coq and the TLA+ implementation would take all day

Considering that learning Coq would take me much longer than a day, that would
be a wise decision.

------
xvilka
Would be nice to use it for Frama-C.

[1] [http://frama-c.com](http://frama-c.com)

------
vinceyuan
The font size on this page is very small. My eyes hurt when I read it on my
non-retina 13-inch Macbook Pro (on low-resolution external monitor, it looks
bigger.). Is it because I am not a native English speaker? Do native English
speakers get used to this font size?

Some websites including Hacker News use very small font size. Reddit's font
size is bigger. I created a HN web reader (
[http://hackernewsroom.com](http://hackernewsroom.com) ) with bigger font size
(for myself and friends) to read titles and comments easily. But I can't
change the font size of the original article. (Have to zoom in the page by
pressing cmd+=)

~~~
testtest
It might be because the font size is defined in the CSS in pixel units,
instead of em, which is a point size (DPI dependent).

[http://www.w3.org/TR/REC-CSS2/syndata.html#length-
units](http://www.w3.org/TR/REC-CSS2/syndata.html#length-units)

If you have a problem reading text on the screen, I suggest you use something
like Clearly extension that reformats pages with a much more reading-friendly
style.

You don't have to sign up to use this extension :
[https://evernote.com/clearly/](https://evernote.com/clearly/)

You can also try to print the page into a PDF and read it like that.

~~~
wereHamster
Note that CSS pixels are not the same as physical pixels. Browsers are free to
rescale CSS pixel values. Indeed, it's written in the document you linked to:

> If the pixel density of the output device is very different from that of a
> typical computer display, the user agent should rescale pixel values.

So actually 12px font should look about the same on all output devices,
regardless of their physical DPI. But I agree that 12px is rather small for
copy text.

~~~
Someone
12px font also could become very small in the future, if typical computer
displays end up having high dpi.

I guess a conforming browser should randomly sample computer displays before
deciding how big 1px is.

Serious version: I don't think text like that should make it into a standard.

------
rav
In a sense, Robbert's thesis can be viewed as a more mathematically stringent
(and thus arguably clearer) C11 specification. For instance, page 42 has the
following mathematical gem:

> The C11 standard defines the usual arithmetic conversions as a binary
> operation on integer types. This definition is complicated and contains some
> implicit duplication due to symmetry, which is inconvenient when proving
> properties. To improve the situation, we show that the usual arithmetic
> conversion of integer types τ_i and σ_i is a least upper bound τ_i ∪ σ_i in
> a join semi-lattice.

------
JabavuAdams
> Surprisingly, correctly describing the set of programs that have undefined
> behavior in a formal manner is the hardest part of formalizing C.

Isn't this undecidable?

~~~
mikeash
Yes, but that doesn't mean you can't describe it.

The halting problem is undecidable, but it's really easy to describe the set
of programs that halt. I just described it right there!

That quote is talking about defining the set, not determining whether a
program is a member of the set (which is indeed impossible in the general
case).

------
LukeHoersten
awesome

