
Writing correct lock-free and distributed stateful systems in Rust, with TLA+ - guifortaine
https://github.com/spacejam/tla-rust
======
unboxed_type
Lets say you model-checked some distributed algorithm with TLA+. You then
implement it in Rust. How are you going to check that your implementation
implements exactly the algorithm you have checked and not some other algorithm
which looks very similar?

I think the phrase 'reliable systems' is more appropriate to what you are up
to, as opposed to the phrase 'correct systems' which usually corresponds to
formal verification.

~~~
munin
This is a gap in this kind of verification work. Systems like coq get around
this by having a facility to "extract" code from the proof itself. You would
define your algorithm as a Fixpoint, prove properties of that Fixpoint, then
extract that Fixpoint into some ocaml code and use it directly in your real
programs. This closes the loop you describe, insamuch as you trust the
extraction process and the ocaml compiler and runtime (but other work is
ongoing to close that loop as well).

~~~
chrisseaton
The loop is always open though. Who verifies your verification code? Who
verifies the processor implementation? All you can do is reduce the gap in the
loop surely?

~~~
munin
You can close all of those. You can have large amounts of your verifier itself
be verified. You can verify the CPU design (hardware people tell me this used
to be standard, and a lot of the formal methods community has roots in
verifying hardware).

What you can't close is the language that you use to do all the verification
in, itself. You wind up with a core calculus that you have to stare at really
hard and trust / prove via other means.

~~~
roblabla
Isn't this susceptible to a trusting trust[0] attack, where your verifier has
a bug that makes it verify itself, even though it shouldn't ?

It's always possible there is a loop somewhere. The best you can hope for is
make it infinitesimally small.

[0]:
[http://wiki.c2.com/?TheKenThompsonHack](http://wiki.c2.com/?TheKenThompsonHack)

~~~
seagreen
There's been a simple counter to trusting trust attacks since 2009:
[https://www.dwheeler.com/trusting-trust/](https://www.dwheeler.com/trusting-
trust/)

~~~
nonsince
If I understand the abstract correctly, that relies on having a trusted
compiler, which assumably would have to be bootstrapped ultimately from a
trusted hand-written compiler in machine code. This effectively counters
malicious trusting trust attacks but does not effectively counter trusting
trust attacks due to error, because your entire trusted stack has to be
correct.

That's not to say there's no way to close the loop here, simply that I don't
know that this is it.

~~~
dllthomas
You don't understand the abstract correctly. It relies on having compilers
from diverse sources, unlikely to share _the same_ malicious modification. It
does not rely on any of them being trusted.

------
dikaiosune
There's a lot of interest in formally verifying things about the unsafe subset
of Rust, for example the Rust Belt project: [http://plv.mpi-
sws.org/rustbelt/](http://plv.mpi-sws.org/rustbelt/). One thing I've not well
understood is how these efforts may be affected by some of the conversations
around the unsafe code guidelines effort:

[https://github.com/rust-lang/rfcs/pull/1643](https://github.com/rust-
lang/rfcs/pull/1643) [https://github.com/nikomatsakis/rust-memory-
model](https://github.com/nikomatsakis/rust-memory-model)

~~~
vog
Just curious: Are there also attempts in the reverse direction?

That is, are there attempts to improve the Rust language and/or optimizer such
that more and more "unsafe" sections can be replaced by safe code that has the
same performance? (maybe even compiles down to the same machine code)

~~~
sanxiyn
There is interest in enabling safe interface, that is, improving Rust the
language so that you can provide safe interface with better performance. A
good example is so-called "streaming iterator".

There is less interest in changing Rust so that currently unsafe
implementation (not interface) of safe interface can be made safe, that is,
verified by compiler. As grandparent stated, and I agree, as long as interface
is safe I don't think it is important that implementation is verified by
compiler (hence safe) or some external tool or hand proof.

~~~
michaelmior
Maybe I'm missing something, but I could write an interface labelled sage on
top of any unsafe code I want. Why doesn't it matter that my unsafe
implementation is proven correct?

~~~
kibwen
This is going to sound tautological, but the unsafe keyword is for operations
that the compiler _can 't_ prove. The unsafe keyword only unlocks a very small
number of possible new operations (it doesn't turn off any existing checks),
and those operations make unsafe Rust as freeform as C; if you could prove
unsafe Rust correct then you could just prove C code correct.

~~~
nickpsecurity
Which Frama-C, Simpl/C, and Astree Analyzer all do. Then that C code is quite
trustworthy. Rust has stuff like Simpl in the works but needs something like
Frama-C or SPARK for unsafe.

~~~
kibwen
Can you give me an example of something that safe Rust doesn't allow, thereby
requiring the unsafe keyword to implement, but that also contains an error
that passes the Rust compiler silently but that Frama-C or SPARK would
statically detect?

~~~
nickpsecurity
That's a trick question: you know I don't know know Rust enough to answer it.
:P However, I fo know the things I mentioned can prove safety properties about
unsafe code. Rust compiler can't per these threads. So, If A then B...?

~~~
kibwen
But whether SPARK et al can prove properties about unsafe code isn't the
pertinent topic. SPARK and friends are _subsets_ if their respective
languages, implying that there are features of those languages that, if their
use is not restricted, inhibit the production of correctness proofs. Rust
_already_ has such a thing, by default, in the form of non-unsafe Rust, and
the notion of "unsafe" in the context of Rust and Frama-C is not guaranteed to
be the same. So what I'm asking is for you to demonstrate something that these
other languages can prove that safe Rust can't.

By the way, you've been in Rust threads enough years now championing safe
languages that you really no longer have any excuse for not learning Rust. :P

~~~
nickpsecurity
"But whether SPARK et al can prove properties about unsafe code isn't the
pertinent topic. "

I thought that these worries about unsafe code in Rust indicated people wanted
a way to ensure it's as safd as possible. Traditionally, unsafety the language
itself cant handle is dealt with via external tools for analysis, proving, and
testing. These exist for unsafe code in Ada, C, C+×, Java, and SPARK. Tool-
assisted, subsets of each are deployed in safety-critical industries. Anyone
doing unsafe codd in them will get more robustness than unsafe code in Rust.

So, you already have protection measures in safe Rust. Other tools, esp SPARK
Ada and C enhancements, have them for unsafe code that Rust does nothing to
protect. That's a gap in capabilities Rust needs to close. Until then, it's on
the table when choosing which to use for unsafe code. Note that Im also a big
fan of mixing and matching where we used the C or SPARK provers on unsafe Rust
coded to be semantically equivalent. I just know most will prefer Rust to
handle this itself.

"By the way, you've been in Rust threads enough years now championing safe
languages that you really no longer have any excuse for not learning Rust. :P"

Haha. I do plan to learn it for my Brute Force Assurance concept. Remember I
have a brain injury, though, that knocked out memory and good chunk of
learning pace on top of full-time job. That plus a shit-ton of broad R&D
leaves little time for coding.

------
cabaalis
Rust-ignorant here; I was reading your "why rust" section. It contains a lot
of information about how safe the resulting code is and how that is such a
great benefit, and that's why Rust was selected. But then it has this
statement: "However, it needs to be noted that when creating lock-free high-
performance algorithms, we are going to need to sidestep the safety guarantees
of the compiler." .... So why Rust?

~~~
burntsushi
The benefit of Rust is not necessarily to completely avoid unsafe (although,
that does happen in practice a lot). The benefit is that unsafe can be
buttoned up behind a safe API, hopefully in a way that is Zero Cost. So you
might have some unsafe core that you need to pay extra special attention to,
but the rest of your code can be in safe Rust.

For example, I maintain a Snappy compression crate in Rust. It uses quite a
bit of unsafe internally (mostly for being too clever with unaligned
loads/stores), but any users of that crate never need to utter `unsafe`. If
there's a bug somewhere, you can be confident it's not in the code that uses
Snappy, but rather in Snappy itself, which is presumably a much smaller
surface area than "all of the code."

~~~
mhh__
That's still not really unique to rust, D (at least) does that too. However,
good point nonetheless.

~~~
steveklabnik
A big difference is the defaults; Rust made an explicit design decision to
make unsafe the default, to mark it with a keyword and to make unsafe a
superset of safe. It is totally true that these things aren't unique, but
there is a lot of advantage for this specific set of choices in combination.

~~~
staticassertion
> Rust made an explicit design decision to make unsafe the default

You mean 'safe the default'.

~~~
nickpsecurity
C programmers could've had so much fun with that quote. ;)

------
EGreg
If you're working with a language that has closures, I started to have a
heuristic:

    
    
      // if you are writing a comment like this
      // explaining what the next block of code does
      // chances are you should just refactor it
      // into a closure
      _doWhateverCommentSaid(arg1, arg2, arg3);
    

it's true that this needlessly adds stack overhead, but this way, you are more
likely to structure your code properly, re-use it properly (which more than
pays for the stack overhead), and move your closure / method to be more and
more global as needed.

Perhaps if you have a comment like this:

    
    
      f(x); // x matches some assumption
    

then in fact, you should add validation to f itself, in the form of a
function, same as above.

    
    
      function f(x) {
         _theAssumption(x);
      }
    

this habit encourages re-use and proper structuring of code.

in short, comments may be an anti-pattern!

~~~
eridius
How does wrapping up code in a closure "fix" the need for comments? And what's
wrong with having comments anyway? You haven't demonstrated that comments are
an "anti-pattern", all you've demonstrated is that you personally prefer a
different style.

~~~
EGreg
Whenever you put a comment, chances are you should have encapsulated the block
there into a reusable block of code.

~~~
eridius
Why? You're saying that without any supporting evidence. And frankly, what
you're suggesting sounds to me like an unreadable mess. You're going to bloat
your code with closures all over the place, simply so you can attach a name to
the closure, all to avoid having a simple comment. Naming closures is not a
good substitute for comments.

~~~
hoodunit
Naming closures or functions absolutely is a good substitute for a large class
of comments. When you write a comment as a function name, that comment gets
maintained when the code changes. It also encourages modularizing your code-
often when you write a comment it's a sign that your code is trying to operate
conceptually on more than one level and it would be more understandable to
separate those levels.

I wouldn't say comments are an antipattern but they are a code smell and
should be used sparingly and only where the same explanation can't be given
through function/variable naming or refactoring.

------
nicknash
For lock-free data structures, how does this verification encode the memory
model? E.g. high-quality model checkers for the C++11 memory model allow for
outcomes inconsistent with the execution order of the code (in addition to
outcomes that aren't sequentially consistent, but are consistent with the
execution order). They also work on unmodified source. In the past I've seen
SPIN/Promela used as a tool for concurrency-checking, but it's silent on
memory-models (implicitly sequentially consistent)

~~~
pron
You just define it :) TLA+ doesn't have a built-in memory model, nor any
concept of memory whatsoever. You define your constructs at the level of
detail you think is important. Weak memory can be defined using a CPU-local
memory that is occasionally reconciled with RAM or other core-local memory.

------
mcguire
" _Pluscal has two forms, c and p. They are functionally identical, but c form
uses braces and p form uses prolog /ruby-esque begin and end statements that
can be a little easier to spot errors with, in my opinion._"

Nit: that should be Pascal/Ruby, no?

~~~
dmix
> PlusCal (formerly called +CAL) is a formal specification language created by
> Leslie Lamport, which transpiles to TLA+

[https://www.wikiwand.com/en/PlusCal](https://www.wikiwand.com/en/PlusCal)

~~~
mcguire
Right, but Prolog doesn't use begin/end's.

------
tschottdorf
[https://github.com/uwplse/verdi](https://github.com/uwplse/verdi) is a good
example of this (it uses coq and extractions along with minimal glue).

~~~
unboxed_type
Have you tried it yourself or maybe you know someone who have? Honestly, I
think that this development is highly impractical due to very high entrance
ticket price for someone not belonging to UW PLSE group ;-)

------
lenage
thanks for great work

------
alexnewman
I like this a lot

~~~
akoster
Same here!

