
Learn TLA+ (2018) - brodo
https://learntla.com/
======
hwayne
GodDAMN am I behind on updating this

There's at least a couple more major improvements I want to make. One is
fixing all of the small errors, and one is adding c-syntax. They've been
stalled out due to Reasons (tm), but I'm hoping on diving back in soon.

~~~
brodo
There is C syntax? Neat. Is it used in your book?

~~~
strangecasts
Practical TLA+ uses the Pascal-style syntax for PlusCal - the only guide to
the C syntax that comes to mind ATM is Lamport's own:

[https://lamport.azurewebsites.net/tla/c-manual.pdf](https://lamport.azurewebsites.net/tla/c-manual.pdf)

~~~
hwayne
I really strongly prefer the p-style syntax because it doesn't overload the
meaning of `{}`, but a lot of other people in the community use c-style.

------
timClicks
Leslie Lamport has provided his own +1 to this book, as well as a thorough
review here [https://lamport.azurewebsites.net/tla/practical-
tla.html](https://lamport.azurewebsites.net/tla/practical-tla.html)

~~~
moocowtruck
oh wow that was really awesome for him to do that! really appreciate you
posting this link

------
asimpletune
Lamport also made a free video series for learning TLA+

[http://lamport.azurewebsites.net/video/videos.html](http://lamport.azurewebsites.net/video/videos.html)

~~~
dqpb
These videos are great!

------
daenz
Every time I see some TLA+ related article, I read it and then spend an hour
convincing myself that I don't have time to learn it right now, even though I
really want to. That convincing usually comes in the form of showing that the
gains are not worth the effort for the software field that I am in. Can
someone change my mind on this?

~~~
01100011
I love the concept of formal verification, but translating my algorithms into
yet another DSL and hoping I get that right seems like a waste of time. I'd
rather simulate the algorithm with my brain.

~~~
varelaz
Good luck with this. I heard about practical TLA usage. There was hang up in
python asyncio library related to concurrency and it required 24 steps to
reproduce. It was found and fixed with TLA help.

~~~
michael_j_ward
Do you have a source for that? I tried a few google searches and came up empty

~~~
varelaz
It was part of the talk in PyCon Ukraine 2018, which I took part.
[https://pyvideo.org/pycon-ukraine-2018/verification-of-
concu...](https://pyvideo.org/pycon-ukraine-2018/verification-of-concurrent-
and-distributed-systems.html) That's the video

Sorry it's on Russian and amount of steps was 33. Check 28:30

------
akersten
In the landing page example:

> But if we run the model checker, we find that’s not true: Alice could trade
> an item to herself and, before that process finishes running, resolve a
> parallel trade giving that same item to Bob. Then the first trade resolves
> and Alice gets the item back.

How does the model checker know that "parallel threads" are even a possibility
here? It seems like it's checking what amounts to a leaky abstraction from the
actual implementation of this algorithm.

In my mind a strictly logical validator would look at the proposed algorithm
and say it checks out (since in math-land transactions are atomic). Is this
just a feature of the model checker? Can it be turned off/tuned to the
expected real-world implementations of the algorithm?

~~~
loup-vaillant
TLA+ is originally mainly meant to check for concurrency bugs, so the
possibility of parallel threads is kind of in its core DNA. The name "process"
might also give a hint that parallel threads are indeed possible.

I don't know how, but I expect you can indeed model actual transactions.

~~~
hwayne
Depends on what you mean by "actual transaction". Every TLA+ behavior is
describable as a series of "next-states", where _usually_ (not always, but
_usually_) one action is true. In which case you'd model the transaction by
determining the changes that the overall transaction makes and then declaring
them all as part of a single action.

------
endiangroup
We used this resource to put together a model for our distributed sessions
management algorithm, heres a GitHub with Go implementation and PlusCal / TLA+
model [1]. We also wrote a more in depth article about it here [2] (with an
interactive version in JS). It took a few attempts but eventually it clicked!

[1]
[https://github.com/endiangroup/compandauth](https://github.com/endiangroup/compandauth)
[2]
[https://endian.io/articles/compandauth/](https://endian.io/articles/compandauth/)

------
mjfl
What is the difference between formal methods as implemented by TLA+, proof
assistants like Coq, and theorem proving languages like Agda (Idris?)?

~~~
jlrubin
coq allows you to extract a real program, whereas TLA+ just checks a spec for
correctness (and you may have bugs implementing it)

~~~
pron
See my other comment on the matter, but this presents a choice where there is
none. No tool that extracts or verifies actual code (and people have actually
done both with TLA+) can operate on programs larger than a few thousand lines.
So it's not a choice between fully verifying a 200KLOC program or just its
2000-line high-level spec. It's between verifying/extracting a 2000 line
program or a 2000 line spec that abstracts a 200KLOC program. In other words,
for programs of a certain size, verifying a high-level spec is the only real
option for "deep" verification.

------
dqpb
The syntax in that first example is off putting.

~~~
jnwatson
Yes, you can definitely tell the inventor of TLA was also the first developer
of LaTeX. As such, it is a language best read after typesetting.

For a good example, see Appendix B in Diego Ongaro's dissertation [1], the
formal Raft specification. After a little of the Learn TLA+ book, I was able
to muddle through the spec enough to implement it.

1:
[https://ramcloud.stanford.edu/~ongaro/thesis.pdf](https://ramcloud.stanford.edu/~ongaro/thesis.pdf)

------
jnwatson
I guess this is a meta question: do typical US undergraduate university
Computer Science curricula not have Formal Methods as a class?

I'm just wondering because of the number of fairly basic questions here. I
went to a middling midwestern state school 20ish years ago and that was part
of the curriculum. Perhaps we have lots of non-CS-graduates here?

This is not to denigrate the discussion. I'm just surprised that this is news.

~~~
ajxs
While it seems anecdotally true that the mathematical foundations of computer
science are being downplayed in universities in favour of giving students a
more practical education in software engineering, which is an entire
contentious discussion in and of itself. I think that a big factor in
influencing your observation could be that CS is no longer the main entry
point into a career in software development. In the organisation I work in,
many of the younger hires do not come from CS backgrounds, and even many of
the older hires come from other scientific disciplines.

~~~
momofarm
if you want to get into software development, just go to any so called "boot-
camp" class for few couple months.

Then you know how to write some webpages with new shiny JS framework, congrats
you are a "qualified" software guy. Now go to write those sh*ty code
............

Math? what math? CS background? algorithm? if you are not going to challenge
FAAG why bothered?

if you just want to code few pages, writing some sql query, there is no need
for that. and for 80% of enterprises, they need only CHEAP engineers.

sorry for the rant, just another morning with lots of idiots bothered me.

~~~
ajxs
I agree with you. Unfortunately there's numerous perverse incentives within
the business sector that are promoting this. I'm glad I won't be alive in 2070
to fly intercontinental in an aeroplane with avionics written in Node.js...

~~~
momofarm
I don't think NodeJS is an issue, the problem always lies in PEOPLE who wrote
code.

If you want me to said what's the terrible thing? the code that very FEW
people actually use, no one EVER VERIFIED it.

Do you dare to install these piece of crap to the plane you sit on?

------
dqpb
Can someone who has learned TLA+ comment on how useful it is has been for you?

~~~
mavelikara
Someone from AWS posted this few days ago:
[https://news.ycombinator.com/item?id=19634915](https://news.ycombinator.com/item?id=19634915)

~~~
k__
Is this somehow integrated with Java or does it work with a programming
language of my choice?

~~~
strangecasts
Specifications aren't tied to any specific programming language - you're
outlining how the algorithm should work, not the implementation.

(You _can_ write Java code and invoke it during model-checking if you _really_
need to, but I haven't come across any situation where it's actually
necessary.)

~~~
k__
Ah, okay.

I had the impression it was about somehow writing a program in a more strict
language and then exporting it to Java.

I don't do much algorithmic design.

------
fizwhiz
Since the author hangs out here on HN, I'd like to ask: how long does it take
to develop a working proficiency to model standard distributed systems (ex: a
souped up key/value store)? It seems like he's written 2 books:

    
    
      Learn TLA+
    
      Practical TLA+
    

Would there be any other resources that you'd recommend to supplement after
those 2 books?

~~~
pron
Not the author, but it would take 2-3 weeks if you learn by yourself (to get
real stuff done, not to be an expert), or 3 full days of hands-on training.

I would recommend Lamport's video course[1] and TLA+ hyperbook[2] (Lamport
says he's stopped working on it, but it's fairly complete). Note that both
Learn TLA+ and Practical TLA+ don't teach much of TLA+, but rather a language
called PlusCal that compiles to TLA+. Some programmers may find it easier to
start with because, unlike TLA+, PlusCal resembles a programming language (and
like all programming languages, it is much more complex than TLA+, but more
familiar). Hillel's book also contains some good real-life examples.

You can also find examples and various other links posted on the TLA+
subreddit[3].

[1]:
[http://lamport.azurewebsites.net/video/videos.html](http://lamport.azurewebsites.net/video/videos.html)

[2]:
[http://lamport.azurewebsites.net/tla/hyperbook.html](http://lamport.azurewebsites.net/tla/hyperbook.html)

[3]: [https://www.reddit.com/r/tlaplus/](https://www.reddit.com/r/tlaplus/)

~~~
nickpsecurity
"Note that both Learn TLA+ and Practical TLA+ don't teach much of TLA+, but
rather a language called PlusCal that compiles to TLA+. Some programmers may
find it easier to start with because, unlike TLA+, PlusCal resembles a
programming language"

Most evidence indicates using familiar concepts aids learning. That makes me
hypothesize the best recommendation is the more mathematical ones for folks
strong in math and the programmer-centric ones for those strong in that. That
also implies that, for each of these formal topics, we need one version for
each audience to draw more people in.

~~~
fizwhiz
Seem's like hillel's material is more approachable for people with a
programming background, and pron's recommended material better for people who
are more math-y. I'm likely going to start with Hillel's books to develop a
working understanding and then move onto Lamport's stuff once i'm ready for a
rigorous deep dive.

------
billfruit
When I did spend a little amount of time trying to learn TLA+, the biggest
annoyance is the two different syntax/form in which TLA+ can be written, a
mathematical symbolic form, and a text form like a program.

Now, this topic is a rather abstract and difficult to comprehend one, and
having two equivalent ways of doing it, introduced right from the beginning, I
thought was too much of a cognitive load, considering the difficulty of the
subject.

I hope this material from the article is simpler for learners.

------
skadamat
Hillel Wayne was recently on the Future of Coding podcast and they go deep
into formal methods -
[https://futureofcoding.org/episodes/038](https://futureofcoding.org/episodes/038)

------
GTP
Is there a validator to verify if an implementation of an algorithm respects
the TLA+ specification? Can somebody please tell me (some of) the main
differences between TLA+ and JML?

~~~
0815test
> Is there a validator to verify if an implementation of an algorithm respects
> the TLA+ specification?

TLA+ has some tools to do this, _if_ you write your implementation in some
TLA+-specific language like PlusCal. But this would be the sort of "end-to-
end" verification that TLA+ proponents would regard as only being feasible for
relatively simple software.

Relatedly, JML basically adds support for "design by contract" specifications
to the Java programming language - expressing preconditions, postconditions
and invariants. These can be quite useful, but the focus with TLA+ - in this
thread and elsewhere - seems to be rather on "deeper" properties that
encompass entire systems in a more global and cross-cutting way, and are thus
inherently harder to manage with simpler tools.

------
lewis1028282
Is this like formal verification/formal methods then?

~~~
chimpburger
It is a formal method.

------
dorfsmay
Is Formal Specification used by proving language like Coq and Idris?

Are they a different way of doing the same thing?

Or two completely different thing?

~~~
pron
They share many philosophical similarities, but their design and goals are
very different. Coq and Idris are mostly used for research. TLA+ is designed
for heavy-duty, everyday use in industry by software and hardware developers
working on large, complex real-world systems.

------
karl_schlagenfu
How does this compare with other formal methods like Z? Has anyone here used
them successfully in the field?

------
acrophiliac
Disappointed. Was expecting to learn more about Three Letter Acronyms. :)

------
OptionX
What's the advantage of this versus normal unit testing?

~~~
strangecasts
They're complementary, IMO: unit tests show that your implementation works as
expected, while a formal specification allows you to check that your approach
is right to begin with.

That being said, TLA+ specifications have uncovered really hairy concurrency
bugs which would be a nightmare to unit test - for instance, Amazon found bugs
in DynamoDB which required 35 steps to reproduce [1].

[1] [http://lamport.azurewebsites.net/tla/formal-methods-
amazon.p...](http://lamport.azurewebsites.net/tla/formal-methods-amazon.pdf)

------
youeseh
So, like writing code and then writing tests for it?

~~~
chimpburger
Not really. Formal verification methods are mathematically rigorous. They can
mathematically prove or disprove correctness of programs and catch bugs that
would be extremely difficult for a human to find
[https://en.wikipedia.org/wiki/Formal_verification](https://en.wikipedia.org/wiki/Formal_verification)

~~~
youeseh
"Formal verification of software programs involves proving that a program
satisfies a formal specification of its behavior."

It sounds a lot like writing your specs as tests that are then satisfied by
the implementation code.

~~~
chimpburger
TLA+ is for building and verifying a formal model of the program (a
blueprint), not for verifying the implementation.

