
Programming Models for Distributed Computation - ingve
https://github.com/heathermiller/dist-prog-book
======
shalabhc
> There are problems that exist in distributed system environments that do not
> exist in single-machine environments. Partial failure, concurrency, and
> latency are three problems that make distributed computing fundamentally
> different from local computing.

IMO this can frame our thinking too much. Concurrency and latency are problems
in single systems as well. It would be great to have single systems that
handle partial failure, can be updated incrementally, etc. Thinking of
'distributed computing' as a different domain than single system computing is
not a useful long term approach. In the end very few systems are 'single
machine' systems, if at all - machines are all connected to each other and
participate in other, larger, systems.

~~~
hinkley
Some classes of problems are masked on a single system. For instance, failure
of the networking card doesn't preclude the processes from talking to each
other. A process that's running on a system that has hung can't make any
forward progress so it doesn't care if another process it's talking to is in
the same state.

Our desktop computers look more and more like clusters in their own right (how
many SoCs are inside of your computer right now?). There's a lot of talk
lately about having high performance and low power CPUs in the same machine
working in tandem.

I figure at some point we will start formally treating desktops and laptops
like small clusters, at which point these distinctions might start to
disappear.

~~~
shalabhc
> Some classes of problems are masked on a single system.

Yes but you can also have IPC failure, syscall hang, single thread hang etc.
Instead of treating these as special cases, these could be handled by generic
patterns that apply to 'distributed computing', if the system was designed
similarly.

------
jchanimal
This looks like a reasonably complete survey. However, one important topic is
missing. That is the strong consistency algorithms used by databases like
Spanner and FaunaDB.

Those two globally consistent databases use different families of algorithms,
so it’s probably worth including both. Here is a comparison between them:
[https://fauna.com/blog/distributed-consistency-at-scale-
span...](https://fauna.com/blog/distributed-consistency-at-scale-spanner-vs-
calvin)

~~~
xtacy
I believe it's covered: Topic 6.

~~~
jchanimal
I don’t see anything in there even about basic building blocks like RAFT.
Maybe I’m missing it, or maybe there’s room for someone to write another
chapter.

~~~
KirinDave
Specific algorithms like RAFT or Paxos are more of an addendum than a core
aspect of the topic, though.

------
ericfrederich
I watched a talk on Dask

[https://www.youtube.com/watch?v=RA_2qdipVng](https://www.youtube.com/watch?v=RA_2qdipVng)

... looks awesome, and something that seems only possible with dynamic code...
or at least code that can introspect it's own AST.

EDIT: the cool part about it, I think, is that you don't have to shoehorn your
algorithm into any particular model like map reduce or something else. You
just write idiomatic Python code and... magic

------
frankmcsherry
I am a bit torn with this, because while I love people reading and then
writing up things, there are so many changes I think they should make but I
don't want to just barge in and make pull requests.

E.g., they largely miss DryadLINQ (disclosure: not an author, but worked with
them), which I would rate as the second most important paper after MapReduce
in the "batch compute" space. It shows up only as what you should use to write
Dryad programs, because Dryad's programming model "is less elegant than
MapReduce"; harsh.

The streaming section (disclosure: Naiad co-author) dances around and then
suggests that Google Dataflow is somehow a unification of previous work, which
is .. well, you can read the Google Dataflow paper

    
    
        http://www.vldb.org/pvldb/vol8/p1792-Akidau.pdf
    

and do a search for "Naiad"; it's not even on their radar.

It feels a bit like the critiques are perhaps too-credulous readings of the
related work sections of other papers.

I'm not sure a great way to get to a consensus understanding of the position
of the works in the historical record, or whether it is even possible with
sensitive authors (hi), but it would be cool to have. Any recommendations
would be great.

------
DonbunEf7
Well-researched and comprehensive. The survey of distributed languages
includes not just Erlang and Akka, but also E and Cloud Haskell. The overview
of CAP is deep and rigorous while not wasting words. This will be a great book
once it's finished.

------
naasking
This should mention Microsoft Research's Concurrent Revisions [1]. I think
it's by far the easiest abstraction to grasp for ordinary programmers,
particularly if you've ever used version control. In that case, you already
know how they work! Revisions scale from incremental, parallel and concurrent
programming all the way up to distributed computation. Very neat work.

[1] [http://lambda-the-ultimate.org/node/4699](http://lambda-the-
ultimate.org/node/4699)

~~~
sudouser
Thanks for bringing that up naasking, quite nice indeed

------
shalabhc
Perhaps
[https://en.wikipedia.org/wiki/Croquet_Project](https://en.wikipedia.org/wiki/Croquet_Project)
should be included in Chapter 4: [https://github.com/heathermiller/dist-prog-
book/blob/master/...](https://github.com/heathermiller/dist-prog-
book/blob/master/chapter/4/dist-langs.md)

------
bogomipz
This looks great. I look forward to following this and reading the updates.
Thanks for sharing!

------
zengid
No mention of session types?

[http://groups.inf.ed.ac.uk/abcd/](http://groups.inf.ed.ac.uk/abcd/)

[https://youtu.be/8GJ2AnCjCvU](https://youtu.be/8GJ2AnCjCvU)

------
macintux
Pretty cool to see Bloom and Lasp covered. Definitely a sign that they're
serious about covering many aspects of the problem space.

------
bootcat
Nice coverage !! For some reason they don't offer a one click pdf and its bit
difficult to navigate !

