Ask HN: What are some good projects/resources to learn distributed systems?

[gyani95]

I'm a kind of person who likes to hack on projects to learn things. If I am learning a new language, I'll think of a problem that I want to solve. After reading some syntax I'll start hacking on a solution.

I have decided to learn distributed systems.

I think this weekend would be a good start to hack on things. I can continue later in the week I understand I won't pickup distributed systems over a weekend.

What resources do you recommend for one trying to learn distributed systems on their own?

[jxub]

I'd recommend the book Designing Data Intensive Applications, by Martin Kleppmann and as for online resources, check out http://book.mixu.net/distsys/single-page.html for a throughout overview.

[gyani95 → jxub]

Thanks for the suggestion. I've added it to my list. Do you recommend some good first apps for learning distributed systems?

[daoran]

Beside Martin Kleppmann's wonderful book, I'd also recommend MIT 6.824 distributed system course, you can find lots of resources on the course website.

[cottonseed]

You might find some of these references useful:

Resources for Getting Started with Distributed Systems https://caitiem.com/2017/09/07/getting-started-with-distribu...

Distributed systems theory for the distributed systems engineer http://www.the-paper-trail.org/post/2014-08-09-distributed-s...

Distributed Systems for Fun and Profit http://book.mixu.net/distsys/

Notes on the Theory of Distributed Systems http://cs-www.cs.yale.edu/homes/aspnes/classes/465/notes.pdf

[simon_acca]

Aphyr, of Jepsen fame, published an outline for a distributed systems class that makes for a good reference on what is relevant for most modern distributed systems: https://github.com/aphyr/distsys-class

[gyani95 → simon_acca]

Thanks for this! This sounds lovely and is 12-16 hours long. Fits my schedule :D

[iampims]

This Github repo is a pretty good start to get familiar with distributed systems:

https://github.com/donnemartin/system-design-primer

[gyani95 → iampims]

I've been through this. It's a good starting point and can be fairly useful for interviews. It gives a good high level overview of systems, enough for you to talk about them.

Maybe I can go through this again and dive deeper into topics that interest me.

[convolvatron]

do you want to learn how to design distributed systems from first principles or build systems from pre-existing distributed components?

unfortunately these are both called 'distributed systems'

#1 is alot more interesting. #2 is what people are hiring for

[gyani95 → convolvatron]

I'd like to say both! I guess I can get started with the second one and then move onto the first one?

Maybe after playing with a few components I can look into their internals.

[dsnuh → gyani95]

For #2 - Maybe try setting up your own Kubernetes cluster on a cloud service provider, without using shortcuts? Perhaps etcd and the Raft protocol would be interesting and helpful in some cases, but leans more towards #1.

For #1 - CAP Theorem, read through aphyr's blog, google concepts/details/jargon you find in white papers on different of the various distributed consensus protocols, etc.

[lambda_func]

Some stuff you can build to learn more about distributed systems (you can start simple, do research, and then iterate to build a more comprehensive solution): build a distributed file system, a distributed lock server, a consensus algorithm (paxos, raft, etc.), two/three phase commit protocol, distributed hash table, etc.

[srkiranraj]

MapReduce paper, though published in 2004, it is foundational, relevant and very well written paper.

https://ai.google/research/pubs/pub62

P.S: it has close to 25k citations.

[brudgers]

Erlang?