
Building Resilient Services with Go - GarethX
http://blog.fogcreek.com/building-resilient-services-with-go-tech-talk/
======
davidw
There's a lot there that looks familiar there to the Erlang developer, where
many things are built in.

~~~
seiji
systemd + Docker + Go is basically re-creating Erlang from the ground up,
except half broken and not learning from Erlang's 20-30 year history. Now
we're training people in the _wrong_ ways to create maintainable distributed
systems even when we know better ways to do things. I cry for our industry.

It's been said Greenspun's 10th Rule, when applied to distributed systems,
results in Erlang.

~~~
nvader
Honest question, from one who doesn't want to re-learn the lessons of the past
and who would really appreciate a 2 to 3 decade boost.

What are the lessons of Erlang, and how do you abstract as much of that as
possible away from the language/ecosystem to others? I'm looking for an answer
a little more accessible than "you have to learn Erlang to find out."

~~~
rdtsc
Here are few lessons of Erlang (mostly stolen from Erlang Factory talks, books
Learn You Some Erlang, Joe's, etc).

* L1: Today we are stuck building distributed and highly concurrent systems. We don't have a choice. Single CPU, single machine, where speed doubles every 18 month era is in the past. Because of the internet and lot of data most systems today are distributed. (How many startups do you know that ship a standalone desktop program these days?)

* L2 : Distributed and concurrent systems to be useful have to be fault tollerant. We don't want a segfault or panic caused by one corner case, triggered by one client, to bring down the rest of the server and the rest of the million connected clients. That would be a horrible page/phone call to get at 4am

* L3 : For the systems to be tollerant they have to be built out of isolated components, such that when they fail that failure doesn't spread through-out the whole system.

* L4 : Isolation can be achieved in a few ways:

\- A runtime system that prevents sharing memory -- OS processes do this.
Erlang's VM does this as well, except you can have millions processes on a
single machine.

\- Proving memory won't be shared. Rust's compiler can do this. It can prove
at compile time that you won't have data races between your threads. Rust is
the best and most interesting new thing in languages in the last decade
probably.

\- Running in a container, VM or a different machine. At the end of the day of
course, if your service is running on a simple (non-mainfraim-y) single
machine, it is not fault tollerant.

* L5 : These isolated concurrent units also send messages to each other to communicate (instead of say reading from a shared memory with a mutex thrown in there some place). These units are often referred to as "Actors" and there is a whole class of frameworks, libraries that implement that besides Erlang (Akka, Orleans, etc.)

* L6 : Erlang in addition to these basic blocks also comes in with:

\- Functional programming approach. Your data and variables are immutable. So
it is easy to look at a piece of code and understand what is being changed.
State updates are very explicit. And that is on purpose.

\- A framework of patterns used to build/monitor/distribute these concurrency
units. This is called OTP.

\- Monitoring and debugging capabilities. You can connect to a running VM
node, inspect, trace, debug even live code update without stopping the system.

\- An decades long ecosystem and experience building these kind of systems.

* L7 : If you are afraid of non-curly braces syntax and do not like typing , instead of ; take a look at Elixir. It is a new language but runs on the Erlang VM.

Hopefully this helps!

~~~
davidw
That's a pretty good list. It'd be interesting to really expand that into a
lengthly article or short book or something.

------
nickpsecurity
Another interesting Go deployment. However, although not about that article, I
think HN readers might like this post by the same team on how they do code
reviews:

[http://blog.fogcreek.com/increase-defect-detection-with-
our-...](http://blog.fogcreek.com/increase-defect-detection-with-our-code-
review-checklist-example/)

I discovered it while reading the parent article. I think it's a good
checklist that will contribute to resiliency better than a language choice. Of
course, adding a language with strong type and memory safety to such a good
development process will certainly drive quality up from there. Efficiency,
too, as recent article indicate.

