
Elixir multiple processes: Basics - ck3g
http://whatdidilearn.info/2017/12/17/elixir-multiple-processes-basics.html
======
sbuttgereit
I'm in the process of learning Elixir and the observation I'm about to make
may be completely off-base. With that said...

Seems like the section on "When to use processes" is selling processes short a
bit. Certain kinds of state management would seem to call for processes to
either manage or even hold state... but processes (as I understand them) are
also key in designing for reliability. So I would think I may well want to
organize my code relative to processes as I'm also figuring out the
supervision trees and various failure scenarios. And yes, concurrency issues
as well. If I'm wrong on this, I'd be happy to be set straight.

Anyway, yes, the section I speak of does get to some of the other parts of
what I mention, but the emphasis on state management seems to distort and
unbalance the view of what you might want as a process.

[edited a touch for clarity]

~~~
macintux
You are correct.

That's something I've been a bit worried about with people coming to Elixir
thinking it's basically a faster/robust Ruby: are they absorbing the full
Erlang Weltanschauung?

~~~
udfalkso
(I had to look it up)

Welt·an·schau·ung ˈveltˌänˌSHouəNG

a particular philosophy or view of life; the worldview of an individual or
group.

\--

Personally I came for the faster/robust Ruby first and then over time
discovered (and am still discovering) the other powerful pieces. I think
that's ok.

~~~
macintux
Absolutely that's ok. Everything's a journey.

(I tend to get lazy about learning, so perhaps I doth project too much.)

------
nickjj
Maybe I'm thinking about this incorrectly but when it comes to web application
development and concurrency, the things I would typically want to run in a
separate process are very important tasks.

For example, let's say you're sending emails out.

In Rails, Flask, etc. you would typically offload this to a background worker
like Sidekiq or Celery. These are dedicated libraries that run in their own
process and handle job processing.

Both tools allow you to see if that job was successful or failed and deals
with retries. They also use Redis as a back-end so your state persists outside
of the application code.

If you just willynilly spawn a process and start sending stuff through it, how
do you ensure everything works as expected and what happens to all of your
state if the Erlang VM goes down?

I love the idea of doing this, but in real world practice, it sounds like you
would still need the Elixir equiv. of Sidekiq / Celery to handle ensuring
these spawned tasks are trackable, right?

~~~
sbuttgereit
I don't think so. First, just to make sure there aren't any points of
confusion... Erlang/Elixir processes are not OS processes ([https://elixir-
lang.github.io/getting-started/processes.html](https://elixir-
lang.github.io/getting-started/processes.html)). As such, we really can't
speak of processes in Erlang/Elixir and other kinds of systems on any sort of
equal footing. In Erlang, processes are much lighter weight than OS processes.
And you do expect them to go wrong... so Erlang has the idea of "Supervision
Tree" where there are Supervisor processes whose job it is to monitor other
processes and to manage their failure when it occurs. There can be multiple
Supervisors which influence one another, or not, as you design them too (thus
the "tree" bit). Naturally you plan for these sorts of failures in designing
what is/is not a process, what dependencies there may be, what any Supervisor
watches and how the Supervisors relate to one another.

Erlang/Elixir also seems to have one of the strongest availability and
concurrency stories out there on its own. Part of this comes from the
aforementioned built in assumptions around isolating and managing failed
processes, but the other is in terms of the relative ease of starting
processes as needed, and accessing the processes of other Erlang VMs (same
server or not). It's clear you have to architect and build things correctly to
take full advantage of these capabilities, but I think you can get to the
place you describe without necessarily taking on a bundle of different
applications for different purposes.

For example, for many of the use cases I think of Redis, Erlang offers a
number of different ways to deal with such task out of the box:
[https://blog.codeship.com/elixir-ets-vs-
redis/](https://blog.codeship.com/elixir-ets-vs-redis/) Again, you have to
think about how you architect things, but you can use the same toolbox for
much of the work.

I'm still new to Elixir/Erlang, but I've been studying capabilities and
architecture for several months now. I think there are sufficient differences
in Erlang's approach as compared to other more common stacks that it really
does benefit one to come at it fresh and not make very many comparative
assumptions. In my own learning I've found that to just get the basic feel for
what this Erlang/Elixir animal really is, that the Elixir side is more
approachable. The documentation tends to be a little more direct and tends to
be more beginner friendly. After I got the very basics down, I was able to
look at the Erlang side and understand really what they were trying to say.

~~~
auxbuss
Saša Jurić's Solid Ground talk is a lovely demonstration of Elixir's
processes:
[https://www.youtube.com/watch?v=pO4_Wlq8JeI](https://www.youtube.com/watch?v=pO4_Wlq8JeI)

(Saša is also the author of Elixir in Action; a highly recommend read once you
have your head around Elixir's fundamentals.)

~~~
sbuttgereit
Yep, that's one of the Elixir books I've read. Agree that it's pretty good.
Thanks for the tip to the video.

------
redshirt
Very cool. I'm adding this to my list of easy concurrency tools. The main
thing I like is the statelessness. That's where most people screw up parallel
programs.

Seems like there are many languages/libraries trying to make concurrency
easier to implement in practice. Most notably for C++ (my fav since I have to
use it for most of my work projects): Intel TBB (definitely the go to for most
things), RaftLib (saw at C++Now last year) is probably the easiest to
understand (same theme as this post, super easy concurrency for c++). Even
Java seems to make concurrency rather easy with it's thread pools and
relatively strait-forward synchronized sections.

~~~
lostcolony
Okay, taking a step back - this isn't just another concurrency tool. This is a
language built for concurrency.

Taking another step back - this isn't just a language built for concurrency.
This is a language built for -resiliency-.

Taking another step back - this isn't just a language built for resiliency,
this is a language built atop a VM built for resiliency -over 30 years ago-
(the Erlang VM, aka, the BEAM, which Elixir runs on).

Okay. Why does this matter? Well, the thing is, resiliency encompasses
concurrency and distribution, both. It prioritizes error minimization and,
more importantly, the ability to recover from errors. This isn't just a
try/catch; this is a "something you completely failed to even expect caused
things to fail in a way you can't even imagine, and the system still handled
it".

It achieves that via immutability, concurrency, and distribution. Ensure your
data is immutable, so that state has to be very explicit (it's not
stateless...a process has state. But it's very explicit state; as a developer
you can't help but handle it and be very aware of it). Ensure bad states are
dropped, and the system can recover the execution unit from a good state. If
it can't, allow a user defined subsystem to fail, as the intricacies between
the entire subsystem are implicitly stateful, and restart the whole thing from
a known good state. If even that fails, keep climbing the supervisor tree,
restarting larger and larger subsystems, until you restart the entire
-application-, assuming that the intricacies across subsystems have gotten
into a bad state, and again, restart from a good state.

These principles have been around a long time, but Erlang is one of the first
languages to put them into practice, again, over 30 years ago. There's been a
lot of time since to see they actually work, and to further refine them. The
difficulty with concurrency is not actually being concurrent (per your post,
there are a LOT of ways to implement concurrency); the difficulty is doing it
in a way that it behaves how you want it, even in the face of user's doing
things you don't anticipate, external resources doing things you don't
anticipate, your own code doing things you don't anticipate, etc. The design
decisions that went into Erlang focused on minimizing errors...in so doing, it
provides a way that most kinds of errors are handled transparently (from logic
bugs to actual machine failure), while making it much harder to do things that
it can't recover from (memory leaks are comparatively difficult to cause, as
are deadlocks, for instance).

To give you an idea, the first commercial product built with Erlang boasted
(famously) 9 9s of uptime. Meaning something on the order of ~30ms of downtime
a year. That includes planned downtime, visible errors, etc.

I've seen Erlang systems in production...even with a rather critical bug in
one, the system just -worked- for -years-, before someone noted an oddity in
the logs, dug into it, and went "Oh my God" over how severe the issue was.
But, again, Erlang's supervisor process just restarted it, and it was never
noticed.

I helped write CNN's current video ingest system in Erlang. It's been working
without issue for years, despite no maintenance or attention (to where even
most of the developers have left, but it still just...works). Even much less
complex Ruby, Java, and Javascript systems are plagued with constant bugs. I
would not say the devs on this project were just that much better (though the
process was a little different, with little product owner involvement), but
that the language we picked was so much better geared toward fault tolerance.

