
Elixir in Depth – Reading and personal notes - rudi-c
http://digitalfreepen.com/2017/08/16/elixir-in-depth-notes.html
======
randomstudent
The author doesn't talk a lot about preemptive scheduling, which is probably
the best thing about the Erlang virtual machine. This video explains what
preemptive scheduling is and why it is extremely useful:
[https://www.youtube.com/watch?v=5SbWapbXhKo](https://www.youtube.com/watch?v=5SbWapbXhKo)

First, the speaker creates an application endpoint that runs into an infinite
loop on invalid input. Then, he shows how that doesn't block the rest of the
requests. Using native BEAM (EDIT: BEAM = EVM = Erlang Virtual Machine) tools,
he looks for the misbehaving process (the one running the infinite loop),
prints some debug information and kills it. It's pretty impressive.

Another great (and lighter) resource, is "Erlang the Movie"
([https://www.youtube.com/watch?v=xrIjfIjssLE](https://www.youtube.com/watch?v=xrIjfIjssLE)).
It shows the power of concurrency through independent processes, the default
debug tools, and the power of hot code reloading. Don't miss the twist at the
end.

~~~
KallDrexx
Preemptive scheduling is great, but there are some times when it is bad. For
example, I was having problems in my Elixir code for live streaming that it
was taking too many reductions to fully deserialize and process the incoming
message (nothing major was in this code path, just a parsing the TCP packet
and unwrapping the different layers). This ended up causing the process to get
(what seems to be) overbalanced and I ended up at having delays transferring
5+mbps video through it.

I broke the deserialization up into 3 separate processes (1 for handling the
I/O, 1 for packet deserialization, 1 for message deserialization) and while
this fixed all the issues (presumably because it can balance the processes
more effectively since no single process went over the reduction limit) it
turned the effectively synchronous parsing process into a split async process
and caused a lot of extra complexity around that.

I'm sure there's more to what caused the issue than just reduction limits but
it also made me wary about the magic happening under the hood.

~~~
bitwalker
You may have been able to bump the process priority to give that process more
scheduler time if it's a critical process in a hot path. Perhaps you tried
that though. Definitely have to be careful with it, but always an option to
keep in mind!

~~~
KallDrexx
I haven't tried that, but I'd have to make sure that every inbound and
outbound RTMP connection had high enough priority to correctly execute, and
that seemed like a lot of tweaking that had to be done in order to maintain
the constant flow of video throughout the app with low latency.

At least for my operation it looked very much like I would really need to
become a BEAM vm expert to make sure things ran smoothly (not just with that
but other potential issues I had in the back of my mind as well).

~~~
randomstudent
> At least for my operation it looked very much like I would really need to
> become a BEAM vm expert to make sure things ran smoothly

From someone with zero experience in building concurrent systems: isn't this
true for every sufficiently complex system? If your system is complex enough,
then no general framework will help much, right?

~~~
KallDrexx
At some point sure, but I can get concurrent systems up and running pretty
easily in Go, C# and Rust without going too deep into the underlying details
of the asynchronous framework. Erlang's VM does a lot more than any of those
frameworks or runtimes do though because of how it handles per process garbage
collection yet globally shared binaries, the preemptive scheduler and how it
balances process load, the hot-reload capabilities etc... It does a lot of
things really well but there's a lot of magic behind it, and I seemed to brush
against that much faster than I have in other languages.

------
jesses
The author says it's "hard to use Elixir with services like Heroku because
your instances won’t find each other by default (the way they’re supposed
to)".

I just wanted to mention [https://gigalixir.com](https://gigalixir.com) which
is built to solve exactly this problem.

Disclaimer: I'm the founder.

~~~
skrebbel
We're considering to become your customer, but currently our backend isn't
only Elixir but also a nodejs app that mostly prerenders our React app. How do
you suggest people combine Gigalixir with non-elastic services or, say,
databases?

~~~
jesses
For databases, you can provision one through Gigalixir, or you can provision
one yourself with Google Cloud SQL in the us-central1 region.

For other apps, anything running in GCP us-central1 will be in the same high
latency network, but some customers interop with AWS or Heroku just fine
albeit with slightly higher latencies.

~~~
skrebbel
Thanks, clear! Any chance for a EU-based region soon? Most of our customers
are in the EU and some are required to have their data inside the EU at all
times for legal reasons.

~~~
jesses
I actually get requests for EU-based region quite a lot. I think I'll have to
start working on that soon, but I can't make any promises.

------
acconrad
All of my pet projects are run on Elixir/Phoenix. If there is a
language/framework to learn, this would be it. As approachable as Rails with
the benefits of handling real time applications (chats are trivial with
Channels) and faster runtimes than Rails (by orders of magnitude).

Happy to help anyone out if they're interested in learning!

~~~
christophilus
Serious question, though I know it will sound negative... I _really_ don't
like Rails and don't like Ruby or VB.NET, either (syntactically), which means
Elixir is off-putting for me, at least in how it's been marketed. Do you know
of anyone like me who has come to Elixir and liked it?

Note: I like Clojure, F#, and other functional languages, so that's not a
hurdle.

~~~
rlander
I also come from Clojure and Erlang and, tbh, couldn't get into Elixir for
various reasons (I've followed the development closely since its inception):

* Most of the Elixir community comes from the Rails world, so most of the libraries focus on the web.

* Coming from the Ruby world, a lot of these new developers tend to be ignorant of the underlying Erlang machinery, believing that Elixir is the magic that makes Erlang a "modern" and usable language, which is simply not true.

* The tendency to favor frameworks over libraries.

* I'm probably alone in this one, but I find the syntax off-putting. It adds lots of sigils and complexity to Erlang's very succinct syntax.

* In the end, I think Elixir adds very little besides the excellent tooling to the Erlang ecosystem.

~~~
christophilus
You're definitely not alone in disliking the syntax.

I agree with most of your points, from what I've seen, except the last one.

One thing that Elixir seems to get right is a sane string implementation out
of the box.

~~~
passer-by-123
Not only strings, but better error messages, the new breakpoint support,
functions for working with collections, protocols, tasks, etc. It goes quite
beyond tooling.

------
sjtgraham
I don't think the Gartner hype cycle applies to Elixir, and I think that is
largely because it is built on top of a very mature, production-tested
platform (Erlang OTP). I have been using it in production for almost two years
without issue on [https://teller.io/](https://teller.io/), so if the GHC
applies, it's very elongated!

~~~
Exuma
So you use Elixir on your finance app? I was wanting to use Elixir on a
payment processing app... any details with your experience running in
production you care to share?

~~~
vforgione
Seconded. I've been trying to find good write ups on production deployment
best practices, especially people who have used it commercially for over a
year.

------
brightball
The only part of that article I'd clarify is around deployment and devops best
practices.

You can deploy Elixir exactly the same as any other language. In some cases,
it just means making a decision that you don't need some of the extra features
that are available like hot reloading...which every project doesn't need.

You can still use immutable images and take advantage of the built in
distributed databases by using mounted block storage if need be.

You can use everything out there. Early on figuring out some of the port
mappings and how to handle things was more difficult but as far as I've seen,
those problems have mature solutions all around now.

~~~
sanderjd
I'm less interested in what you _can_ do, and more interested in which
techniques have emerged (or are emerging) as the best practices. For instance,
if I want both hot reloading and deployment on AWS, what's the best way to
configure and run this? Etc. Are you aware of any devops-for-Elixir focused
resources?

~~~
brightball
The only reason I say that is that every project doesn't need hot-reloading.
It's one of those things that is only recommended where you need it.

Just think of it in the same way that you don't need every available jar file
for a Java project. The capabilities of the system are expansive, but not
always called for.

Distillery is the main deployment related library out there for Elixir and
reduces hot reloads to a an argument on the build.

[https://github.com/bitwalker/distillery/blob/master/docs/Wal...](https://github.com/bitwalker/distillery/blob/master/docs/Walkthrough.md#building-
an-upgrade-release)

~~~
KallDrexx
Hot reloading also adds a ton of hidden complexity to your projects, since you
essentially have to be very careful about every module boundary (as my
understanding is that every time code goes from one external module to another
non-OTP module it uses the new version of that new module, and thus any non-
backwards compatible signature change will crash the process). You can't
really casually support hot reloading and need to fully commit to it, unless
you don't care if things fail when you hot reload at which point you might as
well just do a full reload and keep confidence in the system.

------
palerdot
If someone is on the fence to make a jump to the elixir world I recommend
elixir koans - [https://github.com/elixirkoans/elixir-
koans](https://github.com/elixirkoans/elixir-koans)

I have not started looking into phoenix as I'm still exploring elixir, but I'm
happy to have started learning elixir with koans along with official elixir
guide.

------
sergiotapia
Great write up! Guys, if you're on the fence about learning Elixir - dive on
in!

You won't be disappointed and you'll be surprised how many times you'll want
to reach out to auxiliary services and find out "Oh I can just use
ETS/OTP/GenServer/spawn".

------
esistgut
I don't like the debugging workflow described in the linked article "Debugging
techniques in Elixir", it reminds me of DDD: a separate tool not integrated
with my main development environment and requiring extra manual steps. I
tested both the Jetbrains plugin and the vscode extension, both failed
(unsupported versions, bugs, etc...). To elixir users: what do you think about
the state of debugging tools? What is your workflow?

~~~
red_bikeshed
Well you have the basic IO.inspect/2 that you can use for "printf" debugging.
It works really well with the pipe operator because it prints the structure
and returns the structure itself not the string representation, so you can do
stuff like:

some_function() |> IO.inspect |> some_other_function

Also with Elixir 1.5 (OTP 20) you can set breakpoints easily from within IEx.

There is also IEx.pry where you can set the "breakpoint" in the code itself.

There is the observer tool where you can inspect each process and see numerous
information that can be useful. When in IEx just type :observer.start and try
it out yourself.

I suggest reading Designing for Scalability with Erlang/OTP, particularly the
chapter about Erlang/Elixir tracing facilities that can be really useful when
debugging live production systems.

------
tschellenbach
Yes Elixir handles concurrency better than Ruby. In terms of raw performance
it's nowhere near Go, Java, C++ though. Rails/Django are fast enough for
almost all apps, if you need the additional performance improvements of a
faster language you'd probably end up with one of those 3. Wonder how much
need there is for a language that takes the middle ground in terms of
performance. Looks very sexy though, really want to build something with it :)

~~~
pdimitar
> _Rails /Django are fast enough for almost all apps..._

Absolutely not true for Rails. In an Elixir app with Phoenix my response times
are anywhere from 0.2 to 12 ms while the Rails app _that uses the same
database_ has response times anywhere from 300 to 2000 ms.

(Before you ask, I did went the extra mile to rewrite one of the small Ruby on
Rails apps from my job to Elixir & Phoenix and maintain it as a feature-
complete clone, just so I can have objective data.)

If we compare web app stacks, Elixir & Phoenix app is orders of magnitude
faster than Rails in particular and this is not an exaggeration -- can't talk
for vanilla Ruby though, I've had positive experiences with mid-sized Sinatra
apps; but they still weren't capable of more than 20-30 requests/sec if you
are looking for consistent throughput.

I truly like Go but to me it has always been a much better choice for highly
performant microservices. I tried 3 separate web frameworks 12-16 months ago
and I simply concluded Go isn't a good fit for a full web app -- at least
compared to the conveniences and dozens of other advantages of an Elixir &
Phoenix app. Go is definitely better in several other areas.

TL;DR -- Rails is very far from "fast enough". It's very okay for internal
apps used only in business teams where the load won't ever be more than 20
requests _a minute_.

------
bitwalker
I think the discussion around deployment may have been unnecessarily tainted
by their experience using edeliver - it's an automation layer for building and
deploying releases, but as mentioned it is configured via shell scripts and
because it does a lot, a lot can go wrong.

The basic unit of Elixir (and Erlang for that matter) deployments is the
release. A release is just a tarball containing the bytecode of the
application, configuration files, private data files, and some shell scripts
for booting the application. Deployment is literally extracting the tarball to
where you want the application deployed, and running `bin/myapp start` from
the root of that folder which starts a daemon running the application. There
is a `foreground` task as well which works well for running in containers.

My last Elixir gig prior to my current one used Docker + Kubernetes and almost
all of our applications were Elixir, Erlang, or Go. It was extremely painless
to use with releases, and our containers were tiny because the release package
contained everything it needed to run, so the OS basically just needed a
shell, and the shared libraries needed by the runtime (e.g. crypto).

My current job, we're deploying a release via RPM, and again, releases play
really nicely with packaging in this way, particularly since the boot script
which comes with the release takes care of the major tasks (start, stop,
restart, upgrade/downgrade).

There are pain points with releases, but once you are aware of them (and they
are pretty clearly documented), it's not really something which affects you.
For example, if you bundle the Erlang runtime system (ERTS) in a release, you
must deploy to the same OS/architecture as the machine you built the release
on, and that machine needs to have all of the shared libraries installed which
ERTS will need. If you don't bundle ERTS, but use one installed on the target
machine, it must be the same version used to compile your application, because
the compiled bytecode is shipped in the release. Those two issues can
definitely catch you if you just wing a deployment, but they are documented
clearly to help prevent that.

In short, if there was pain experienced, I think it may have been due to the
particular tool they used - I don't think deployment in Elixir is difficult,
outdated, or painful, but you do have to understand the tools you are using
and how to take advantage of them, and I'm not sure that's different from any
other language really.

Disclaimer: I'm the creator/maintainer of Distillery, the underlying release
management tooling for Elixir, so I am obviously biased, but I also suspect I
have more experience deploying Elixir applications than a lot of people, so
hopefully it's a wash and I can be objective enough to chime in here.

~~~
pdimitar
Extremely valuable and informative text, made even better in the end by the
fact that you're using the best (IMO) Elixir deployment solution!

Thank you!

------
innocentoldguy
I don't know why, because I don't think they are all that similar, but I'm
often asked to defend my choice of using Elixir professionally vs. Go. From
the article, this is one of the big reasons I chose Elixir over Go:

"Go’s goroutines are neither memory-isolated nor are they guaranteed to yield
after a certain amount of time. Certain types of library operations in Go
(e.g. syscalls) will automatically yield the thread, but there are cases where
a long-running computation could prevent yielding."

goroutines also take up about 10 times the memory.

~~~
Rapzid
It's true, but would very rarely come up in practice; it's an extremely
simplified statement. Any function call(that isn't inlined) is an opportunity
for the scheduler to preempt the goroutine. I has been described as partially
preemptive.

I thought an erlang process takes up at least 309 words of memory? That would
make it <4x on a 64 bit system?

------
Exuma
This is a really good write up, thank you.

~~~
brightball
I'll second that. Extremely in depth.

~~~
rudi-c
Thanks! Much appreciated.

------
RobertoG
That was a nice article. Thanks.

I'm curious about the first table in the "Interop with other systems" part.

It seems to say that an Erlang deployment doesn't need Nginx or a HTTPServer,
anybody knows how that works?

EDIT: I read the cited source ([https://rossta.net/blog/why-i-am-betting-on-
elixir.html](https://rossta.net/blog/why-i-am-betting-on-elixir.html)) and it
seems that is the case.

It looks too good to be true, yet. It would be nice, if somebody with Erlang
deployment experience, could comment.

~~~
devmunchies
Wait a minute. In the graph in the article you shared, it says that instead of
storing persistable data in Redis/MongoDb, you would use Erlang. Do people
seriously not use a database in Erlang?

~~~
andy_ppp
You have persistent processes which you can send and receive messages from
which can be treated like a data structure store, as well as ETS if you need
more advanced querying and Mnesia if you need embedded persistence.

I'm not sure how these compare to Redis in terms of performance/latency but
Mongo is more of a "proper" database than any of these.

~~~
devmunchies
Ok. I've used Elixir and read Thomas' _Programming Elixir_ , and from the
little exposure I got I knew it was normal to use it as a kv store but still
use something like postgres for a traditional db. I did a double take when I
saw Mongo in the table.

------
gfodor
One thing I didn't see covered that I'm currently trying to understand with
Elixir is the relationship between process state and disk-backed state. (For
example fetched via Ecto.) Does the role of a traditional RDBMS change in an
elixir system? What are the durability guarantees of process state? Etc. Any
real world experience would be super helpful to hear about.

~~~
pdimitar
If you don't do a `kill -9 <your_app_pid>` then _any_ of your processes inside
will be restarted with last-known good state which the Erlang VM (BEAM) keeps
internally. It's not backed by disk state. It's a VM-wide guarantee, so to
speak.

The official Elixir tutorial has an OTP chapter in which you are encouraged to
actively kill processes and observe them being restarted with good state in
real time:

[https://elixir-lang.org/getting-started/mix-otp/dynamic-
supe...](https://elixir-lang.org/getting-started/mix-otp/dynamic-
supervisor.html)

------
brudgers
I can see why a person might choose Elixr over Ruby or vice versa. The
tradeoffs between Elixr and Erlang are a lot less clear to me.

