
Erlang/OTP by Example - jxub
http://erlangbyexample.org
======
jlouis
If you get to the part about monitors, consider erlang:demonitor(Ref, [flush])
in your own code. This also removes the monitor message from your mailbox if
it arrived in between, which is a real problem in an async setting.

Though in some situations, it is better to just ignore the spurious message
when it arrives by tracking what monitors you have enabled in the process
state. Unkonwn monitors are just gracefully ignored. The same pattern is
useful with timeouts as well. Cancel the timeout, but if it arrives in your
mailbox while the cancel is happening, you can just detect an old timeout and
ignore it.

~~~
jhgg
demonitor flush can be harmful in some instances as it invokes a selective
receive to yank the down message from your process's mailbox. In the event
that you are demonitor+flushing on a process that has a growing message queue,
every flush gets more and more costly - especially if the monitor has already
delivered a DOWN message for it to yank.

it's almost always better to ignore down messages that you don't care about.

------
plainOldText
In addition, two other highly informative resources:

* [http://spawnedshelter.com/](http://spawnedshelter.com/)

* [http://beam-wisdoms.clau.se/en/latest/](http://beam-wisdoms.clau.se/en/latest/)

------
davidw
This is also a fantastic Erlang learning resource:
[https://learnyousomeerlang.com/](https://learnyousomeerlang.com/)

------
the_clarence
I'm kind of sad that Elixir is getting all the love and the cool resources
since I much prefer Erlang's syntax. Thanks for this page!

~~~
innocentoldguy
I love Erlang too, but I don't think Elixir's popularity is stealing love away
from Erlang. If anything, I think Elixir compliments Erlang and provides a
path for people to enter the world of Erlang, BEAM, and OTP that they didn't
have before. Elixir helped to lower the bar of entry. My understanding is that
the Elixir team has also helped drive some improvements in Erlang, which is
nice.

Personally, I love both Erlang and Elixir and hope to ride out the rest of my
career on these platforms.

~~~
richjdsmith
I agree wholeheartedly. Elixir gave me an opportunity to start to play around
with Erlang and OTP. While I prefer using Elixir, its ability to mix and match
with Erlang has made it a great gateway.

------
ilovecaching
As someone who has worked two jobs now writing, deploying, and operating
Erlang clusters, I recommend switching to Rust. Erlang requires a lot of TLC
to get right, it's super slow, and it's hard to burst. Like, super hard to
burst. Erlang nodes are meant to cluster as a k graph and never go down.
Modern ops, especially container ops, does availability through ephemerality
of services. The BEAM just doesn't like to be treated like cattle. Also Erlang
has notoriously bad error messages, very little abstraction, and converting
between binary strings and lists is a pain. Gaining Erlang operational
knowledge also takes a while. We eventually had to rewrite things like
gen_server, ditch mnesia etc. as we scaled.

So why Rust? Like Erlang, it's damn good at concurrency and enables functional
programming. It also enables event driven programming through tokio, which is
a better fit for web servers than green threads (you're mostly waiting on the
network). Unlike Erlang, it's super fast (even at math), has a great type
system, amazing error messages, low memory usage, and the community is already
quite a bit bigger.

~~~
sargun
I'm going to disagree with you on Rust. It's a very different, very verbose
language. It doesn't have any of the stories around immutability that Erlang
does.

You say you had to rewrite Mnesia, but Rust doesn't even have transactional
memory to start with.

Suggesting that event-driven programming is anything like language-level
threading like Erlang and Go is crazypants. A common error in event-driven
languages is that you end up writing code that gets slow, and blocks the
entire event loop, and everything falls apart, and you get paged at 2 AM,
until you add another event loop.

One of the best parts of BEAM is that since processes are isolated and
preemptively scheduled, you don't have to manage your own call-backs by hand,
and although things may get slow, they'll typically only get slow for that one
given process.

In addition to this, the GC in Erlang is great, compared the lack of GC in
Rust. I think most of us can agree that unburdening yourself of having to
memory management code is a good thing.

Of course, BEAM isn't perfect, after all, it's had no more as much investment
as the JVM, and CLR, but I believe its semantics are right for writing
_predictable_ , low-latency code.

Also, containers have nothing to do with ephemerality. Cluster management
systems which dynamically schedule containers may result in scheduling.

Erlang isn't really a dataplane runtime. Often times, you implement your
control plane in Erlang, and farm out your dataplane to something NIFs, ports,
or something else entirely.

You're right, disterl is a fucking mess. But, it's better than nothing, and
having to write your own IPC.

I suggest you read Joe Armstrong's thesis, or a History of Erlang for more.

~~~
ilovecaching
> I'm going to disagree with you on Rust. It's a very different, very verbose
> language.

Erlang is incredibly verbose. It has very few abstractions. Rust has a lot of
abstractions. I've rewritten a few Erlang projects now and Rust and I've been
able to come out with close to or under the same LOC (I always use specs
though).

> It doesn't have any of the stories around immutability that Erlang does.

let expressions are immutable by default. You specifically have to ask for
mutability and even then, the Rust borrow checker will always enforce a single
writer. Rust definitely has a story around immutability.

> You say you had to rewrite Mnesia, but Rust doesn't even have transactional
> memory to start with.

Which is good, because Mnesia is crap and I've had to deal with chucking it
out the window several times now. Rust has good library support for STM that
is completely optional.

> Suggesting that event-driven programming is anything like language-level
> threading like Erlang and Go is crazypants.

They're both models of concurrency, and you can achieve parallelism through
either. Oftentimes one is better than the other for a given task (usually
determined by the bottleneck), such as serving web requests bottlenecked by
IO.

> A common error in event-driven languages is that you end up writing code
> that gets slow, and blocks the entire event loop, and everything falls
> apart, and you get paged at 2 AM, until you add another event loop.

Most high performance web servers use event loops. See the paper "An
Architecture for Highly Concurrent, Well-Conditioned Internet Services" for an
overview. There are lots of issues with green thread models. See some of the
work done by Brian Cantrill for examples, and why it may be a bad idea to bake
them into a language.

> One of the best parts of BEAM is that since processes are isolated and
> preemptively scheduled, you don't have to manage your own call-backs by
> hand, and although things may get slow, they'll typically only get slow for
> that one given process.

I think you're caught on the idea that Rust is Javascript or Python. In Rust
you can parallelize an iter chain by changing one method call. You can also
use multiple event loop to dispatch to handlers. There is no one size fits all
solution to concurrency.

> In addition to this, the GC in Erlang is great, compared the lack of GC in
> Rust. I think most of us can agree that unburdening yourself of having to
> memory management code is a good thing.

Disagree strongly. See Steve Klabnik's latest posts on static garbage
collection in Rust for an enlightening take. Rust does have a GC, and it has
no runtime performance hit.

> Of course, BEAM isn't perfect, after all, it's had no more as much
> investment as the JVM, and CLR, but I believe its semantics are right for
> writing predictable, low-latency code.

Erlang isn't low latency. It has _predictable_ latency.

> Also, containers have nothing to do with ephemerality. Cluster management
> systems which dynamically schedule containers may result in scheduling.

This is pedantic. Any non-trivial container deployment will have to deal with
ephemerality. If you're replacing an Erlang cluster, it will be even moreso an
issue because you'll need some level of fault tolerance from the orchestrator.

> Erlang isn't really a dataplane runtime. Often times, you implement your
> control plane in Erlang, and farm out your dataplane to something NIFs,
> ports, or something else entirely.

NIFs are extremely dangerous. We've had critical bugs that have taken down
entire clusters thanks to NIFs. The architecture your describing is also
exceedingly rare. Most Erlang deployments are handling soft real time
workloads like routing chat, queue messages, web requests, that have no
language separation between control and data.

> You're right, disterl is a fucking mess. But, it's better than nothing, and
> having to write your own IPC.

In many cases it is better than nothing. It will take a huge amount of wasted
effort to fix some of the scaling issues I'm currently having with our Erlang
cluster.

> I suggest you read Joe Armstrong's thesis, or a History of Erlang for more.

I've read Joe's thesis. I'm assuming your point is that I somehow don't know
anything about Erlang despite having worked on it for years professionally and
attended multiple Erlang Factorys/given talks on the subject.

~~~
toast0
If you don't mind, can you give some more details on what you don't like about
mnesia? Everybody says it's awful, and it surely has warts, but I've seen it
scale ok to 512GB+ datasets in disc_copies tables, and it works alright with a
couple caveats.

a) Not sure if it changed, but mnesia startup was very brain damaged -- fixing
up the local data from disc, then throwing it all away to load from peers is a
lot of wasted time. It's much faster to remove all the local tables on disk
before starting the node so it short circuits to copying from the peers. Even
when it's faster, sending over half a terrabyte of data takes a while. Some
sort of persistent transaction logs for peers would be nice.

b) network partitions aren't fun at all

c) we direct mnesia read and write for a key into a specific process to
enforce serialization, and then we use dirty read/writes; so we skip all the
locking.

d) we've certainly patched a lot of things in transaction sending and
receiving over long distances, especially needed if your network isn't clean.

Anyway, thanks for your thoughts all over this thread.

~~~
ilovecaching
\- One big issue is that when there is a replication stream going between
peers it will bottleneck how many fragments can be updated.

\- Large tables make restarts really slow (have to read everything from ETS).

\- Table dumps cause spikes in load.

\- Uses a lot of RAM.

For net partitions, I just alert on it and fix it manually. Mnesia isn't bad
as a cache, bad as a database.

------
xq3000
Cool site!

PS: Content looks cropped on an iPhone 7 nonmatter how you resize it

~~~
Waterluvian
Can't read the left half of any code examples on Android chrome.

