
What's all this fuss about Erlang? (2007) - krat0sprakhar
https://pragprog.com/articles/erlang/
======
chrisseaton
> Your Erlang program should just run N times faster on an N core processor

But only if your program is embarrassingly parallel with at least N times
available parallelism in the first place! If you have one of those it's
already trivial to write a version that runs N times faster on N cores in C,
Java, multi-process Python, whatever.

If your program has sequential or less parallel phases or needs to communicate
then you are subject to Amdahl's law like you always were.

Armstrong has that claim in the Erlang book and I was gobsmacked to see it
written down with no caveat or mention of the limits from Amdahl's law
whatsoever. I was sure it was a joke and it would be followed by 'ha ha ha of
course not - nobody knows how to achieve that despite decades of intensive
research', but no it's a serious claim made with a straight face. Erlang's
solved it!

Erlang helps you write parallel programs... but only if you program is
entirely parallel in the first place.

~~~
devishard
>> Your Erlang program should just run N times faster on an N core processor

> But only if your program is embarrassingly parallel with at least N times
> available parallelism in the first place! If you have one of those it's
> already trivial to write a version that runs N times faster on N cores in C,
> Java, multi-process Python, whatever.

You've made two claims, one irrelevant and one false:

1\. _only if your program is embarrassingly parallel_ is irrelevant, because a
almost every program is embarrassingly parallel in Erlang. The language is
built around concurrency to the point that parts which wouldn't be obviously
parallel in another language are in Erlang. Further, slow hashes in crypto
have taught us that is actually quite difficult to make something which can't
be parallelized.

2\. _it 's trivial to write a program that's faster in X language_ I'm not
sure how toot define trivial, but I've yet to find a language that can
communicate between threads as performant-ly. Even languages like Clojure
which use similar thread semantics can't do what Erlang can because the
underlying threads aren't as lightweight. Spinning up a million threads in
Erlang isn't even unusual, whereas in any of the languages you mention it's
either crippling-ly slow (Python or Java) or very difficult to synchronize (C
most, but Python and Java aren't easy).

~~~
BeetleB
>only if your program is embarrassingly parallel is irrelevant, because a
almost every program is embarrassingly parallel in Erlang. The language is
built around concurrency to the point that parts which wouldn't be obviously
parallel in another language are in Erlang. Further, slow hashes in crypto
have taught us that is actually quite difficult to make something which can't
be parallelized.

OK. Not an Erlang user, but I can't let this statement go.

I've studied parallel numerical algorithms. Many/most of them _will_ involve
blocking because you're waiting for results from other nodes.

If you're saying Erlang has somehow found a way to do those numerical
algorithms without having to wait, then I'd love to see all those textbooks
rewritten.

Amdahl's Law reigns supreme.

~~~
aeturnum
I'm not devishard, but I parsed his statement slightly differently. He's not
saying that Erlang makes things parallel magically. Rather, he's saying that
Erlang forces tasks that /could/ be parallel to be parallel by default. Thus,
Erlang will tend to maximize the sections of your program that are run in
parallel compared to other languages.

~~~
BeetleB
Which would make the original commenter's point valid:

>Your Erlang program should just run N times faster on an N core processor

No, it won't. It will only be true for tasks that /could/ be
(completely/embarrasingly) parallel (as you say). Which is kind of circular.

~~~
brightball
Not really. Think about every object you'd have in Java that's being passed
around your system. Now imagine each of those objects are their own processes
and you're passing around references to them.

Just on that one case, you've taken huge chunks of a linear execution pattern
and parallelized it. Now make that your norm and amplify it to everything. Now
realize that the message passing allows this mode of operation to spread each
part of this workload over not only more cores but more machines across the
network.

And then realize that you can deploy updates to this codes individual parts
while other parts continue running without taking down the whole system.

Then you get Erlang.

~~~
BeetleB
None of what you said will prevent the need for waiting for the majority of
numerical algorithms.

No one's disputing Erlang's prowess at parallelism. What the critic in this
thread was saying was that you can only get Nx speedup on an N core processor
for a limited set of algorithms. Most parallel algorithms will not fall in
this category. Amdahl's law is a general truth - it doesn't matter what your
architecture/language is. There is nothing special about Erlang that will make
any parallel algorithm scale linearly with nodes.

------
rdtsc
I posted this in another thread on what I like about Erlang (I'll shamelessly
copy and paste it here):

* A better concurrency model. Lightweight processes vs callbacks and futures. Or even channels. I find processes as concurrency units maps better to problems I had to solve. So there is less impedance mismatch. That makes a huge deal in a larger project. (eg.: a user request as an isolated process vs a chain of callbacks). OS design got this right years ago -- think how most modern popular operating systems represent concurrency - an isolated process.

* Safety & fault tolerance. Processes are isolated. So you can build very robust systems. That simply puts money in your pocket just due to ops costs. A non-critical / experimental part of backend is crashing at 4am and restarting? No, problem, keep sleeping. Rest of the service will stay up and you can fix it in the morning.

* Debuggability and inspection. BEAM VM comes with built in ability to debug, inspect and trace out of the box. That has saved so much time (and money) over the years.

* Hot code reload. This is a first class feature. We don't rely on it to do upgrades. But it proved invaluable to fix an issues or two for a high value customer without taking down the services. Or simply to add extra logging to monitor a pathological edge case.

Some people might prefer Elixir and that's fine, they are great friends. I
personally like Erlang, I think it is simpler. Someone new coming from Python
or Ruby might like it better and also it has very beginner friendly community.
Jose Valim and and other authors really put user-friendliness and approach-
ability at the forefront. I really like that, mad props to them.

------
slashdotdash
Dave Thomas has described Elixir as follows: "Elixir took the Erlang virtual
machine, BEAM, and put a sensible face on it. It gives you all the power of
Erlang plus a powerful macro system."[1]

[1] [https://startlearningelixir.com/elixir-for-
rubyists](https://startlearningelixir.com/elixir-for-rubyists)

~~~
mnd
Erlang is a great programming language after one takes the time to understand
the principles underlying it and its design. Everything fits very nicely into
place. There are definitely things that could be done better, but then that’s
the case with most programming languages and technologies after they have
accumulated some dust. Elixir on the other hand, in my personal opinion, is a
false prophet simply because it looks like something which it isn’t. It looks
like Ruby and gives the programmer the feeling that he’s right at home, except
it is in fact a very different beast with very different semantics, Erlang
semantics, as opposed to Ruby. Therefore, if you want to program on the BEAM
(the Erlang virtual machine) you better make sure you actually understand the
system, its design, principles and also some of the semantics of Erlang
itself, point at which you might as well just learn Erlang. I’m not saying
that you cannot learn those things coming from Elixir, I’m just saying that if
you want to build systems of reasonable complexity (like the ones that Erlang
is known for) on top of the Erlang virtual machine, as opposed to CRUD web
applications, then you must understand a lot more than just Elixir or Phoenix.

~~~
digitalzombie
Elixir is prettier than Erlang but it really mess up on certain things.

Erlang have pattern matching via function with same name and you can tell if
it's a group of pattern matching with semicolon and period. But with Elixir
you can't tell it's just def and end.

Erlang's:

-module(recursive).

-export([fac/1]).

fac(N) when N == 0 -> 1;

fac(N) when N > 0 -> N * fac(N-1).

You can tell fac(N) both are in a group of pattern matching cause ';' and '.'.
The '.' denote the last pattern matching function.

Elixir:

defmodule Factorial do

    
    
        def of(0), do: 1
    
        def of(n), do: n * of(n-1)
    

end

You can't tell because there is no ';' and '.'. This is a trivial case but
when your Elixir's module have a tons of function in it, this issue become
relevant.

~~~
slrz
I think you make the Erlang definition look unnecessarily awful.

fac(0) -> 1;

fac(N) -> N * fac(N-1).

Or is there a particular advantage to write it using guards?

~~~
jknoepfler
having a semantically significant distinction between ; and . is unnecessarily
awful. That kind of crap is one of the reasons Elixir has so much support.

~~~
mnd
What you said is equivalent to a Python programmer saying that adding
semantical value to the “end” keyword in Ruby is unnecessarily awful because
you can achieve the same by giving meaning to the indentation level. It’s an
entirely subjective matter.

I would also like to point out that Erlang has also very good support and has
seen adoption in some very critical systems as opposed to CRUD web
applications which is the main domain of Elixir. Most of Ericsson’s products
use Erlang to a certain degree, there are a lot of banking systems and
aviation systems which make use of Erlang as well, quite a few Internet
companies use it to great success, and many more.

And by the way, the semantic meaning of “;” and “.” is an awful lot similar to
their use in the English language, you are blowing it out of proportions. This
is a trivial thing which you learn after a 10 minutes introduction to Erlang.
For me, personally, if one has a problem understanding the meaning of “;” and
“.” or learning a new syntax for that matter, I can easily conclude that I
probably should not give that person any decision power in designing systems.

------
lngnmn
A practical language based on well-researched first principles. Details in the
armstrong_thesis_2003.pdf

    
    
       * java is unsuitable
       * no pthreads, please (wrong concept)
       * immutable data (no locks - no problem)
       * isolated lightweight processes (share nothing)
       * communication by message-passing (grom Kay's OOP)
       * dynamic typing (good-enough, quick prototyping)
       * pattern matching (on receive)
       * supervision hierarchies (fault tolerance)
       * reusable components for server development
    

I probably forgot something

~~~
XorNot
I've not found immutable data to be the big win people tout it as. In most of
what I do acting on _stale_ data is just as bad, which means I wind up needing
a lock anyway.

Doing a < b and taking an action based on that result is as harmful with stale
data.

~~~
kqr
I'd like to tell you that stale data will always be a thing in any serious
application because light only travels so fast. Therefore, you'll need to have
sensible logic in the face of stale data either way, so it's not so bad as you
think.

...and while that's true, I'd still like to hear the opinion of someone more
knowledgeable because it's an issue that's been bugging me too.

------
davidw
This was back in a brief period when Erlang was being hyped by people as a
potential Next Big Thing. Unfortunately it never really got that much momentum
though, until Elixir and Phoenix came along.

~~~
derefr
I've always thought that Erlang would be more successful if it had a better
deployment story, along the lines of Go or Rust: e.g. if you had the option of
baking an Erlang runtime into a static binary. As it stands, software like
ejabberd and rabbitmq are pretty ugly in OS package form (because distro
maintainers think such packages need to rely on a common, usually very old,
Erlang runtime, instead of allowing them to vendor it in as you're supposed
to.)

Now that I think of it, Canonical's push toward "snaps" (sandboxed and
vendored—but not fully containerized—software bundles, as a package format)
might bring a flourishing of Erlang-based application software, given that
it's much more compatible with Erlang's style of release-management.

~~~
lobster_johnson
I think that would help, but in my opinion, Erlang's syntax and overall
projected surface has always been its biggest hurdle to wide adoption. You can
have the most amazing, powerful language ever, but it still won't receive
popular adoption if you're too strange and you can only attract a certain type
of highly-skilled senior devs.

I'm not a fan of Elixir's quasi-Ruby syntax — I think something closer to Go
or Nim would have served it better — but it's certainly a step in the right
direction.

The Erlang toolchain has a ton of ergonomic issues that could have been
polished down a long time ago, but haven't been. Erlang simply _feels_ oddly
quirky and antiquated for anyone used to modern GNU tools or modern REPLs.

For example: Lack of Readline support (needs rlwrap to be usable, at least on
Ubuntu), lack of GNU-style long options, the obnoxious ctrl-C behaviour that
doesn't respect Unix conventions and gives you a prompt with confusing options
(quick, what's the difference between "abort" and "kill"?) that seem aimed at
developers and are completely wrong if you're just a user of an Erlang tool.
(Couldn't they have relegated this to USR1 or something instead of INT, like a
normal program?) On the server, the epmd process is a thorn in the side of any
system administrator. Erlang devs also seem to think that emitting Erlang
stack traces is a good replacement for proper English error messages, and if
you're not an Erlang dev, you haven't seen stack traces until you've seen the
kind of monstrous, obscure, deeply nested contextual dumps that Erlang
programs can produce.

And so on. Little things, but important things that can completely kill the
joy when you're a potential adopter.

Erlang shouldn't need Elixir to modernize, but there's probably very little
incentive within the community to change quirks that they're all used to
dealing with by now.

(I encountered similar issues with OCaml, which has many parallels to Erlang:
Quirky, odd syntax, antiquated toolchain, etc. Facebook's cleanup effort [1]
looks very promising.)

[1] [https://facebook.github.io/reason/](https://facebook.github.io/reason/)

~~~
derefr
Oh, sure, there are plenty of things getting in the way of developers being
_excited_ about using Erlang and thus deciding, in a bottom-up sense, to build
more stuff in Erlang.

But I think Erlang's use-case isn't really the type of software developers get
excited about developing, either way. The type of software Erlang is "best at"
(and where it would reduce codebase size the most) is exactly the type of big
Enterprise-y bloatware-apps and business-process servers that make Java's
market-share so large.

Because Enterprise software gets designed top-down, how _excited_ developers
are to be using the (dictated) language aren't very relevant. How easy the
language is to _learn_ is relevant; how many engineers you can either hire or
train to code in the language is relevant. But whether they enjoy their day-
to-day lives writing in the language isn't, really.

Thus, I think, why things like deployability are more important to Erlang: for
the type of software projects Erlang _would_ be good for, considerations like
deployability (and maintainability, remote debugging, easy hot-patching, etc.)
are what guide language choice. Erlang wins on most of those fronts (as it
should: its OTP framework is effectively "30 years of ops best-practices
constraining development"), but for various reasons, [non-embedded] deployment
has always been a pain-point.

~~~
lobster_johnson
But did it have to become niche? I think Erlang could very well have branched
out to become a more even general-purpose language, rather than doubling down
on distributed computing to the exclusion of more quotidian tasks.

Sure, Erlang will never be able to compete with certain languages such as C
and C++ for many use cases. But I often grab Go to create small command-line
tools, or do some minor parallel data processing where a lighter language like
Ruby will not do well. There's nothing in Erlang that _conceptually_ prevents
it from being a general-purpose language; it's just that its ergonomics don't
really "scale down" to the stupid, simple stuff.

I remember, years ago, trying to write a basic parallel non-OTP log-processing
pipeline for some log files, thinking Erlang would be ideal... and being
surprised at the number of roadblocks I had to deal with. Around the same
time, Tim Bray went through the same process, but spent a lot more effort on
it than I [1].

[1]
[http://www.tbray.org/ongoing/When/200x/2007/09/22/Erlang](http://www.tbray.org/ongoing/When/200x/2007/09/22/Erlang)

~~~
im_down_w_otp
I don't think it helps that the largest non-Ericsson commercial entity that
attempts to curate and bounding-box the community has a kind of identity
crisis around whether or not it's a services or products company.

For whatever reason they didn't do much in the way to create and maintain
modern tooling to make on-boarding easier, which would have facilitated
increased adoption, and lead to more revenue potential. Neither relx, nor
rebar, nor erlang.mk, etc. were born out of said commercial entity, despite
those kinds of things being _perfect_ candidates for a services company to
produce to make their own lives and the lives of potential new users easier.

There have been some _huge_ high-profile Erlang use cases that would normally
feed a Silicon Valley/Hacker News style hype-cycle, but somehow those came and
went with almost nothing to show for it. It almost feels like if Elixir hadn't
shown up on the scene to attract new users and splinter off a chunk of the
massive Ruby community that the Erlang side of the world would all but dead
already. Sometimes I wonder how it is one goes about blowing a 15+ year
technology lead and 10 years of huge and publicly visible wins, but that's
sorta what's happened.

In the curation of said community there's also been an overtone of nostalgia
for "the good old days" & "yeah, that part is _supposed_ to be painful", and a
sort of long-running tone deafness around on-boarding, usability, general-
developer-ecosystem-friendliness, etc. complaints that certainly could have
had a lot more done to be ameliorated by the commercial entities who stood to
gain the most by Erlang having broader adoption.

Frankly, much like Elixir breathing a bit of new life into aura around Erlang,
if it weren't for people like Fred, Tristan, Alisdair, Garrett and a few
others (who again aren't part of the core supports for Erlang commercially)
being really passionate against all odds about making Erlang easier to use,
and to explore new use cases, and to meticulously and engagingly document all
of it... Erlang would probably be dead already.

All of which might be overly harsh, but I'm having a very acute existential
crisis around Erlang at the moment. I love it. I love developing in it. I love
introducing it to teams and projects. And as time goes on I feel rapidly and
increasingly more guilty about that given the decline it seems to be in and
_has_ been in if I'm being honest with myself in retrospect.

Blargh :-(

edit: look at the increasingly dwindling sponsors list of Erlang Factory San
Francisco over the last several years as an indicator of the above rant/cry-
for-help.

~~~
lobster_johnson
Thanks for the insight. Out of interest, what company are you referring to in
your first paragraph?

~~~
im_down_w_otp
Erlang Solutions

------
kriro
I'm not well versed in the entire (phone) network space. Ericson switch uptime
(powered by Erlang) is always cited and admittedly initially drew me to the
whole ecosystem. Well mostly the "it's used in a network that has basically
worked all these years so it must be rather robust one way or another" line of
thinking.

However what I'm wondering is...there's other vendors and Ericson isn't
powering the entire network. What software stacks are used by other vendors?
I'm assuming it's mostly C-ish stuff with some ASM sprinkles like in Cisco
IOS?

~~~
hellofunk
Ericson makes a switch that some other networks use. That switch contains lots
of Erlang inside it.

------
B1FF_PSUVM
Incidentally, does anyone have a screenshot of the "all Erlang" HN front page
of similar vintage (perhaps 2009 or so)?

------
aaron-lebo
I think the "more cores == faster" benefit is overstated.

Without jit Erlang is so slow that languages without concurrency support will
kill it. It's even more overstated when something like Go can mostly scale as
well over multiple cores but be x times faster while doing so. Not to mention
it's 10 years later and we haven't seen massively parallel architectures take
off.

I suspect other languages are going to fold the good parts of Erlang/BEAM into
them and limit its long-term adoption.

~~~
stcredzero
_I think the "more cores == faster" benefit is overstated._

I've tried to create servers for massively multiplayer games in Clojure and
Go. What I've found is that it "takes some doing" to be able to efficiently
use many processors in parallel. You can't just slap on a pmap call in Clojure
or do the equivalent cheesy channel trick in Golang and have good utilization
of all your cores in parallel.

 _It 's even more overstated when something like Go can mostly scale as well
over multiple cores but be x times faster while doing so._

For what value of "mostly scale as well?" For me, that means doing a little
extra design/architecting to break out your Golang server process into
multiple worker processes.

 _it 's 10 years later and we haven't seen massively parallel architectures
take off._

Because there are still barriers that are a bit too large. You either wind up
having to do weird things to do what amounts to efficiently passing data
between cores, or you end up using a slower language that's a bit far from the
computing mainstream.

~~~
zazibar
Can you elaborate on how you implemented those servers? Would you still use
Clojure or Go for them now? If not, what alternative do you suggest?

~~~
stcredzero
My first attempt was a server for a dungeon crawler that superficially looked
like a rogue-like, but was actually 12 frames/sec realtime instead of turn
based. It started out on Clojure, but I afterwards ported it to Go. In this
server, the world was divided into 80x24 subgrids that did most of their
processing in parallel. Processing would happen in two stages: 1) local data
processing, where each entity got updated in a loop, sending out updates to
clients and 2) collision resolution/movement. If I had to do it over again, I
would relax the grid's invariant properties, and make each 80x24 subgrid
completely independent, which would eliminate the collision resolution step.

In my current server, I'm basically sharding my space-game into star systems,
with free travel between star systems though "Hyperspace." There is a farm of
"worker processes" that is coordinated by a master server. Any instance of a
star system can be idempotently spawned when a client is attached to it. <
[https://www.emergencevector.com](https://www.emergencevector.com) >

I think Go is a good option for writing a game server. You'd have to be pretty
profligate to make the GC pause overrun one 16.66 ms tick, and if you miss a
tick here and there, who cares?

~~~
protomikron
> In this server, the world was divided into 80x24 subgrids that did most of
> their processing in parallel.

Is there any reason why you do update() in parallel? There is certainly some
interesting challenges to solve there but did you have a reason to make it
parallel other than intellectual curiosity (e.g. better performance)?

Afaik a game loop's update() is not the intense part in the game loop (maybe
that's different for games with lots of world information update, like
roguelikes), but rather render() and IO() are so I am wondering what your
design thoughts are.

~~~
stcredzero
_Is there any reason why you do update() in parallel?_

I wanted to come up with an architecture that could support an arbitrarily
large world given enough resources. Up to a certain density (from 150 to 250
users per "subgrid") I was close to succeeding.

 _Afaik a game loop 's update() is not the intense part in the game loop but
rather render() and IO()_

"Render" in the context of a server like this is the IO. One thing you should
know, is that this game supported Conway's Life cellular automata at 12
frames/second, completely shared and multiplayer.

------
eternalban
Erlang's secret sauce is OTP.

~~~
yellowapple
OTP is what makes Erlang Web Scale™.

------
thinkling
Are there online classes (eg MOOCs) teaching Erlang? I looked at Coursera,
EdX, and Udacity recently and didn't find anything.

~~~
cxa
There are some good introductory videos at
[https://www.cs.kent.ac.uk/ErlangMasterClasses/](https://www.cs.kent.ac.uk/ErlangMasterClasses/).

------
KirinDave
For folks who like Erlang, also check out Pony.

------
soyiuz
How to make sure I don't read your post:

> Our site has been optimized for use with newer browsers. We also require
> that your browser has JavaScript enabled.

> It looks as if your browser has JavaScript disabled.

> This site has information about enabling JavaScript, if that's something you
> want to do.

Really? I am just trying read a few paragraphs of text on a fairly static-
looking site.

~~~
LukeShu
It's even worse than that. It is static. It would work without javascript if
they didn't put

    
    
          <noscript>
            <meta http-equiv="refresh" content="2;url=https://pragprog.com/no_js">
          </noscript>

~~~
LukeShu
As a slight followup: In the Advanced settings panel for NoScript, there's an
option to disable <meta> redirects in <noscript> blocks.

------
andrewvijay
"How do programmers fix these problems? They don’t. They pray." :D LOLed at
this!!

------
hota_mazi
> How do programmers fix these problems? They don’t. They pray.

Ah! No, Joe, that's what _you_ do.

Most of developers have figured out how to do concurrency and parallelism with
mutable state and locks. It's not as hard as you make it sound and it's
supported in a lot of languages which, as opposed to Erlang, are statically
typed, hence much more suitable to the kind of large scale software you
describe.

And yes, that was already true in 2007.

~~~
gamache
Do you realize how silly this sounds? You are disregarding not only Erlang,
but decades of research and development on concurrent programming best
practices. Immutability has been promoted for parallel programming since the
1970s. For the past fifteen years, even languages like C++ have moved to
promote this style of programming.

(Also, Erlang has a compile-time static type checker, if type safety is
something you're after.)

~~~
hota_mazi
I'm well aware of the literature.

The fact is that still today, we have millions and millions of lines of code
that work in parallel and a crushing majority of it is operating in mutable
languages (C++, Java, Javascript, C#, ...).

Pushing immutable languages as the silver bullet to solve concurrency problems
is trying to solve a problem that nobody has.

We've moved on to higher abstraction grounds (fork join, work stealing,
coroutines, etc...), Erlang is still stuck in the past.

~~~
toast0
I think immutability isn't really that important for concurrency in Erlang,
since there's conceptually no shared state between Erlang processes -- if you
use the mutable process dictionary, no other process could see it anyway[1].
In reality there is ets which is mutable with locks, and refcounted binaries
which become immutable once there's more than one reference.

That Erlang forces you to build a shared nothing system is the real win for
concurrency. Of course, you can do that with other languages too.

> We've moved on to higher abstraction grounds (fork join, work stealing,
> coroutines, etc...), Erlang is still stuck in the past.

Erlang can do fork/join, the schedulers can work steal, and erlang processes
are approximately equal to coroutines -- I don't understand what you think is
stuck in the past (other than the Prolog like syntax, but you get used to
that).

Immutability is a big win for garbage collection and memory allocation
however, since there are no circular references, mark and sweep isn't required
and a simple (generational) copying collector can be used instead.

[1] actually you can look at other processes' dictionaries if you want, but
that requires a lock on the destination process.

