I'm hoping that this trend continues because out of all the noise of new languages and frameworks I feel like Erlang/Elixir are very deserving of becoming the new way forward for developing anything that runs on a network.
It is a super exciting language to learn and those moments where the ball finally drops on concepts like pattern matching are glorious, it has reignited that magical feeling of being some sort of tech wizard that I first got when I started programming as a child!
Anyway, enough of the gospel of Elixir, I too look forward to seeing how it/Erlang develop and feel like it is already getting some noteworthy traction.
P.S. I also scan HN daily for anything Erlang/Elixir, and frequency of posts seem to be increasing.
* 'event driven' things like Node.js got popular because they use fewer resources to serve the same amount of data (generally). But for a lot of people, JS is not really their idea of a good time. Erlang (and Go) fill this niche pretty well.
* Elixir finally made Erlang more palatable to more people. Jose has done a superb job with it, because it's not just a nicer syntax, there is a bunch of nice stuff he's built into it. It doesn't hurt that as a former(?) member of the Rails core team, he has a deep understanding of web programming and its requirements.
* It is solid, solid tech. There's lots of new development happening on top of, say, Elixir, like Phoenix, but the underlying system is pretty hardened.
However among all the languages which run on top of Erlang virtual machine (BEAM) like LFE, Elixir, Efene, Luerl, Erlog, and such the Elixir has an active and bigger community, more interesting frameworks and wider adoption. So it could be claimed that it helped to introduce Erlang values to a wider range of developers.
in case of high level platforms, languages or libraries
it can be claimed that Erlang virtual machine is almost
unique because JVM threads depend on operating system
schedulers, CAF which is a C++ actor library uses
cooperative scheduling, Golang is not fully preemptive
and it also applies to Python’s Twisted, Ruby’s Event
Machine and Nodejs.
Implementing NIFs should be done with the same care you would use adding a new machine instruction to your processor ;-)
However when there is no other options except writing a NIF, there are ways to protect yourself:
1. Your NIF should return less than a millisecond.
2. If the item 1 is not possible, split it into shorter NIF calls.
3. If item 1 and 2 are not possible so you have a dirty NIF. It is a NIF that cannot be split and cannot execute in a millisecond or less. There is an experimental feature in Erlang virtual machine which is called "dirty scheduler". When it is enabled some other schedulers are ready to execute the dirty NIFs, so they won't interfere with the normal operation of schedulers.
4. If item 1 and 2 are not possible and you don't want to use dirty schedulers, the +sfwi emulator flag is available to force normal schedulers to wake up again from the collapse situation.
These items are some solutions to remain in normal scheduling state even in case of writing the native functions in C (NIF), but what the article says is about just Erlang code which is run by schedulers and are preempted with no trouble as soon as they reach the reduction limit.
A short note on how hard this is: Until recently in 18.0, there were many BIFs (built-in functions that are part of the VM) that could possibly cause the same scheduler collapses. If the VM developers don't always get it right, the chances that some C code will, is very small. Tools like QuickCheck can help in testing the inputs and outputs but it's hard to setup complex VM stress states and thus very hard to make guarantees about NIFs.
While there is definitely overhead, I'd say regular external "ports" (OS-level subprocessing) are quite underrated from what I see in more recent Erlang code. There's a lot that can be done this way if port communication is carefully designed.
Great article though!
I see people down thread repeating this claim. Can anyone explain how the scheduler is preemptive?
So in other words, a task (i.e. process) doesn't have to cooperate with others. It can do whatever it wants, be it I/O or CPU bound, and doesn't have to yield explicitly. Due to aggressive "preemption" it won't block other pending tasks significantly.
In contrast, if this is not guaranteed (which is not in most other Erlang "alternatives" I've looked at), then a task might accidentally paralyse large portion of the system.
I personally feel that if someone made a Pythonic version of Elixir, say "Hydra", it would take over the world. If anyone starts such a project, look me up!