Not mentioned in the link above but something super cool with this release is that
> CPU quotas are now taken into account when deciding the default number of online schedulers.
> Thus, automatically making Erlang a good citizen in container environments where quotas are applied, such as docker with the --cpus flag.
I'm currently learning erlang/elixir and I'm really enjoying the language constructs.
I had originally taken a Programming Language Paradigms class in college with racket, and I really didn't appreciate functional ideas(i.e. syntax is my excuse). I'm now a really big fan of functional language idioms.
I never understood why people rave so much about functional programming wrt Erlang/Elixir, when its functional programming is clearly only a means to an end (fast and safe message passing requires immutable data, which requires FP) and not a driving design goal in its own right.
I mean, unlike in typical hard FP languages like Haskell or Elm, mutable state is rampant in your average Elixir app, it's just spread out across many (global singleton) little processes. Only inside a process you're doing "true" FP but given how small the average process is in scope, in practice the only real big difference is that you can't to an `i++` style for loop. Oh no!
But once you leave the process boundary, and often even before it, all bets are off. The amount of Elixir forum messages I've read that go "you can't do X at that point, because Y hasn't completed yet" is nuts.
Eg you can't broadcast in a phoenix channel `join` because the channel hasn't been fully initialized yet. So you send yourself an :after_join message and do the broadcast in there. I don't know about you but to me this feels a lot more like C++ than like Haskell.
Or consider the library module Agent which is exactly identical in semantics to a global singleton variable in an OO/imperative language. It's just a blob of data that you can get or set.
Now, I don't think any of these are disadvantages. I did mostly C# and JavaScript before Elixir, so I'm used to the occasional mutable state flying around.
But I'll never understand that people like Elixir for being FP. You just get such a small subset of the usual advantages of FP that it feels like an implementation detail. There's lots of advantages, but freedom from thinking about state isn't one of them.
If you look closely enough at Haskell, you’ll realize that it also can have a lot of mutable state. Haskell just puts state into various monads and STM to make its functions pure. You can even spawn numerous isolated threads, all with their own state, and have them communicate with one another like you do in Elixir. It does provide a lot of structure and guarantees compared to your standard imperative language, but I assure you mutable state is still there.
To be clear, the Elm model of putting everything into a big tree and and transforming that at every user input is very unusual among even FP languages, and is not the model typically used in Haskell. This might be what you are thinking of.
Edit: I also want to point out that Elixir processes can be registered globally to act as singletons, but by default you can spawn any number of agents or other processes at runtime, meaning they are not singletons.
Actor frameworks with message passing, just like direct mutation, don't compose.
On Haskell, yes, you can just work off a thread and write shared buffers. But you are better off using STM or Async (these are a bit like futures), which compose, and you can write pipelines out of them.
Actor frameworks don't compose, because actors are an atomic compute unit. You can't really take a horse actor and a bird actor and turn them into a pegasus actor. There are non-trivial interactions between message handling logic that would prevent such a thing with some fairly trivial-to-generate breaking cases.
Maybe composition for compositions' sake isn't that important?
> Maybe composition for compositions' sake isn't that important?
Definitely. I was about to ask "but when did you actually need to compose actors and how does that even make sense?" -- and while I am sure there are people who would find a scenario I feel that would still be tarrying on minutiae.
Obviously the actor model is not a panacea. But for my commercial work Elixir -- and thus Erlang's OTP -- has been a true blessing. There aren't many commercial scenarios where OTP is a very poor fit. They do exist but I'd dare saying they are no more than 10-15% of everything you can stumble upon out there.
This is definitely true, I am just not sure how well it applies to actors. For all data-modelling scenarios and OOP/FP ways of doing polymorphism, composability is a life-saver though.
The point of the actor model, at least in OTP, is not to have the actor model as a unit of computation (as was imagined by Hewitt, et, al) but as a failure domain. Erlang was not designed to be an actor system, it was designed to be a robust system and they later discovered that what they had built was more or less the actor model (if an impure version) and it was easy to communicate the architecture's computational capabilities and mental model as actors. Actually I think if anything it's led to a fair share of misunderstandings, and truly misses the important and unique aspects of Erlang processes.
In the sense of being failure domains, OTP "actors", or processes, ARE composable. With very little boilerplate, that is built in as primitive BIFs in the standard library, I can bundle failure domains together and orchestrate them using links, monitors, and supervision strategies, with the ultimate failure domain being a single erlang VM operating inside a cluster of erlang VMs, all supported without third party libraries as a part of the way of doing things.
Interesting idea, but I'm not sure I agree. I was doing a lot of telephony/mobile back in those early 1990s and there was a huge amount of formalism. We tried to model everything as FSM that transitioned states based on received messages. We even used code generators to generate code directly from diagrams. I believe Erlang grew out of that same approach. And initially, systems that failed would simply restart by default.
Then the idea of adding a supervising and observability was a natural addition to these systems.
That said, I think OTP is a great tool for modeling failure domains and I think it does a great job.
I’m not sure what you mean. A singleton is something your program only has one of. You can have any number of processes. By your logic, every object in an object-oriented language is also a singleton.
> But I'll never understand that people like Elixir for being FP.
To me the answer is: using mutable state is opt-in. I disagree that "mutable state is rampant".
By opting in to the mutable state constructs you are basically saying "I know what I am doing, let me do my work" which is IMO quite fine because "pure" FP languages like Haskell can be a huge hassle when you actually need to deal with the real world.
Mutable state is essential to the actor model. Local arguments to a tail-recursive message loop which change based on the last received message and the previous arguments, and determine behavior (i.e. messages sent and side effects), are equivalent to local mutable state. State machines are a lot better than unstructured, freely mutable global variables, but they are still mutable state.
Sure. That's absolutely unavoidable in any FP language. All their compilers invisibly produce a lower-level code that's intrinsically using the mutable paradigm. It's how our hardware works currently.
No, in an actor based model the mutable parts that the comment you replied to mentioned are surfaced to the user.
It's not about how the Erlang VM is implemented. You could implement it in pure Haskell and compile it so some hypothetical pure CPU. Wouldn't change that part of how you interact with it that is stateful as described.
And I still disagree with that comment that "mutable state is rampant". There are realities which we all have to take into account. To give an extreme example, should we get rid of all mutable databases and use append-only journals? That might help eliminate another class of bugs but there are a number of (practical and political) limiting factors in commercial projects.
But maybe we'll talk the same language if you give a few examples. I was under the impression that my parent poster made the point of "but your FP code gets compiled to imperative mutable code so FP is not good" or something. If I was mistaken in my interpretation then we're talking past each other.
My point was that the actor model has the semantics of local mutable state. The fact that an actor model language is compiled for a von Neumann machine is irrelevant to my point.
Okay, that's true, but especially in Erlang's BEAM VM the mutable state's access is serialized / centralized. An actor's state is not a globally modifiable volatile variable a la the C/C++ ones.
As far as I can tell, that's not what the great-grand parent comment was about.
Have you ever programmed in Erlang?
About your first paragraph: not all databases should be append only, but it would be a good idea to make append-only the default and only deviate for a good reason. (Instead of making mutable the default, and only weirdos go for append-only.)
I understand the point as: being able to send a message to a particular actor, passing it a value that you can later retrieve by sending that same actor another message, is semantically exactly the same as calling a setter on an object reference, and later calling a getter to get the value back. In both cases you have a pocket of mutable state, potentially accessible by multiple unrelated places in code, that isn't reflected in function signatures.
Mostly true, minus the fact that the mutable state's access is serialized / centralized due to the nature of Erlang's actors (lightweight threads; usually called fibers in other languages, and even that is not a good analogy since they are preemptive and not cooperative). So the semantics being similar is not strictly and 100% true. You can't do non-atomic volatile modification like you can in C/C++.
Well, my main Elixir project is the backend of a web service. The input is JSON from the frontend and JSON or XML from a number of third party APIs. We process it, hit the db with select, update, insert and usually return a value. The same function with the same inputs usually returns different values because the db is stateful.
What I like of Elixir is not functional programming, it's the extensive use of pattern matching. What I like less is the convoluted syntax of GenServer with all those handle_cast/calls that obfuscate the real code. As a GenServer is usually a way to store a state, they should have had the courage of calling them objects and give them an object like syntax. After all they initially sold Elixir to developers coming from OO languages. If they had given it a Java like syntax maybe it would be 10 or 100 times as big by now.
Thankfully they didn't give it Java like syntax! The syntax is currently a bit more verbose and explicit on dealing with local/micro state which I now like, but without using 100 lines of boilerplate getter/setters. Using pipes gets one enough "OO" syntax feel.
While GenServer's store state they are, in my mind, more akin to microservices than Java/C++ Objects. They're like microservices, but without needing a separate service bus, global naming system (pg2), etc. Now I do wish Dialyzer/Dialyxer also had better support for checking GenServer handler's and messages, especially intra-process. You have a point the syntax their could be spruced up some perhaps. The process / GenServer paradigm kind of remind me of Smalltalk in a way, where you're passing messages that object may or may not want to respond to. That's not really possible with C++/Java objects.
I think you are misunderstanding what GenServers are and are for.
GenServers do have their own state, but they are used for way more than just storing state. It's probably best to literally think of it as a server in your running application, analogous to a server running on a network.
If storing state is all you're trying to do you might want to look at Agent or possibly ETS.
Further, GenServers come from Erlang, not Elixir. They're intended to model all sorts of runtime properties of code.
"Dogmatic" is a great description of OTP, though. You have to adopt the OTP mindset, adhere to the OTP principles, and organize your architecture in the OTP style. There's a long initiation process where you learn esoteric vocabulary, become familiar with bespoke tooling, etc.
This isn't meant as criticism -- I think Erlang/OTP is a brilliant piece of work. But arguably Erlang/OTP is a practical tool that achieved practicality by embracing dogma.
I mean, yes? OTP is a framework—a paradigm for writing your code in, essentially—but it's an optional one. That's why it's split out from Erlang itself. You can write Erlang however you like. Most people choose to write it in the OTP paradigm. But sometimes that's not the best choice (e.g. the type of code that leex/yecc generates, does not obey OTP principles, nor would it help it in any way if it did.)
Fair point. Big frameworks and dogma go hand in hand, and I suppose Erlang without OTP is at least as practical as any other language.
I wonder though if Erlang might have been a footnote in programming history if it weren't for the mindshare that OTP generated. "Simple functional language" is attractive, but "simple functional language with world-class platform" turned out to be a game-changer.
> I wonder though if Erlang might have been a footnote in programming history if it weren't for the mindshare that OTP generated.
It absolutely would be. There's plenty of "simple FP languages" out there. OTP is very definitely the main selling point of Erlang, and nowadays of Elixir as well.
I think it's more accurate to say that OTP's dogma comes specifically from it having chased down pragmatism to a fault, to the point where whether or not OTP's doing something is a reasonably-reliable indicator of whether or not that something is a good idea (at least for Erlang's typical use cases) and therefore arguably warrants the dogmatism.
what is my array? I don't know. It could be anything. The company I work for just hired a sloppy python programmer that I don't want anywhere near my code, and you know what, if we change a section of our code to Elixir I am way more willing to have him work on our team.
I have a prototype system where I am passing over 100k msg/sec between a dozen backend services written in python (asyncio+redis) and I keep on wondering when my bottleneck will become functional programming and safe message passing by making copies. When will the madness end?
i don't think erlang is requiring FP + immutable + message passing for performance, but rather for correctness. Making copies is actually more expensive in terms of perf.
me too, and I have no clue as to how changing language can increase correctness in my case. I can think of other use cases where in-process concurrency and mutating data structures could cause problems, but I avoid those scenarios entirely anyway.
Tradeoffs. In the setup you describe I'd pick correctness over performance any day.
If you hit a performance bottleneck you might as well just use RabbitMQ or Kafka to queue up stuff and process it as it comes along. Or apply back-pressure if your current code allows for it.
I was in the same boat until discovering elixir/erlang it has by far been the most approachable functional language for me and it has been a gateway drug of sorts into experimenting with ocaml and various lisps.
I personally think elixir is really special and an amazing fit for any project that needs high IO concurrency.
IMO erlang (and arguably even more, elixir) is "functional for the working programmer". It doesn't drown itself in academic abstractions, uses functional programming as a tool to guardrail you from mistakes (in the same way that C guards the programmer from making asm mistakes), with escape hatches that are battle-tested and justified based on decades of experience.
I would say the only other functional language that has the same bent is Julia, which is "functional for the working scientist". It makes different choices about where to expose state, understandable, since scientific computing has different tradeoffs from systems programming.
Yeah I think that's a great way to frame it, historically one of the turn offs for me with FP was that it was overly academic when I am much more about how practical a tool the language is for what I'm trying to accomplish. Elixir was definitely the first FP that felt that way for me.
That said I have grown an appreciation for more esoteric languages and enjoy seeing the way they handle various problems it's just likely they'll never have a place in my toolkit.
Agreed! I found learning Elixir to be a great gateway to learn Erlang as well. When I first tried learning Erlang (before Elixir was released), it became a bit too much to wrap my head around. It was a lot easier when I had gotten the fundamentals of FP down through Elixir, though!
Yes definitely, I would have never even bothered with Erlang before using Elixir and having it introduce me in a friendly way to Erlang, which is a shame because honestly the EVM and OTP are amazingly strong at what they both do.
On the Elixir/OTP side I've really enjoyed this course too: https://pragmaticstudio.com/elixir They have a current discount code LIVEVIEW (not affiliate), which is advertised on their currently free early access Phoenix live view course.
One of the really nice things about shared-nothing concurrency is that it scales to multiple machines. The same code that works locally will work across a data center. Along the same lines, shared-nothing tends to scale better as you add cores. With lock based programming, as you add cores, many times your contention increases minimizing the benefits of the additional cores.
Same. Or similar, it’s taken longer than it should for me to appreciate the functional paradigm.
Rust and Scheme have been gateway drugs ha. I found I really like OCaml-y languages.
Now, I’m very interested in Erlang (and to some degree elixir). I’m learning as much as I can about Erlang and the ecosystem; I’m trying to answer the question, “why isn’t Erlang more popular?”
I might sound bitter but after about 3.5 years with Elixir my answer is very simple and boils down to:
Habit, confirmation bias, sunk cost fallacy.
Namely: people have gotten a lot of battle scars by working with what pays their bills -- PHP, Ruby, Python, C#, Java -- and they refuse to look at an alternative because that would render their huge time and energy investment moot (in their eyes at least; I don't see why this has to be the case but plenty of people have been adamant about this without giving an explanation).
I've only become a better programmer since I adopted Elixir but I never stopped using other languages.
All of that plus what PG calls "the middlebrow dismissal" are the main reasons IMO. People are just too set in their ways.
I've got those scars but I still find erlang refreshing when working with it. Unfortunately, as software engineers, we work in groups and there are many people having these scars. However, the erlang experience changes one forever.
As you said, unfortunately programming turned out to be quite the tribalistic activity indeed.
I got severely disheartened that 2-3 casual mentions of Elixir were enough for several people in this thread to attack and quickly stereotype me. I think I'll just keep quiet, or at least not mention what I work with. This seems to get the message across much better.
Couple of times when somebody posted an OTP erlang update I had no clue what I was reading. Then I came back to HN and somebody broke down why some things are really useful/changed and it was mind blowing.
Can some erlanger explain why you are excited about certain things from the update?
Erlang may be the sturgeon of the software world. It has survived since the Triassic and yet is somehow prized for specific use cases and discerning tastes.
It is nowhere near as ancient as sharks, nor as vicious, but it employs some new physiology, and can survive just as long.
But that was a very different world and it sticks out like an outsider, when really it's been there the whole time.
the story I like to use is -- "imagine a world where java never happened, where servers were abstracted into HA clusters automatically by your programming language, and where horizontal scaling was automatic and free"
much of it is a lot of hard work getting to support TLS 1.3, which is nice. I have some TLS-dependent things in prod and let me say, it is really easy to do TLS in OTP. It's also easy to do TLS wrong in OTP, so I recommend watching all of Bram Verburg and Ingela Anderton Antin's videos first.
the socket library (if I am correct) is the beginning of redoing the entire network stack in a way that is more commensurate with the way that typical programmers expect (but with the right erlang-ey).
> Allow underscores in numeric literals to improve readability
This is great whenever a language does it.
> The embedded documentation is created as docchunks (EEP 48)
It looks like the erlang community is converging on the elixir strategy for documentation. This is fantastic, since elixir documentation is quite good, and it shows that the two languages are starting to put their friendly animosity aside and really working together.
> distributed named process groups is introduced
This is kind of huge. Sometimes you want a locking, fully consistent process management (so CRDTs are inappropriate), and you don't want a full raft/paxos under the hood.
What I appreciate about this update is how little code it breaks and how little future code is likely to be backwards incompatible. Or to put it another way, what I appreciate is the sameness of the programming environment. To a first approximation nothing needs to be rewritten and newly written code will look just like existing code. What I appreciate is OTP is better and there's nothing programmers have to do (to a first approximation) to get that betterness. Instead, rolling out the betterness is a system administration task.
Basically, what is exciting is how well engineered it is.
> What I appreciate about this update is how little code it breaks
Other than a bunch of code reliant on the stuff removed from erl_interface, of course. No idea how big of an impact that has on real-world projects, though.
The deprecation of erlang:get_stacktrace/0 also gave me a brief heart attack for one of my projects¹, but luckily I had the foresight to gate that to older OTP versions (and with newer versions the relevant code already removes dependence on that function), so I can rest at least a little bit easy.
Looking at the changelog I don't see what the current situation with erl_interface is. I've been working on some `erl_interface` code using C [2] and Nim [1], and the headers warn that older `erl_interface` headers would be deprecated in OTP 23. All the release note highlights mention is a new `erl_call` program.
Anyone know about the status / rational for the `erl_interface.h` API?
Much of the erl_interface functionality prefixed with erl_ names has been deprecated since OTP 22 and has been removed. The API has shifted to newer ei_ prefixed functions. I believe there are some new deprecations in place as well which you can find the longer form release notes.
erl_interface itself is not going away but it is evolving with the BEAM VM. Deprecation warnings should be checked when compiling code on each major release to avoid surprises as features are usually deprecated for one release and then removed in the next release each year. The erl_interface documentation should be up to date with regards to new APIs and might be worth browsing again to get an idea of the what changes look like.
Thanks! That's good info. The hardest part was that in OTP 22 the examples given used many of the deprecated erl_* apis. I was able to update the C examples (see the second link) to use non-deprecated ei_* api calls. Mostly small changes and a bit better buffer management. Though, I don't like the lack of buffer length check in even the newer ei_encode_* functions. :/ I added a 24 byte padding guessing most single item encodes are less than that, and then check variable length items for size. Still hard to use safely without a buffer overrun. I'll take a look and see what else may have changed. It's exciting seeing all the continual beam improvements!
Though I am not working with Erlang or Elixir right now I am so pleased that this is seeing improvements. I had as much fun learning Elixir as I did learning Ruby ages ago. It's just a fun language with great construct!
I've wanted to work with Elixir for 5 years now. I attempted to go through The Pragmatic Programmer's Elixir textbook and couldn't make it through. This comment applies solely to me, I feel I didn't have the aptitude to pick up functional programming (and that's coming from someone who graduated in CS from a T3, had years of experience, and starting med school next month).
There's a reputation that functional programming is harder than object-oriented. I think this comes from the very difficult functional programming languages that exist out there. I think that can be an obstacle. People hear "fp" and think "hard".
I personally find Elixir's functional programming is easier, because there are fewer things you have to deal with than object oriented. There are basically no constructs in Elixir that you don't have out of the box in an OO language like Javascript or Python. You just have to deal with that you can't do things like assign variables in if statements (you just have to explicitly export them) and you don't have for loops.
Having said that, Enum.map and Enum.reduce are really hard to get used to and can be an obstacle in terms of "why can't I do this thing that is so easy in X".
In the long run, once you get used to reading them (which honestly took me about 4-5 months, I have like 20+ years of programming experience) I think they are easier because they are declarative -- you can see exactly what is happening to each piece of your list and in the case of reduce, what state is being passed through each iteration.
For loops are far more unstructured, literally anything in any of your parent scopes could be what you're keeping track of through iterations, which means, your mistake surface area is much much higher. The same goes for getting used to "if statements that export values" but, maybe, less dramatically.
I know this is difficult to hear since you gave up, but I promise you if you push through it, eventually it will feel like writing elixir is "programming with training wheels on" you can do crazy hard things without worrying about large classes of logical and structural programming errors. Hope you come back to it!
You have a list of N elements and want to trasform it into another list of N elements -> map
You have a list and want to transform it into a single value -> reduce
Fin.
But hey, you ask, can’t a list be considered a single value? Yes, of course, so map can be implemented using reduce, making it the most useful FP construct
Since it's already been five years, it might be worth considering Erlang instead of Elixr. I'd recommend Programming Erlang: Software for a Concurrent World. It was written by Joe Armstrong who designed Erlang and "sold" it internally across Ericsson to people who were not programmers. They were people who were using Erlang to solve some other problem.
The paradigm of Erlang is message passing. The primary idiom is logic programming (like Prolog). Functional programming is related to logic programming. Between message passing, logic programming, and simple primitives for concurrency; Erlang programs tend to look a little strange. But it all hangs together in a way that has been field tested for thirty years. Erlang is a language that was engineered for use by engineers for solving engineering problems.
Elixr is mostly a way of avoiding learning Erlang and making the Beam VM popular. It is a procedural programming abstraction layer. For me, Elixr's abstractions generate an impedance mismatch and Erlang's syntax better expresses the engineering mechanisms of concurrent systems and logic programming. YMMV.
I've tried learning both and Erlang seems to be less magical and more consistent. That said, I wouldn't dare to start an important project in either language for the fear of getting stuck somewhere along the way. With Go, there are no such concerns, even though the language itself is by far less elegant.
A valid concern but you should have in mind that ElixirForum is one of the friendliest and most helpful programming communities that you will find out there. And that's not coming from me or Elixir veterans but from a ton of newbies, regularly.
I did Ruby since 2011, and tried Elixir in December 2019. I really hated the verbose syntax compared to Ruby, which has a comparatively simple syntax. The next thing I hated in Elixir was the boiler plate for GenServers and having to deal with pids and named processes. However, this weekend, I rewrote about 2k lines of sloppy elixir that I hastily copy pasted, and condensed it down to around 700-800 lines in a few hours with barely any effort. It was easy because it's just functions calling functions. I guess I'm kind of a believer now. Besides that, Phoenix channels are very performant, which makes me think I might switch from Ruby to Phoenix permanently.
The use of prior matches in guards is super useful for parsing TLV style payloads where you first match on a size/length field and then match on the value itself as that number of octets. Does anyone have any insight into how / when similar support will make its way into Elixir?
Erlang/OTP 22
Interactive Elixir (1.10.3)
defmodule Test do
def matchme(value) do
x = :foo
%{^x => bar} = value
IO.puts(bar)
end
end
> Test.matchme(%{foo: "bar"})
bar
:ok
My goodness, you're right (well, your example isn't, but the feature I'm talking about has been present since at least 1.8 (the earliest install I have handy):
Your example is not what EEP-52 adds. The code that didn't used to compile before Erlang 23 is when the `size` attribute of a match-segment requires an expression (i.e. math) to be resolved.
One common use-case, is where a payload's prefix-length field encodes the number of bytes of payload minus one. You'll see this in many protocols as an optimization: if the payload is always at least one byte, then one byte of payload can be represented as a length of 0, allowing up to 256 bytes of payload rather than 255.
Besides the reduction in code, I believe that the old code has an optimization fence (the math expression) that the new code doesn't—you get more optimized code out of the newer expression, because BEAM's runtime loader gets a bigger contiguous chunk of bitstring ops to specialize across.
Oh, and since the newer code is all in a head-clause, if it fails on matching Payload, it'll move on to attempting to match on the next head-clause, rather than generating a badmatch error. Just like if you produce an error in a head clause's guard clause expression.
-----
On a separate note, despite this example being simple, you can actually do arbitrarily-complex things:
oh prior as in lexically prior. The example in the erlang release I believe uses something that looks like what I wrote. I use the sort of matches you are talking about extensively in network packet size matching for a project I'm working on.
I would love it so much that someone took the time to correctly evaluate the pros and cons of OTP vs the current stacks we see those days (stateless services deployed and configured with kubernetes, with an event-bus backbone for service 2 service communication).
I've always wanted to try OTP for a real world project (simply because it looks more elegant and has less parts), and now may be the time, but i'm so scared of discovering huge pitfalls down the road that are better solved by more recent techs..
I think I understand your concern as the foundation is arguably old. But as someone actively following Erlang and Elixir the sentiment feels absurd. Elixir is very recent, very modern and Erlang/OTP has a very strong track record.
There are drawbacks of course but having worked with Python, PHP and JS. I haven't seen any real drawbacks. I wouldn't choose it for minimal overhead or the fastest compute maybe..
Are you "following" or are you actually using it in production for a real service ?
I'm asking, because to my knowledge, the only "huge number of users" recent success story is whatsapp, and after having viewed all the talks about their stack and experience, it seems that the team was so extremely talented that they probably would have made something good with any tech.
Discord uses Elixir heavily. But if you are scaling to 10s of millions of concurrent users, you're going to need those types of skills no matter the language.
I'm not using it in huge scale. But I've worked with companies that run it in production, yes.
There are a lot of Elixir products and systems out there and before those there were quite a few in Erlang. I don't have a sufficient sample size from my own experience but it is hardly old and disused or too new to be sure.
Changelog podcast runs on it. Dockyard build with it. It seems the new default for many previously Rails shops.
I've used Erlang/OTP only on stateful servers, and observed its use in semi-stateless containerized deployment (but in that case, we were generating the containers to have a fictional state; we always had nodeX@hostnameY even if the container being that node moved around). However, OTP ships with a diskless mode, where all of the code and config, other than BEAM and modules critical to starting dist come from dist.
Stateful servers seems like an atrocious point of failure nowadays (except if it's a DB server, or a persistent queue). And i barely see it anymore, now that everything seems to be running on containers or VM that are supposed to be destroyable at any time.
That's why the value of OTP seems to be less obvious at the moment (from the point of view of someone who doesn't use it). But there's surely something i'm missing.
> Stateful servers seems like an atrocious point of failure nowadays (except if it's a DB server, or a persistent queue)
I think that's the perception, but it isn't necessarily true. I work on an Elixir team that quite successfully runs a stateful server. The service is pretty stable and the stateful portion is never the source of problems. That's one anecdote, but if you look at servers for online games, they're often quite stateful and work rather well.
Decently built servers don't fail that often. If you're going to run a couple thousand servers, you'll want them to be bare metal, because VM overhead is real.
If your system is designed properly, it's not unreasonable to have a node out of service for a couple hours while it's repaired. Nodes which don't have database state can often wait until business hours to be repaired, if there are enough nodes that you can have a few down.
I must admit, I have been watching erlang for a while trying to decide when to hop in, and I have increasingly come to the conclusion that I will not be learning erlang as such, although I will be buying the books and reading them, but I will be learning gleam, which is erlang with types.
I'm too used to the safety that rust or F# gives at compile time to give that up.
Does cloud-based serverless computing compete with or complement Erland/Elixir? As a dotNetCore developer who writes a lot of Lambda and Azure function code I am trying to understand how OTP relates to these paradigms or if they are apples and oranges.
OTP paradigms are definitely intended for a larger system that needs its own distribution schemes, handles its own internal process uptime and restarts, and for some use cases, hot code updates (running servers that update without restarting). OTP nodes can also easily and natively communicate between each other across the network, allowing for some pretty sophisticated distributed systems.
There are many solutions that could be more easily implemented on some sort of serverless architecture, but in cases where you want to maintain a service with a high degree of reliability and distributed capability, OTP is a good choice. After all, it was developed for the telecom industry to handle phone system communications loads.
Worth noting: there are other solutions (some of which are already in serverless environments) that solve the same problems of service uptime during code updates/deployment by keeping an older instance running until the new instance is ready to accept traffic. Services like Kubernetes also implement a lot of cluster management capabilities that solve the same problems OTP was attempting to solve, and you can use whatever language and engine/framework you want. Personally, I think if you wrote a service purely in Erlang/Elixir and directly applied all OTP solutions for uptime, process communication, and distribution, it would be faster and more efficient than a typical serverless or container-based system. But, it would also lock you into that system and make any non-OTP tech more difficult to integrate.
I’ve been out of the Erlang world for a few years now, but it doesn’t seem like a great fit for serverless. The VM has a non-trivial overhead, and the language (and VM) design is optimized for long-running, high availability services.
I would say it's mostly that nobody optimized it (perhaps yet) for that environment. There's generally nothing inherently slow in the VM boot sequence - it's just that this was not a priority, so it's slow. There are some attributes suggesting it could be a good fit for that environment, for example, the VM is generally very small - you can get a full system in ~20MB - and that again wasn't something that was heavily optimized.
The VM has an overhead, but looking at our kube cluster right now which runs elixir and ruby pods (treating elixir as shared nothing, old school stateless, rather than distributed), mean RAM for our ruby pods is 1GB, Ex ones are 200MB. Response times: Ex: ~30ms, Ruby: ~150ms.
Sounds about right. With wasm being relevant for edge computing and serverless I'm keeping a close eye on the Lumen project compiler as I expect that to be more feasible for this sort of work. The advantages for OTP remain mostly in long-running services. But for Lumen that might be a more open question.
Erlang VM startup time is not great (in order of hundred milliseconds), but it has pretty unique features like hot bytecode patching in a running VM.
So, naturally, Erlang shines when you need to build a system that runs for a long time, as opposed to short-lived lambdas.
Erlang is a message passing language for writing concurrent applications. Architecturally, Erlang programs share logical similarity with microservices. Architecturally, OTP shares some conceptual similarity with serverless frameworks.
Together, Erlang and OTP provide a batteries included environment for building and running scalable concurrent systems. Scalable down to any device that will run Linux, not just up into the cloud.
> Elixir v1.10.3 is fully compatible with Erlang/OTP 23.0 - so up to date Elixir users are ready to give it a try!