"The BeamAsm JIT-compiler has been added to Erlang/OTP and will give a significant performance boost for many applications. The JIT-compiler is enabled by default on most x86 64-bit platforms that have a C++ compiler that can compile C++17."
Having worked with erlang for some time now, https://www.erlang.org/eeps/eep-0049 is _very exciting_ to me. One of the issues that I have with the codebase that I'm in is deep nesting, which could be solved with this feature. Very compelling.
"This is the most general control flow in this document, being fully flexible with regards to which values it can handle. This was done in part because there is not a strong norm regarding error or valid values in either the Erlang nor Elixir APIs, at least compared to other languages here.
This high level of flexibility has been criticized in some instances as being a bit confusing: it is possible for users to make error-only flows, success-only flows, mixed flows, and consequently the ˋelseˋ clause can become convoluted."
Thanks to the EEF for funding my work on the float to string algorithm!
And also, if you wonder if you could do it for your current platform of choice, please do not engage. Inside float to string conversion lies madness...
I've seen more than once recently an engineer saying that software solutions outside of the Serverless ecosystem (FaaS, DB, etc) is "the new legacy". And that cloud providers have solved all the pains that Erlang was supposed to address.
What's your feeling on this? Elixir sounds very compelling to me, but I worry that I might be going in a direction that's not where the industry is going.
People also overestimate what “the new” is, constantly. Otherwise, PHP would be dead by now rather than flourishing.
Serverless is essentially PHP in CGI mode, with a lot of marketing behind it. There are certainly places where it’s beneficial.
Servers and databases are not going away anytime soon. They are simply too effective for what they do.
The “everything is legacy crowd” hasn’t been correct about almost anything in my lifetime. Mainframes are still going strong. SQL databases are better than ever and the NoSQL crowd has largely died out.
90% of the cloud ecosystems of today are just doing what Java app servers were doing in the late 90s.
Erlang and the BEAM offer a set of trade offs that happen to be ideal for a lot of server based use cases to make hard things really simple. Probably not going to take over the world but if it fits what you do you probably won’t want to go back.
yeah, that was rather curious. Similar to how XML died out. Not with a bang but with a whimper. Not that NoSQL or XML are completely gone today, just like PHP. But suddenly one day you recognize these topics are no longer appearing on HN or elsewhere. Then you realize you're getting old and... oh my god... we're actually stuck working in a fashion industry and nothing matters, life is meaningless, and all this is dumb dumb dumb.
When HN and other techies find a new hammer they end up beating that hammer against every problem they see. Right now it's SQLite. I love SQLite. But man. Can we quit talking about it for one fucking day already? This is probably how it felt to be an Erlang fan when Facebook bought WhatsApp. I remember the decades of Joe Armstrong talking about Erlang on Usenet and no one caring. It's just cargo cult with no thought behind it. WhatsApp was successful using X, so if we use X we must be successful. People have this perception that Python is new and modern despite being, relatively speaking, the same age as Perl. Erlang was cool long before WhatsApp. It's really quite old tech.
>Then you realize you're getting old and... oh my god... we're actually stuck working in a fashion industry and nothing matters, life is meaningless, and all this is dumb dumb dumb
Yup I know what you mean ! There is no zen. As I mentioned some time ago, the 'final zen state' is coding on a mountain top for yourself in a LISP with Emacs on an ultra book charged by happy sun rays :D - only somewhat joking here...
Between XML, JSON, YAML and TOML, I would rather my configuration files served in XML, with schema validation and graphical IDE tooling, thank you very much. :)
I personally prefer JSON to XML. I've seen XML abused in some pretty terrible ways. For example, I worked at an education start-up several years ago and the education company Pearson delivered bubble test questions to us in XML where half of the questions' text were in tag attributes and the other half was in the body of the tag. JSON (and YAML/TOML) make it a lot harder to do stupid things like that. They're also easier to parse, in my opinion.
> SQL databases are better than ever and the NoSQL crowd has largely died out
Especially with json types in Postgres for example. I can store data that’s suboptimal to use traditional databases to access (think complicated and varying form data) but retrieve it and reference it from table relationships traditionally.
Well if you try out the GraphQL API its API gateway is Elixir! The fields themselves provide data from a multitude of services downstream, but the gateway unifies it and sits in front.
Yep, it’s by no means the most common stack but it does power a few critical services. We’re actually hiring for an elixir role right now (non-senior/mid-career, prior elixir not required).
> I've seen more than once recently an engineer saying that software solutions outside of the Serverless ecosystem (FaaS, DB, etc) is "the new legacy".
There's, as I see it, three different kinds of legacy: (1) code that has an inordinately costs to maintain and especially update to changing requirements because it lacks tests, documentation of existing business intent and/or design, or otherwise has lost institutional knowledge needed for nontrivial work, relying on the foggy memories of a high priesthood increasingly, over time, working through ritual without understanding, (2) software with deep and fundamental dependencies on platforms or external components that are no longer supported or no longer available, and (3) software which doesn't meet the current arbitrary tech platform decisions of the organization using it, but which has no fundamental problem preventing maintenance or use.
In lots of places, software that isn't adapted to a particular serverless architecture that the org has adopted is now the third type of legacy.
> And that cloud providers have solved all the pains that Erlang was supposed to address.
They haven't, though for some use cases, the parts of the stack Erlang would be most useful for implementing are the parts you are likely to just get from a cloud vendor (that's not true of all use cases, though.)
> Elixir sounds very compelling to me, but I worry that I might be going in a direction that's not where the industry is going.
Elixir is probably not the best bet if your concern is to direct the most focussed possible effort learning the things that are most likely to dominate the largest share ofnthr general software dev marketplace in the near future.
To that end, cloud platforms like fly.io[0] provide first-class support for Erlang/Elixir and related frameworks. If the industry is converging on Erlang-like solutions, it makes sense that Erlang is a natural fit.
Problem is that outside of Fly, somebody still has to do the system admin. Erlang (used in an web application context) won't manage itself no matter how well designed it is. Stuff like serverless is meant to completely outsource DevOps.
I mean, `mix phx.gen.release --docker` generates a nice dockerfile for packaging up a release (which is all Fly does). At that point it's essentially no different than any other containerized application. I've shipped Elixir apps to Heroku, Fly, Render, and Azure with basically no devops on my part.
Serverless systems in almost every implementation are still stateless or limited stateful-serverless (the best examples are probably Microsoft's Durable Functions or Flink's Stateful Functions - there's also solid attempts by the Akka folks).
These are largely built on top of either a stream processing (in case of Flink) or some flavor of Actor model based on event sourcing for peristence.
What I'm getting at is that you'd build a serverless system on top of Elixir/Erlang/OTP and be served well by its mature actor model implementation and excellent distribution options. See https://eigr.io/ .
The cloud provided options are interesting for some experimental / tactical use cases but I don't yet trust their abstractions to not be leaky and they typically come with vendor lock-in trade offs.
- the supervisor system in Erlang/Elixir is relatively complex. It goes much beyond the "restart my service" when it crashes but allow to restart some processes, without restarting the service, and in a specific order to keep the system coherent.
- local and remote calls behave the same in Erlang/Elixir, allowing to start with a monorepo having different Erlang "apps" than run on the same node, but then running the different apps on different nodes in the future, once the application, scales. This can be done very little changes to the application.
> - local and remote calls behave the same in Erlang/Elixir, allowing to start with a monorepo having different Erlang "apps" than run on the same node, but then running the different apps on different nodes in the future, once the application, scales. This can be done very little changes to the application.
I wouldn't really say they behave the same. They have the same interface, so it doesn't take much to send to a remote node vs the local node, but there are a lot of details in the behavior that are clearly different, as is the nature of the beast. With a local process, if you monitor it, and receive a kill signal, you know it's dead; with a remote process, you may get a process died signal or a connection died signal; if the connection died you can't know if the remote process is dead or alive. That's new behavior and can be tricky sometimes.
> - local and remote calls behave the same in Erlang/Elixir, allowing to start with a monorepo having different Erlang "apps" than run on the same node, but then running the different apps on different nodes in the future, once the application, scales. This can be done very little changes to the application.
A lot of popular web frameworks have their take on how to arrive at the same goal which is to be able to create responsive feeling sites without writing a lot of JS and creating API backends.
Rails has Hotwire Turbo (which is technically back-end agnostic), Laravel has LiveWire, there's also HTMX which is a bit lower level and back-end agnostic. There's also Django Unicorn and Rails Stimulus Reflex.
All of these solutions let you return HTML back from your server and partially update areas of a page while writing no-to-little JS. They're all implemented differently but for a huge classification of web apps the implementation doesn't matter.
After jumping between Phoenix and Rails for quite some time (years), personally I find Hotwire Turbo's implementation and overall using it extremely intuitive, it "just works". It's been legit one of the best development experiences I've encountered. The best part about it is that it works with any back-end too, so you're not even limited to Rails. DHH and the Rails team never cease to amaze me in what they release to the world.
I also worked with Rails before and I'm happy to see their progress,
I however wouldn't go back to work with it if I can choose,
for me Phoenix and LiveView doesn't feel like magic or too much convention unlike Rails does unfortunately.
I'd actually characterize it the other way around: Elixir and Beam give you most of the benefits of serverless with better portability and lower costs.
Also, meta: don't worry too much about where people tell you the industry is heading. 1) They don't actually know, 2) it's much more conservative out there than you might think. Even when new paradigms do gain traction, they pretty much always augment rather than replace, for a good long while at least.
If you're wanting to use Elixir, it's not going to be nearly as mainstream as, say, Node, but fly.io has support for Elixir apps (to speak of cloud), and I think there's some cool stuff going on in Elixir myself.
Of other note is that Erlang seems to enable folks that use it to punch above their weight class when it comes to running services internally and such.
Serverless is far from the only valid way to run things these days.
While Serverless does address some of the points which were the design goals of Erlang, it doesn't address them all. In no particular order:
Hardware interaction is more or less non-existent in the world of Serverless.
Timing/Latency is not really controllable. It's at the whim of your provider.
FaaS doesn't support very large systems well. It's far more about scaling down than up.
Erlang is functional for a reason: it minimizes faults. Quality follows.
Complex feature interaction is limited in the stateless serverless world.
Cloud providers tend to solve for a different set of problems, which is fine. But I'm going to claim that many of the things Erlang was designed to solve runs dual to the problems in the cloud space.
I will bet real money Erlang and elixir will be around and doing just fine in a decade. I would not make that same bet about any cloud services provider.
One current advantage: FaaS is sold below cost by every cloud provider at the moment, so if you can design your app to take advantage of it, you'll save $$$.
Don't know how that'll hold up into the very long term though...
I highly doubt they are sold below cost - when I've checked, the costs are actually higher than a regular VM instance for the equivalent amount of compute.
One good business cycle in the XaaS spaces might disabuse those people of that notion. Even on more stable providers, the whateveraaS products are a good solution for some cases, but far from panacea.
I taught myself Erlang last year and loved it until I tried to write an application. The OTP documentation, while voluminous, was confusing and seemed to have important omissions, and I found little if any community.
I know there's a lot of power there. Whatsapp was written in OTP and had something like half the world using it with a development team of maybe 12.
I'm glad to hear the community is bigger than what I found. I don't actually remember that search process. What I remember clearest was running into the brick wall of the OTP documentation.
If you'd want to try again, I can recommend "Elixir in Action" [1].
It has a great introduction to the basics of OTP later on in the book. I understood it much better after reading that book that any other materials.
While the book is about Elixir (which is a BEAM language) the underlying OTP principles are the same. The book covers "basic" OTP before moving on to some abstractions that Elixir provides.
I'm not sure on your exact timeline, but the documentation has also seen several upgrades.
For example, see the "telemetry" library, written in Erlang and published to the Hex repository. Its documentation is built using the Elixir-based ex_doc https://hexdocs.pm/telemetry/readme.html
I mean we already got a stable 25% or more improvement, to the point that Jason is faster than NIF based json library... we already saw massive perf improvement with the JIT
It is very much that. The Erlang team have not been too proud to steal good ideas from Elixir. Elixir has been a good source of fresh thinking for the BEAM ecosystem which has helped both the Erlang and Elixir side.
The Erlang 'maybe' expression expands on what 'with' allows in Elixir, mostly because the 'with' construct allows a list of conditional patterns and then a general 'do' block, whereas the Erlang 'maybe' allows mixed types of expressions that can either be conditional patterns or any normal expression weaved in together at the top level.
It is therefore a bit more general than Elixir's 'with', and it would be interesting to see if the improvement could feed back into Elixir as well!
The initial inspiration for the 'maybe' expression was the monadic approach (Ok(T) | Error(T)) return types seen in Haskell and Rust, and the first EEP was closer to these by trying to mandate the usage of 'ok | {ok, T}' matches with implicit unwrapping.
For pragmatic reasons, we then changed the design to be closer to a general pattern matching, which forced the usage of 'else' clauses for safety reasons (which the EEP describes), and led us closer to Elixir's design, which I felt was inherently more risky in the first drafts (and therefore I now feel the Erlang design is riskier as well, albeit while being more idiomatic).
So while I did get inspiration from Elixir, and particularly its usage of the 'else' clause for safety reasons, it would possibly be reductionist to say that "the good ideas were stolen from Elixir." The good ideas were stolen from Elixir, but also from Rust, Haskell, OCaml, and various custom libraries, which have done a lot of interesting work in value-based error handling that shouldn't be papered over.
I still think these type-based approaches represent a significantly positive inspiration that we could ideally move closer to, if it were possible to magically transform existing code to match the stricter, cleaner, more composable patterns that they offer.
In the end I'm hoping the 'maybe' expression still provides significantly nicer experiences in setting up business logic conditions in everyday code for Erlang user, and it is of course impossible to deny that I got some of the form and design caveats from the work done in the Elixir language already :)
Also as a last caveat: I am not a member of the Erlang/OTP team. The design however was completed and refined with their participation (and they drove the final implementation whereas I did the proof of concept with Peer Stritzinger and wrote the initial EEP), but the stance expressed in my post here is mine and not the one of folks at Ericsson.
> the 'with' construct allows a list of conditional patterns and then a general 'do' block, whereas the Erlang 'maybe' allows mixed types of expressions that can either be conditional patterns or any normal expression weaved in together at the top level.
This seems slightly incorrect to me. You can write expressions in Elixir's with macro too, by simply swapping the arrow for an equals sign. For example, this is perfectly valid Elixir code:
with {:ok, x} <- {:ok, "Example"},
IO.puts(x),
len = String.length(x) do
IO.puts(len)
end
See https://news.ycombinator.com/item?id=31425298 for a response, since this is a duplicate. TL:DR; I had never seen it and had no idea it was possible because I don't recall seeing any documentation or post ever mentioning it! Ignorance on my part.
You're right. After all these years (and even writing a book that had Elixir snippets in it) I had never seen a single example showing it was possible and did not know it could do it.
Well there you go, I guess the pattern is equivalent but incidental.
I was the same initially, but now having onboarded multiple people on to Elixir projects, the more familiar type of syntax does ease the first week or so of getting your head round things.
I hope to think that it's just that the bar to make significant changes to Erlang is very high. Stability and backward compatibility over time is something worth valuing. For me, this is a property that makes Erlang a great ecosystem.
I think this shows the inverse. DESPITE being a language with a long history, and a proven track record of dependability, they are continuing to add novel improvements. This release includes a new core bit of language syntax, and the expansion of the JIT which is a massive performance boost now on the two most important platforms.
I've seen more than once recently an engineer saying that software solutions outside of the Serverless ecosystem (FaaS, DB, etc) is "the new legacy". And that cloud providers have solved all the pains that Erlang was supposed to address.
What's your feeling on this? Elixir sounds very compelling to me, but I worry that I might be going in a direction that's not where the industry is going.
>engineer saying that software solutions outside of the Serverless ecosystem (FaaS, DB, etc) is "the new legacy"
It's complete blindness to the reality. Trusting one single provider for your whole service (often not even completely opensource) is the same trap as Windows or Oracle in the 90's...just worse (because not on prem)...but hey history repeats itself ;)
> cloud providers have solved all the pains that Erlang was supposed to address.
Assuming this is talking about tools like Kubernetes, they operate on a different level. The OTP's failure tolerance model operates on a (relatively) very small scale inside your application, whereas Kube operates on a whole-application level. Rather than obsoleting OTP, Kubernetes actually complements it.
Are strings in Erlang rather inefficient? Or does the memory usage or string performance not matter in practice?
I have always ignored anything related to Erlang because I dislike the idea of a language that lacks an efficient string representation . . . even though I have spent over a decade using JavaScript! (so maybe I don’t actually care?)
There are binaries and strings. Binaries is what are used as efficient "strings" most often these days. Binaries allow matching, even building bitstrings (binary strings not ending on an 8 bit boundary). Even more interesting, strings and binaries can be combined in iolists, which is an arbitrarily nested or combined list of binaries or other lists. Those can be used for efficient IO and IO drivers know how to emit or consume those.
BEAM is never going to win for straight line speed. This is going to be an improvement for a language optimized for resilience, concurrency and consistency.
Running millions of concurrent processes, with their own heap, with consistent response times that individually tolerate failure without negatively affecting the entire system is just a different animal with a different set of trade offs.
Not for you. But I know quite a bit about the guts of the BEAM VM and I think this is a very interesting development and if the jump is large enough it may well obviate the need for a whole slew of situations where you now have to fall back to externally loaded functions in different languages, and that would make a bunch of stuff far more elegant. Yes, those externally loaded functions will (always) have a speed advantage, but if the advantage drops below the level of the cost of adding an external module then that would be pretty good news.
If you are allowed to write e.g., a Java program and run it on the JVM, the speedup will typically be on the orders of magnitude in favor of the JVM. Static type systems and extensive optimization matter.
However, chances are you have an Erlang program, and it's quite large. In this case, the speedup compared to the older bytecode interpreter is good for many programs. It is performance you get "for free" by upgrading.
The underlying reason for picking something like Erlang is robustness. Java isn't really built for this kind of programming, and the JVM doesn't directly facilitate it either.
I don't think the two JITs really do the same work, so it's hard to benchmark them against each other.
Erlang JIT takes BEAM bytecode and turns it into native code once at load time in order to avoid the overhead of the threaded code interpreter; if Erlang JIT is enabled, the interpreter is not used at all. Java's HotSpot JIT takes JVM bytecode and turns it into native code at runtime in order to improve performance of code that is run frequently (thus the name), guided by runtime information; lightly used code will be interpreted, heavily used code will hopefully be JITed to native. When HotSpot JITs, it may apply assumptions based on runtime use to produce faster code and if the assumptions don't hold for a call, the interpreter can be used instead (and if that happens often enough, the native code can be discarded and perhaps rebuilt), so the interpreter needs to be present always.
Erlang JIT has been focused more on always applying and speed of application (because it delays code loading), whereas HotSpot focuses more on getting performance gains when it applies, and speed of application is less important because it happens asynchronously. So if you benchmarked speed of applying the JIT, Erlang would probably win, and if you benchmarked optimization of the code, HotSpot would probably win, but either way you're comparing apples and oranges.
The JVM is nearly as fast as hand optimised C, this is not intended to be that fast, rather a simple bedrock to build further optimisations into. Immutable everything also will limit performance as will the focus on latency at the cost of everything else. Not sure of actual performance compared to JVM though, I wonder if it’s as far away as I expect…
Erlang JIT is still very simple, unlike highly optimized super complex V8 jit. It's faster than non-jit erlang, sometimes twice as fast, but nowhere near nodejs-fast. Erlang team don't have resources to make and maintain such complex beast. (everything said here is about raw performance)
Some of the things I like better about Elixir and Erlang vs. Go and Java are:
- In Elixir/Erlang, all processes are stored in private memory and can only be accessed via defined message interfaces. In both Go and Java, processes are stored in public memory.
- Elixir/Erlang processes are a lot smaller, memory-wise, than Go/Java processes.
- We moved all of our development at my current company from Java to Elixir because Elixir, being functional, makes writing multi-threaded applications a lot faster/easier. Elixir is also cheaper to deploy. At least that has been our experience.
- I like the syntax of both Elixir and Erlang better than Go, Java, or JS.
- I don't have as much experience with Node/Express, but in my limited testing with Node/Express vs. Elixir/Phoenix, the latter is several times faster and much more scalable. It also makes better use of hardware (which I believe is discussed in the link above).
I hope at least some of this helps answer your question.
There are some really high integration ARM servers, like Cavium Thunderheads. And others. They're all unobtanium.
Could be repurposed for Android development. I don't see it happening soon. Binding to Erlang is pretty difficult, see https://github.com/joshnuss/elixir-ffi.
If it did happen it would be useful, just... take some time.
** Warning: Turns out this is a really bad idea, do not use! When the BEAM calls native functions it needs to know how many reductions should be used. Otherwise calling a long running function can break the scheduling completly, because with this approach a long running function is considered 1 reduction **
I haven't used it, but in a recent Thinking Elixir episode, they talked about Elixir-Desktop, which supports Android and iOS, as well as the standard Windows/Mac/Linux
I don't see any serious attempt at UIs with erlang. I think erlang processes would provide a great foundation for UI components, but has it even been tried?
IMHO, you want to do it from scratch if you want it to really work with Erlang, but that means a ton of work. ex11 exists, but I don't know that it was ever very complete, and it was built in the old era of X11 where clients didn't render fonts, etc, so there'd be a lot of work to complete it. The most active fork has moved towards using it for embedded systems.
Personally, I used ex11 because the wxWidgets interfaces are hard for me to grasp and then accept; it was easier for me to understand X11 as a distributed communication protocol that has a side effect of occasionally outputting graphics; but my application only outputs a bitmap once in a while, so really I could have made anything work. ex11 doesn't help you with mobile though, because Android and iOS don't run X11.
There's also gtkNode, but I don't know how well that works, because it didn't seem any less confusing than wxWidgets. gtk has some ability to run on mobile, I think.
Erlang comes with a binding to wxWidgets that is used to implement the graphical debugger and observer it comes with, but they are fairly utilitarian in their styling and I don’t know of any other Erlang/Elixir GUI apps. Which is to say: it probably could be done, but doesn’t seem to be popular even among Erlang programmers.
basically wxwidgets comes with a webview, and that one is loaded up - and everything is packaged as an app - so you can use phoenix (liveview), or even react or anything to your liking..
believe certain improvements were made to OTP 25, so multiple things will land soon - even livebook https://github.com/livebook-dev/livebook are aiming at shipping a native app..
Not really, because UI components inherently need tight integration for event propagation, layouting, drawing... Sure, you could render individual components in their own processes and have one process accumulate a render tree , but I don't see much sense in that.
The lack of GC also is a problem, so you might have to do each redraw in a fresh process or something like that to avoid accumulating too much memory.
Processes would be good for all sideline functionality like API/db requests and expensive computations.
