When Erlang was designed/invented the goal was never to make a new language, the goal was to design a way of building systems with a set of specific characteristics, massive concurrency, fault tolerance, scalability, etc. One part of this was the language, Erlang. But at the same time we were developing the language we were also looking at how you would use the language and its features to build such systems. The language and the system architecture went hand-in-hand with support in erlang for the type of patterns you would need in the system architecture.
These ideas and patterns existed before OTP. So there were Ericsson products built on top of Erlang which used these ideas and patterns before OTP. OTP "just" took these ideas and design patterns and formalised them in a generic, cohesive way. Plus of course a lot of useful libraries.
So we were never really out to design a functional language based on the actor model, we were trying to solve the problem.
(We had actually never heard of the actor model but were told later that Erlang implements it)
Funny, I recall some surprise in the community because Joe was asking some easy question about C for some glue code. It's been 5 years since I touched Erlang, only for learning, but is message passing style and the way you create functions like in prolog (or in Shen or picolisp) is unique and a great virtue. I know that tcl has secure interpreters, I wonder if there is such a think in erlang.
There is also a mailing list at https://groups.google.com/forum/?hl=en#!forum/lisp-flavoured... and an IRC chat at #erlang-lisp.
P.S. I also have a prolog running on erlang as well.
In my new job, I was given the task of writing a server for a messaging application - which would allow users to send a "hand-drawn" message from our own proprietary handheld device to Windows phones. I was told to learn Erlang and get it done - the company was barely 30 people and had no formal training programs. While I have a degree in theoretical computer science, I didn't do much functional programming before - learnt some Haskell in 2001 in college (just the basics, equivalent of first 6 chapters of LYAH, no monads). I remember learning Erlang over the weekend and delivering it in the first week of my job. Obviously, the code was neither great nor scalable - but I write this not to boast but to tell people that Erlang was so beautiful and easy that even an average intelligence person like me could use to produce functional software in a week. Today, my only regret is that I am not a programmer.
"Hardcore feel" is definitely there and the language is anything but "writing config txt files to some underlying system".
I've built applications in the > 10k SLOC range that use a consensus algorithm for distributed computing and it was hard but made tremendously easier because of Erlang's built in primitives.
No hardcore feel was lost at all and neither was it writing a config txt file...
[EDIT] I would love to know why the downvoter disagrees with me, unless you're being a downvote kiddie.
Anyway, just a guess, I don't even have enough karma to downvote.
Mind to share some more details? What project/company it is?
Maybe I assume good faith on the part of the speaker in a wider range of situations than many folks.
Aside from that, it's obvious he has more experience with Erlang but the tone still matters in a discussion.
You know, there is karma involved. If down votes are supposed to be used the way you describe, people that are intelligent and polite and educated on their answers, but share a different view than the majority here, should not be part of the community. Is that the intent?
Down voting is for completely irrelevant comments, inappropriate comments, etc...
[EDIT] To be clear, I did not downvote you and I disagree with the down votes on your comment here, even though it's off-topic it's important to acculturate people that are accustomed to "downvoting for disagreeing". Which is why I'm commenting.
> pg 2455 days ago | link | parent | flag
> I think it's ok to use the up and down arrows to express agreement. Obviously the uparrows aren't only for applauding politeness, so it seems reasonable that the downarrows aren't only for booing rudeness.
> It only becomes abuse when people resort to karma bombing: downvoting a lot of comments by one user without reading them in order to subtract maximum karma. Fortunately we now have several levels of software to protect against that.
Why not just a change in color to something different, not harder to read? If down vote is to be used however the down voter wants then why attach the "punishing" color to it?
It's an interesting system in that I have not seen it anywhere else and indeed does seem to work. But still I can't understand it.
Downvotes instead of comments is still a personal peeve but I guess I can't chide people for it on HN, lol.
For anyone looking for the citation you can see the full (and quite short) guidelines here.
I've been having a lot of success with remotely debugging things with observer ( http://www.erlang.org/doc/apps/observer/observer_ug.html ) and Recon ( http://ferd.github.io/recon/ ). I've been using the latter to compile code locally and quickly deploy it to a remote machine (while it's running, of course) to debug problems without going through a whole commit/pull/build cycle.
Some of the other responses in the thread are pretty good too:
it's callstack will never grow. you can have multiple of these running and the scheduler will still split work between them.
a process is a tail-recursive function where the argument becomes its state.
a process ceases to exist when it stops being tail-recursive.
erlang's distributed computing strengths can in ways be attributed to tail-recursive approach to network programming & beginning with concurrency in mind. everything else came as a side-effect.
for anyone interested for more examples of expressiveness/composition, this is from my talk at functionalconf
2. Node's or machine's are first-class citizens.
you can decide to replicate data over to them, make the data on one node location-transparent, or decide to just abstract it all in a distributed system.
3. binary patten matching
you can pattern match not just [A,B] = [1,2] but also contents of A,B. or do gaurds against contents . eg if the contents of this binary variable begin with "foo".
4. you never have to import any header or module. the vm uses all modules available in its (ebin) path. big cyclic dependency headaches good bye.
5. as many schedulers as there are cores in your machine. (configurable)
6. hot code swapping.
albeit little contrived for the beginner on production systems.
comes bundled with the fsm,client-server,etc skeletons called "otp", the same that runs 40% of the worlds telecom traffic.
PS: the root link describes more such features you may find useful
Loïc Hogun (author of Cowboy, other projects) said:
For me Erlang is first fault tolerant, then concurrent,
Is Go moving up from concurrent to fault-tolerant in a real way?
That's not to say they can't inhabit a lot of the same areas, but Go's area of interests will probably remain somewhat separate from Erlang.
But there are more here and there if you hunt for them.
I think for things where they are sort of similar, Go will definitely be the more successful language because it looks so much more familiar and because it's moving faster with more momentum, at this point.
I'm using Erlang for a semi-embedded thing at the moment, though, where stability is more important.
What I do see eating into Erlang is Scala and Akka, though.
On the other side of the coin, if I wanted to write a simple command line tool quickly that was easy to deploy to many servers, Go might be a great choice, where Erlang is a bit painful for small and simple programs.
AOT compilation is also being discussed for OpenJDK 9, or 10 depending on the overhaul progress.
Edit: even Haskell seems to be about 7x faster than Erlang (according to ). Suddenly, Haskell looks like a good candidate for writing a distributed server :) Can somebody comment on this?
Erlang is built first for fault tolerance, not speed. The decisions that went into that make it also really, really good for concurrency and distribution (and so that's what it often gets touted for).
The decisions that went into C++ were for speed, and compatibility with C (and all that entails).
If you've got something truly resource intensive, yeah, you're going to need a lower level language. That's largely unavoidable. Erlang.org flat out states don't use it for number crunching.
I'd guess that 90% of what is programmed in the wild isn't that resource intensive. And given that, I'd go so far to say that most of what I've coded in Erlang is more performant than what I've coded in Java (haven't done enough C/C++ professionally to have useful data points). Why? Because the abstractions and conventions lead me to writing better code; the tooling lets me profile and find bottlenecks more easily, and even code I tweak for performance is oftentimes cleaner than it started, rather than more arcane.
Joe Armstrong, the founder of Erlang, is quoted in OTP in Action as (paraphrasing as I don't have it on hand) - "First, make it correct. Then, make it beautiful. Then, if you need to, make it performant. Chances are once you've made it beautiful you'll find it sufficiently performant". And that's been my experience with Erlang.
Nothing wrong with it just an observation. Say they shed their runtime system. libgreen got moved to a separate package. Instead of relying and encouraging creating mutliple tasks and have them fail!( well panic!) now they are moving to have error traits as return values.
Again nothing wrong with it. I think it is a good move. Just observing how it started off at one side and moved towards the other.
I think it calls for a Rust-Erlang interop.library just like there is a Java one. I would like to be able to have a Rust node. Or say create NIFs in Rust perhaps.
Well, if I need all of these options, then Erlang is obviously not the way to go. In that case, Erlang might work as a prototyping language.
My impression is, from what I've read here, that Erlang might be the preferred tool for situations in which latency is not of paramount importance. For example, when sending text-messages over the internet, it doesn't hurt if the message takes 1 or 2 seconds longer to reach its destination. However, for a webserver, every second it takes longer to load a webpage means that you lose customers.
Do you think that's a fair characterization?
In fact, since you started with benchmarks, let's return to benchmarks (I find them to generally be useless, but you're looking at them, so why not) - http://www.techempower.com/benchmarks/
Compare CPPSP (the fastest! C++!) with cowboy (a popular web framework in Erlang). They're being run on the same hardware (so we're not talking about ability to scale across nodes even), serializing JSON.
Cowboy is about 1/4th the throughput (requests per second). Now let's look at latency. Cowboy took about 4 times the time, .8 ms on average per request to the CPPSP's .2. Great, C++ all the way, right? Err...no. Look at the max and standard deviations.
Cowboy had a max latency of 3.7 ms. CPPSP had a max of 134.6. Standard deviation of cowboy was .3 ms. Standard deviation of CPPSP was .5 ms.
So Cowboy is still really quite fast, more predictable, and has far smaller outliers. What are people going to be more likely to object to, waiting 3.7 ms, or 134 ms?
Couple that with the reliability Erlang provides, the ability to scale straight outta the box (so if you find you're hitting a throughput issue you have a clear path forward), and the joy of functional coding...and it's pretty compelling, to me at least.
This, times a million. 99th percentiles and standard deviation are what truly matters in benchmarks. To liken benchmarks to automobiles, requests/second is horse power and 99th percentile/standard deviation are torque.
If you can write reasonably idiomatic Erlang from the get-go (design things in terms of gen_servers and such, instead of single-process monoliths), Erlang is a great prototyping language, because anywhere you send a message and receive something back is a point where you can drop in some other implementation of that particular bit without affecting the system as a whole.
You'll find in practice that the only time you'll be tempted to do that is if something is computationally expensive. Erlang is pretty darned good at reading from and writing to sockets in general.
It's a fine tool for this, too.
I still haven't experienced a programming environment that replicates the Erlang/OTP environment. Period.
Haskell is amazing, I've built my company on it, but we're also building our company on Erlang - Haskell's tooling around distributed computing is much more immature than Erlang's (although Haskell's concurrency primitives and STM libraries are extremely mature and arguably better than Erlang's actor model).
It's a fun product :)
Ruby is also slower than C++ & Go, But Ruby on Rails is not only popular but a very productive web framework which can help you build your app with a productivity that C++ or Go can't match.
When you write a high quality, fault tolerate system, the raw speed comes at the end. Erlang shines when you write a concurrent system plus its design is very unique. Erlang I would say isn't only a language but a whole philosophy of Software development. Once learned you can apply to many other platforms.
All in its a joy to work with functional languages and Erlang is perhaps the most commercially successful functional language.
This is similar to the ADA argument -- it's great and can be used to write safe software, but do you need safe software a month from now or unsafe a week from now? Global networks with millions of users are extreme outliers.
- A large system never runs on a single machine if you want any level of realistic fault tolerance. Two nodes is a bare minimum, 3 an acceptable one.
- The number of nodes will always depend on where your bottlenecks lie. Erlang developers would be rare and few to write CPU-bottlenecked code directly in Erlang. The usual approach would be to write your system in whatever is appropriate, and then too coordinate things with an Erlang layer.
- I would a hundred times more willingly maintain and debug a running Erlang system than a C++ one. I'm kind of sold on the idea though, and wrote http://erlang-in-anger.com to share my experience there.
Out of interest, what makes you say that?
Looking at the TIOBE rankings (not that this is definitive), Erlang doesn't even make the top 20 languages, when F# and R do:
Is there a reason you think it's more successful in the commercial world?
(btw, I'm a fan of Erlang - unfortunately I don't get to use it on a regular basis; other than products built on it: RabbitMQ mainly)
Just to mention one, RiakDB, a successful distributed software system is developed on top of Erlang. Actually I've yet to see any other product used by 100s or perhaps 1000s of developers developed on F# or Haskell. Not to mention, almost over 80% of telecom industry is running on Erlang.
Rabbitmq -- probably the most popular messaging system
Riak -- distributed, fault tolerant database
WhatsApp -- managed to route billions of messages a day with only a handful of engineers and servers.
Ericsson -- pretty much got the market for cell base nodes cornered. Chances are about 50% if you use internet on your smartphone, that Erlang will be involved.
Some firms on Wall Street use Erlang -- remember Serge Aleynikov case, he is an Erlang programmer.
Ejabberd -- a very popular XMPP server
CouchDB/Cloudant(IBM) -- another database and database-as-a-service company use Erlang.
So I would still say the original statement holds. By success might mean the amount of work being done not amount of people writing code. Think about WhatsApp. It was only 10-20 engineers that worked on the back-end yet think about the massive amounts of data they were able to handle.
His erlexec thing is quite useful: https://github.com/saleyn/erlexec - I started contributing to it myself before I realized who he was.
1) There are trade offs, sometimes you trade development time for execution time.
2) Erlang is "fast"/scales, ask whatsapp
3) A lot of people worry about execution but have enough spare CPU cycles to compute the universe.
4) Benchmarks mean nothing, you must figure out what is meaningful to you. You won't compare a ferrari and a minivan? Well, you could, but yet, find that a minivan is more practical for your daily needs.
But whatsapp is not a latency-critical application.
For most other things, Erlang is a fine choice. In your other comments you talking about web pages. People compose those all the time with languages like PHP and Ruby, and things mostly work out.
Odds are, with a lot of web stuff, a decent portion of your latency is going to be in the data layer anyway.
I think WhatsApp is latency critical, no user ever said it's okay to have their message show up minutes after it was sent if both parties were on a good internet connection...
Try to look for something that works at scales you're interested in, break it down to its components and rearrange them into a simple architecture, then only can you consider performance. It turns out the lack of performance you feared at the beginning doesn't actually happen; in the meantime you'll have built a robust software (and, if you want to start the language, you'll have learnt a whole new way to program)
"Most (all?) large systems developed using Erlang make heavy use of C for low-level code, leaving Erlang to manage the parts which tend to be complex in other languages, like controlling systems spread across several machines and implementing complex protocol logic."
The fun fact is that most distributed systems don't really care about 7x slower too much. Those care about scalability, reliability and fault tolerance more. One of the most reliable software systems out there was written in Erlang and as far as I know, there are few system can match that sort of uptime. There are few large scale systems in Erlang, including one Amazon AWS service, and bunch of others like WatsApp.
I also like to mention the VM that is pretty amazing. There is per process (not an OS process) garbage collection that enables Erlang to meet with super tight SLAs in terms of latency. Ideal for high scale websites, doing the routing, but also for moving data from A to B. The concurrency model is also one of the strongest features of Erlang. Message passing is very powerful and enables you writing asynchronous code that is still easy to read and follow. Go can match that probably with channels and go routines though.
If you care about single node performance you can either use NIFs or if you not tied to Erlang chose a language that is focusing on that more.
damned lies and benchmarks. hipe enabled? VM tuned properly? attempt to use halfword emulator? comparing C++ the compiler optimized down to SSE2 instructions to an emulator VM loop?
How much slower is Ruby than C++? Ruby runs every startup in the world. This website is the home of "DO THINGS THAT DON'T SCALE" including using fast to write programming languages at all costs even if 3 months later their "we have modules for every occasion!" design will be inadequate (you are testing your _ideas_ after all, not programming language speed, at least of the majority of use cases for people on this site).
a good candidate for writing a distributed server
distributed means you need monitoring. monitoring means you need supervision. supervision means you need restart strategies or failover or other promotion mechanisms. all that is built into erlang for free. optimize for _reliability_ not for speed when creating distributed systems.
If you'd looked at that web page you'd know the answer to "hipe enabled?"
Same as you'd likely do with Ruby or PHP or whatever.
Are you saying that such projects are better written in a combination of Erlang and C/C++?
It's easier to find C++ people than it is Erlang. That is a consideration.
It's why Facebook chat was originally in Erlang, then switched to PHP. Facebook already had a lot of PHP devs, they are a lot easier to find, so rather than having to bring people up to speed, or keep a dedicated team always slightly free to be able to handle issues for chat, they rewrote it in PHP.
There is no magic bullet, not even Erlang. You have to weigh a variety of considerations, and pick appropriately based on them. But dismissing Erlang because a benchmark showed it to be slower is likely a mistake; dismissing it because you can't hire fast enough for it is a reasonable business decision.
You sure about that? I was curious and googled and from the Quora reply below it seems they switched more from Erlang to their own C++ libraries because:
"...some of the abstractions Erlang had that allowed one to transparently have a distributed system across multiple machines actually caused reliability problems -- one server in the group would fail and cause cascading issues. Secondly, we found a number of instances where Erlang wasn't scaling well to multiple cores."
Facebook has the money to throw at something like that with no problems. Not everyone does.
One of the best kept secrets of Haskell, it can be very fast, supports multiple concurrency/parallelism options. The Yesod team does interesting performance work on the Warp server:
That said, Erlang is really fast at things it was designed for. Try downloading Zotonic or some other Erlang app, get it running, and compare to others like Rails. Sharp, responsive, fast.
Erlang is not something you want to do for numerical crunching.
Erlang is for a very specific set of problems and it's concurrency.
For any numerical stuff you can off load it to other languages...
C++ lets you write fast code.
Haskell is interesting.
If you want maintainable AND fast, have a look at Rust.
But when I try to come up with scenarios/ideas where Erlang might be a good fit, I realize I'm not smart/motivated enough (yet?) to tackle such problems.
Erlang isn't difficult. It's the problems it was designed to solve that are difficult.
Just about any sort of web backend (that isn't having to do a whole lot of number crunching), any sort of interaction with hardware, handling/transport of binary data (as mentioned above), and of course any time you need reliability, it's worth considering.
But even in areas where there's not an obvious win, a lot of problems that are traditionally done in a single threaded manner can be made simpler in a concurrent language like Erlang.
An example I like to give is time based task scheduling. In a traditional language, you'd implement something like that with a priority queue. Which is fine, but you have a lot of work. First, what happens if timings change? You have to rejigger -everything-. What happens if you have multiple tasks at the same time? You need a threadpool or similar to dispatch these things onto. What happens if one task itself creates a new task for a later time? You'll have multiple people adding things onto the priority queue, so now you need to start locking. And of course, since you've got your dispatching process sleeping until the next task is to start, you have to make sure any time you modify the queue, you remember to notify that sleeping task, so it can update as necessary. There is a lot to get wrong.
In Erlang? You spin up a separate process for each task, with a timer. Done (with caveats; you likely will need to persist the task information and load it periodically into a task+timer process, but that's pretty trivial).
A similar case can be made for functional compared to a more procedural approach. That's a little too long winded for here, but even examples that seem -ideal- for OO, can oftentimes be described more simply in a functional, data-centric manner, and future refactorings are made simpler as well in doing so.
Absolutely, Erlang can serve well as a general-purpose language.
But there's no denying that it was designed to solve very costly, very difficult problems, and that remains its strength.
Erlang (elixir, really) looked promising for this so I've been investigating off and on while we do some prototyping and flesh out some of the details. So far it seems to be that you get nearly all the advantages of Erlang with running something like RabbitMQ and then being able to write message consumers in whatever language you desire.
Everywhere I think Erlang would be a good fit, seems that relying on RabbitMQ instead and having less code to maintain in-house makes more sense. I'm still very early in my Erlang journey but so far haven't been able to convince myself to use it directly. I must be missing something.
Even just a few seconds of network hiccup will throw it into a partitioned state, out of which you cannot reliably escape without manual intervention. There are also quite a few bugs that you tend to hit only when RMQ is exposed to network issues.
To use RMQ with bad networks, you need to use either the Shovel plugin or the Federation plugin. Both are somewhat awkward to use, and require manually setting up the different routes that copy messages across.
My advice is, if you like Erlang's architecture and need failure-tolerant IPC, just use Erlang directly. Don't use RabbitMQ for IPC if you need reliability.
I wish I could say better things about RabbitMQ, but it is easily the flakiest component of our stack. When you're used to rock solid server software like Postgres, HAProxy and Nginx, RabbitMQ is a big disappointment.
Could you point me to or elaborate on some specifics on the partitioned states and manual interventions required? I'm only really calling the shots on the server side and endpoints will be written in various languages, part of my reason for looking at RabbitMQ since it has so many libs.
If you read the RabbitMQ documentation, they readily admit to the fact that the clustering support is not designed for unreliable network connections. It's mentioned only once, and it's easy to overlook.
Here's a very quick, rough overview (it's complicated):
RabbitMQ works by assigning queues to specific nodes. A queue is a physical thing belonging to only one node at a time; consumers, publishers and so on are distributed, but queues aren't. If you lose that node, you lose the queue. If you get a network partition, the nodes in the other partition(s) will not see the queue anymore. Connected publishers and consumers will start barfing when their bindings no longer seem to exist.
RabbitMQ can mitigate these loss scenarios by mirroring queues. They call this "high availability", or HA . Each queue will have a master node and a number of slave nodes that maintain replicas. In the event of a cluster issue, a slave will be elected master.
Ironically, HA, despite its name and apparent purpose, still requires a stable network. It does not deal well with network partitions. It's not really high availability. It is, I suspect, designed for situations when you wanted to take down a node manually, in a controlled manned, without disturbing a live cluster.
The reason, and this is the important part, is that when a network issue is resolved and the partitions can talk to each other again, RabbitMQ has no way of reconciling queues. If the partition divided nodes into sets A and B, then A will likely have promoted a mirror slave to a master, while B will have kept its master, or vice versa. When the partition is resolved, you now have two masters, whose queues have very likely gotten out of sync in the mean time.
When this happens, RabbitMQ normally needs to be manually restored; it will simply cease to work properly. Note that I'm talking about the case when the network issue has been resolved. The network is back, RabbitMQ isn't. You need to decide which nodes should survive and which should be wiped, and this recovery process is manual. Of course, if this happens in the middle of the night, you have a potentional emergency.
Fortunately, RabbitMQ has a band aid on top of HA called "automatic partition handling" . It offers three different modes of behaviour, of which autoheal is probably the most important one. In autoheal mode, RabbitMQ will automatically pick a winner to solve conflicts, and promote it to a master. This will result in data loss; the losing nodes will be wiped. Autoheal is actually pretty crazy, because it is lossy by design; the only way to find out if you lost any data today is by grepping the logs for autoheal messages.
Worse, due to the "poor man's database" nature of RabbitMQ, doing any kind of manual recovery -- like dumping the conflicts, reconciling them, and dumping them back in -- is probably not realistic when it happens.
Note that even on a flawless LAN, RabbitMQ can get network partitions if your server has high enough CPU or I/O load. Another thing that can induce network partitions is soft kernel lockups, which is common enough in virtualized environments; for example, VMware's vMotion can move a node from one physical machine to another, a process which can take several seconds, during which time RabbitMQ gets a fit. In such cases you'll want to increase the "net_tick_timeout"  setting.
It only dawned on me later that Erlang was only created in the 80s, so I must have misspoken, but yeah, my friend is one of the best developers I've had the pleasure of working with regardless.
For me, this translates to: invest unpaid time for your employer to improve things, on the off chance that it will be accepted. I understand where he's coming from, but that borders on self-exploitation.
On the other hand, if it gets accepted, there might be several benefits that could make the unpaid time worth it:
* getting a raise or a promotion for the initiative
* better work environment due to using the best tool for the job
That's a WIN for both sides of the equation. You're expanding your knowledge and learning something new; The company is getting the benefits of you learning new stuff. This is how 'engineering culture' is supposed to work.
It's complete garbage to associate this with "exploitation." If the company were saying "this is mandatory that you do exploratory programming on nights and weekends for the next 6 months" then you might have half a leg to stand on.
What about devops? Are there any areas where you either aren't collecting data you might want, or you are collecting it but only recovering it after it's caused an issue? Maybe a small utility there will, while nibbling a bit at your productive hours (few hours a week), lead to enough gains over the long term as to offset it.
Sure, if your employer prevents you from touching anything outside your dev box without a TPS report signed by the head of engineering, the head of QA, legal, and Gryphon (the mythical animal one), and any code that passes always has to be in Java, you're out of luck, but that seems like the kind of environment to flee.
Another thing, tangentially related, which I've been thinking about lately: I'd love to do an apprenticeship with an old-school erlang hacker.. haskell hacker or lisp hacker who learnt the hard way in the field.
1. Binaries: this isn't really an issue. We ship the JVM with anything that we do in Java. Unzip, run, done. Go is probably ideal there but it doesn't support embedded resources without some hideous hacks so you're still going to be deploying more than just a single binary in a lot of cases. CLR is pretty good at this as well.
2. Sockets: 0MQ/WCF/JMS/any stream abstraction wired up correctly.
3. ASN.1: Everything has ASN.1 libraries these days. I've never had to use one in the real world in any of the sectors I've worked in.
3. Let it crash: we do this. In fact we force threads to crash if they do anything bad by throwing an exception that isn't caught. All threads are designed to be idempotent, restartable and transaction aware. This is true in JVM/CLR at least.
4. Supervision: New thread, new state, no shared state. Not hard. PHP does this...
5. Inspection: I've seen the Erlang toolchain and it's pretty good but I'm not joking but inspecting and debugging the state of a system is better when there is lots of pooled knowledge (google) and lots of diverse tools. JVM wins there every time.
7. Messaging is not RPC: It's not in JMS either or WCF. It abstracts the transport away. In fact we have inproc transports in some cases that use concurrent collections to throw messages around.
8. Dialyzer: compiler, static typing (Java is less strong here to be honest than say C#).
I really like the idea of Erlang but the worse is better mantra applies here. It's easier to get staff, the knowledge when something goes pop is available, we benefit from the reuse of billions of lines of code written and tested by others if we pick a JVM. If we have to think a bit or throw some more kit at it, so be it.
Edit: just to add, I'm not knocking Erlang and quite like it in principle (I spent 2 months disassembling CouchDB to see if we could maintain it so it's not a foreign language to me), but the justifications in the reply aren't particularly unique ones nor are they major advantages.
2. Apparently you haven't worked with Erlang. The point here is you don't
write much networking, you write message handling exactly in the same manner
as you would for communicating inside the application, without any networking.
0MQ, WCF and JMS don't even remotely resemble the operation model for network
connectivity in Erlang.
3? (Let it crash) And you needed to specifically design your application,
including supervision (you do have it, right?). Erlang provides all that
4. You didn't get what supervision is about. It's about restarting process
when it crashes, crashing all the related processes and aborting when
restarts are too frequent.
> It's easier to get staff,
It's easier to get some staff, but it may be much harder to get good
staff. Signal-to-noise ratio is very bad among Java programmers. And it's
quite hard to find bad Erlang programmer once you find somebody who can write
in Erlang (or learn it quickly).
It's not that Erlang provides one specific killer feature that makes it great
choice. It's that Erlang provides plenty of nice things already built in, and
all that sums up to a platform that one would want to use every day. You don't
have good binaries syntax on top of JVM. You don't have good logging framework
in standard JVM. You still need external libraries for sensible network
programming. You have to design your application specifically to handle
thread failures. You can't just spawn a thread for every request running and
be done with the job. You don't have a distributed soft real-time database.
And the list goes on.
You just need to do much work before you even start programming, and then to
manage all that (which is no easy task), just because your platform doesn't
provide any of the mentioned things and its working model is less convenient
for writing network services.
Erlang is really strong for that.
As to Java, sure, you can do anything with it, and do a decent job of it. But sometimes, as a startup, you are resource constrained, so if you can do more with less because you have a good grasp of a tool like Rails, or Erlang or Lua or whatever, that might make the difference. For a large company, Java is definitely a safe pick - no one ever got fired for choosing it: you'll be able to get whatever you need done, and "If we have to think a bit or throw some more kit at it, so be it." For a group like WhatsApp, Erlang seems to have been a good pick.
Yes it is far superior from a representation point of view at least there. I have no doubts about that.
The difference between having such goals stated during the design phase and then keeping sight of them during all future development and say you forcing a thread to crash in the JVM is that in Erlang this is the way to do it, in Java you're going to have to re-implement your supervisor and clean-up yourself. And you'll probably get it subtly wrong.
Erlang is not a bunch of features that you can get elsewhere, it's an integrated whole on a very solid foundation with decades of proof of reliability behind it.
Not sure you read / understood what that point was about, due to its native binary types, tail recursion and pattern matching make dealing with binaries easier in erlang than most other languages
As far as most of the other points, you are mostly agreeing with the statement "nothing is impossible with other languages compared to erlang, but its better designed for this"
PHP does not so long process thread supervision (I mean it can but close to 0 people use it that way), Being able to inspect running processes is useful as well as the fact google is helpful, the JVM's entire ecosystem isnt based around a let it crash concurrent process isolation philosophy and java in particular is one of the worst offenders at having to ember error handling code at almost every point of your application logic
My point is more that regardless of the syntax, runtime or approach, the same outcome is possible without having to enter a risky niche.
PHP does nothing, but if you throw it in a prefork MPM module in apache, it does that.
Sure we don't let it physically crash and we handle the exception at the base of the thread and decide what to do, but the outcome is the same. Java error handling is absolutely fine and in some cases, far less painful that the "err" semantics of Go.
In most other languages, the caller is forced to deal the the exception, which litters the responsibility to resolve the exception across all consumers.
This is qualitatively different from erlang
3. it's not only letting it crash, it's supervisors, restarts, cascading crashes when the parents retried enought times. Yes you can do it in other languages, but in erlang it's already done and battle tested.
4. that's not supervision
7. in erlang sending a message to a process in the same cpu and in another server has the same syntax and it's the common way of building software, in other languages calling a method and doing a jms message are two different things that you have to deal differently.
8. I would add quick check to the mix, but yes, dialyzer is just a kind of progressive typing
For example, having an receive expression and pattern-matching and simple typing (like everything is a term - self-evaluating or an expression or a list of terms) makes your code shorter, self-evident, easy testable.
Now let's look one level down. Pattern-matching on binary data (alternative to costly parsing) is so quick that in OTP they do matching on TCP packets in real-time.
Conversion from terms to binary and back is also quick. So the the "central idiom" - message-passing - is just sending and receiving a list of terms (atoms) on a language-level (guarded pattern-matching, like on function arguments) while messages will be encoded into efficient binary form and delivered by Erlang's runtime.
Again, almost everything in Erlang fits together pretty well because it is layered system founded on proper principles and design decisions. Immutable data, share-nothing process isolation, light-weight processes, common and simple data-representation format based on simple types, pattern-matching on everything, etc.
It is almost like industry-strength Lisp system - a few layers of DSLs, using which one describes the real-world abstractions on different levels on different sub-languages (based on special forms and function composition).
gen_server is the most common example of such kind of "vertical" decomposition. You have to write only "pattern-matching on receive", while lower layers of abstraction (encoding, protocols) and upper levels (process supervision) are clearly separated with proper abstraction barriers.
Of course, it is possible to code something like this in Java, but there are the subtleties. GC for mutable data could not be as efficient as one for immutable data. No mutable state, no sharing, no locking, no problem. These aren't just slogans.
So, Erlang is a small language designed around proper concepts. It "wins" not only in terms of lines of code, but in efficiency, resource usage (Erlang runs on 16Mb RAM).
I could go on, but I hope the idea is already clear. It is not a "list of features" what matters, but which ones we have and especially don't have, and how they are fit together. In small mostly-functional languages like Lisps or Erlang or ML-family they fit perfectly.
I imagine you are referring to the full stack, as JVM and CLR implementations do exist which require a few hundred KB.
As for the rest I fully agree.
I would love to know your thoughts on this.
That comment is just about the exact definition of the blub paradox.
When presented with a more advanced idea, there's no underlying context to join the discussion, so the brain jumps to what it "already knows" instead of a more advanced concept of what's actually being talked about.
Now whether this is worth the tradeoff of switching to a new language and framework (OTP is a framework even if it is lightweight to use) depends on the team I suppose.
This is just based on my reading of the docs. I hope someone more experienced will correct me if my take is wrong here.
Another edit: Getting more down-votes, but not learning anything. Do you think this is the wrong place to say this? Or do you think I'm just wrong to worry about this level of precision in "normal" speech? Reply! Engage! Tell me why I'm wrong.
I'm going to be "that guy" ... warning: Rant ahead.
The wording of this question is like sandpaper on my brain - for me, the "only" is misplaced. Assuming he is asking:
What is there that I can do in Erlang,
but cannot do (or is significantly more
difficult to do) in other languages?
What can I do only in Erlang?
What does Erlang force me to do?
It's very hard to talk quantum using a language
originally designed to tell other monkeys where
the ripe fruit is. -- (Night Watch, 2002)
When using plain language is it difficult
to construct a sentence that a determined
adversary cannot misconstrue.
This comment is intended to be constructive, although I admit freely that it is off-topic for the submission. Even so, I think it's useful to think about these things. I wonder if Erlang makes it easier to be truly precise about something.
OTOH, the actual phrasing here, while awkward, doesn't, to me, present the interpretation you offer of "What does Erlang force me to do?" -- indeed, there's no simple way of keeping the basic structure and moving "only" in that sentence to get that meaning (the closest sentence I can see which can communicate that meaning is "What is/are the only thing/things I can do in Erlang?")
That's not to say its not both awkward and ambiguous -- "What can I only do in Erlang?" -- even absent context -- is hard for me to read as anything other than an awkward phrasing of either "What can only I do in Erlang?" or "What can I do only in Erlang?", the latter of which is apparently the intended meaning, the former of which asks about the authors unique strengths in Erlang that aren't equalled by any other programmer.
In all spoken language (as well as computer language), context is king, so we all knew what the title meant, your sandpaper notwithstanding.
If the author was trying to ask "What does Erlang force me to do?" then he would have written that.
However, to respond ...
> "Only" is an adverb in this case,
> and can be placed anywhere in the
> sentence for stylistic reasons.
Only what can I do in Erlang?
What can only I do in Erlang?
What can I only do in Erlang?
What can I do only in Erlang?
What can I do in only Erlang?
> we all knew what the title meant,
> your sandpaper notwithstanding.
But what I take away from your comment is that, in your opinion, we should just go with the most plausible interpretation we can think of, and not worry too much about being too precise. You are clearly in the (possibly overwhelming) majority.
But the need for style in writing is super-important.
If I was telling you about how to beat the 2nd level of Legend of Zelda, I might say
You can only get to the final boss by taking the door on the right.
You can get to the final boss only by taking the door on the right.
Also note that in neither of these cases am I implying
You can only take the door on the right.
Consider your example. You offer:
You can only get to the final boss
by taking the door on the right.
On the other hand:
You can get to the final boss only
by taking the door on the right.
The question is whether, for any given case, stylistic effects trump less ambiguity. People seem to think it does more often than I do.
> Anything can be done in any language, so the technical answer is "nothing". But the spirit of the question begs for an answer, especially in a company situation with deadlines.
The answer addresses the question I think the questioner intended to ask. What the answer does not do is fully disambiguate the question as asked, nor does it even point out that the question as asked is ambiguous. It just takes what it thinks is the most obvious interpretation, and answers that.
Which is fair enough. Most people are going to adopt that interpretation and understand the intent of the question. That's not addressing the point I'm trying to make, which is that the question as asked is ambiguous, and yet a simple re-wording would not be ambiguous. Or at least, not as ambiguous.
But I guess my point is lost, and people will down-vote this thinking I'm just picking nits. Lost cause, but I honestly don't understand why programmers of all people should care so little about the careful and precise use of natural language.
They appear not to, and I'll have to take that as an axiom without understanding.
To be honest, your reply has made me stop completely and reassess everything. In particular, the fact that you of all people seem not have missed completely my intended point makes it perfectly clear to me that I quite simply have failed utterly to make my point at all. I know your posts and comments on HN, I recognize your nick, so to have you not see the point means that, somehow, it's simply not there to be seen. I'll have to go away and think again as to whether it is in fact possible to make the point, and if possible, whether I'm a good enough writer to do so. Current evidence suggests not.
And finally, you say ...
That's actually the first thing the
answer in the post addresses ...
Probably. I think most of us saw the title and reacted like the guy posting "well, technically, nothing" and moved on to the however, which is a fairly straightforward list of "what are the things this system is the best at", which is mostly what we're interested in as programmers and/or startup people.
I didn't downvote you, of course, since people can't downvote stuff on articles they have posted.
> I didn't downvote you, of course, since
> people can't downvote stuff on articles
> they have posted.
>> Perhaps we're just talking past each other.
> Probably. I think most of us saw the title
> and reacted like the guy posting "well,
> technically, nothing" and moved on to the
> however, which is a fairly straightforward
> list of "what are the things this system is
> the best at", which is mostly what we're
> interested in as programmers and/or startup
The question as asked is ambiguous. People are putting a reasonable interpretation on it and answering that interpretation, and that's fair enough. I'm trying to point out that more precise use of language costs little, and creates significant benefits in more effective communication.
Seems it's a lost cause, even among programmers, for whom ultra-precise communication is part of their work.
I would use Erlang for anything I would have used Node.js for and I haven't touched Node.js since I've learned Erlang (and Elixir).
- Erlang allows both CPU and IO concurrency.
- Like Node.js, all I/O in Erlang is non-blocking. The difference is that in Erlang your code looks synchronous. There is no callback/promise/whatever sewage all over your codebase, the Erlang VM and its schedulers take care of that for you.
- Messaging is built directly into Erlang, so communicating across OS processes, and within processes, is trivial. In Node.js you have to implement this yourself and will screw it up.
- Node.js doesn't give you any tools for fault-tolerance or distribution, Erlang is best in class for this.
- Because Erlang is CPU concurrent and Node.js only offers I/O concurrency, Erlang programs are incredibly consistent in their response times. In Node.js, as your programs grow and take up more CPU time (thus blocking the event loop) you will notice your 99th percentile response times are way above the average.
It also has benchmarked extremely well...
If I have to build a distributed system, I will pick Erlang/OTP over Haskell any day.
Everything in Haskell is wonderful, including concurrency. It does not have a built in DNS system (for node discovery and connection), inter-node RPC in Cloud Haskell is a joke compared to Erlang, and many other issues.
I've built systems where I used Erlang as the distributed "coordinator" and Haskell for on the machine computing.
Not quite sure what HTTP has to do with message queuing, other than being a protocol one might use. If scale is so important, perf is a concern, in which case UDP would be a consideration. Would I go down that road? Doubt it, because I suspect the acceptable answer lies somewhere between HTTP and UDP, and I also suspect that better gains can be made by optimising end-to-end throughout and solution design rather than worrying about the semantics of a protocol. What's the benefit, for example, of streaming data over a TCP connection when that data is XML? TLV or some other binary format will serve you a lot better.
Final words on scale - asynchronous delivery (such as message queuing) is one technique. Another is to scale servers out and/or up. Also, as already mentioned, server state and of course pipeline/routing efficiency. Whilst I stand by what I said, I've not had the opportunity to work at the scale you have, so I'd love to know what problems you've encountered when scaling to millions of nodes.
As a former TIBCO and MQ developer, I could not have done without them when developing trading applications.
I'm reaching here, please forgive (and correct) me if I'm wrong. This comment makes me think that you've not tried delivering the same functionality without TIBCO and MQS. I used to do a lot of MQS and MSMQ work, and have not encountered a scenario these products handle that I can't as described in my link above.