
Go, D, Erlang and C in real life: MQTT broker implementation shootout (2013) - BufordTJustice
https://atilanevesoncode.wordpress.com/2013/12/05/go-vs-d-vs-erlang-vs-c-in-real-life-mqtt-broker-implementation-shootout/
======
rdtsc
> What about readability and ease of writing? I can’t read Erlang so I can’t
> comment on that.

The Erlang code looks very nice. If you read this, great work Patrick!

[https://bitbucket.org/pvalsecc/](https://bitbucket.org/pvalsecc/)

Nice use of gen_fsm + binary matching.

Here is an example of the client code that takes only 200 lines:

[https://bitbucket.org/pvalsecc/erlangmqtt/src/f37505188c1f1c...](https://bitbucket.org/pvalsecc/erlangmqtt/src/f37505188c1f1c62aa2b4b94121330d779975d3d/src/mqtt_client_protocol.erl?at=default)

~~~
playing_colours
Yeah, it all depends on code authors. This is one more example of an mqtt
broker in Erlang - VerneMQ; and it's very readable:
[https://github.com/erlio/vmq_server](https://github.com/erlio/vmq_server) We
are currently estimating it.

~~~
dergraf
Thanks, we're working hard to improve our codebase even more! :)

Umbrella Project can be found on
[https://github.com/erlio/vernemq](https://github.com/erlio/vernemq) as well
as our website [https://verne.mq](https://verne.mq)

------
tedchs
This being from 2013 and using Go 1.2, I'd be interested to see an update
using Go 1.5.

Thanks to the author for including standard deviation numbers and not just the
averages like so many other "benchmarks".

~~~
voidlogic
Thanks, came here to say this. Go GC and code generation are improving all the
time.

------
nanny
I really can't imagine that this benchmark is relevant anymore. Haven't Go, D,
and Erlang had major changes since then? E.g., Go is now compiled by a
compiler written in Go.

~~~
atilaneves
I'm planning on updating it this year once I finally find the time to learn
Rust. Developing an MQTT broker has become my go-to task for learning new
languages.

~~~
pron
How about Java, then? It's a little less esoteric than most of those
languages.

~~~
atilaneves
Funny you should ask:
[https://atilanevesoncode.wordpress.com/2014/01/08/adding-
jav...](https://atilanevesoncode.wordpress.com/2014/01/08/adding-java-and-c-
to-the-mqtt-benchmarks-or-how-i-learned-to-stop-worrying-and-love-the-garbage-
collector/)

~~~
pron
So Java is the pingtest winner, and I'm sure that with just a little more work
it can be the loadtest winner as well.

~~~
atilaneves
I doubt it, Patrick properly profiled and optimised it. Later on I got the D
version to within 3% of Java's performance in pingtest.

~~~
pron
It's clearly optimized for latency over throughput. For one, it is single
threaded -- unlike the Erlang, Go and, I believe D (vibe.d) implementations.
Also, I don't know exactly how the benchmark was performed, but Java takes
time to warm up (otherwise you're testing a completely different
implementation of Java, which is only used at startup time).

I've found that for long-running server-side apps, Java is very hard to beat
in terms of performance, especially relative to the effort spent. I also think
it's a nicer language than Go (I also prefer it over C++ and D).

------
nailer
> pingtest (latency - bigger is better)

This is an odd way to measure latency. Could someone explain further?

~~~
atilaneves
As someone pointed out to me later, it's not really latency but latency-
constrained throughput.

------
hlieberman
"Since the Erlang unit tests are in the same files as the implementation, it’s
hard to know exactly how many lines long it is. It gets worse since it
implements most of MQTT, the D implementation essentially only implements
what’s necessary to run the benchmarks."

Benchmarking the entire spec versus only the minimal set is almost certainly
part of the problem here. If you want to benchmark implementations against
each other, you should probably make sure they implement the same thing!

~~~
atilaneves
No, not really. I understand why you'd think that but knowing the MQTT spec
and the benchmark, it shouldn't make any difference at all.

~~~
xixi77
Are you sure about that one? I am thinking about the situation where presence
of alternative code paths that never actually get executed can lead to fairly
large differences in timings, particularly for tight loops. (At least in
computational code; I'd expect it to be much less common for protocol handling
benchmarks like these...)

~~~
atilaneves
Pretty sure yeah, I looked at a lot of profiling logs.

------
StavrosK
This is slightly offtopic, but MQTT says it can provide "exactly once"
delivery. I thought that wasn't possible?

~~~
taralx
It is possible if you are prepared to wait for an unbounded amount of time.

------
atilaneves
I'm the author but not the OP.

~~~
throwawayaway
Out of morbid curiousity, how many lines of code did the C implementation
clock in at?

~~~
atilaneves
~20kSLOC IIRC. It's not a fair comparison since it does more, but let's face
it, if these implementations did the exact same thing I doubt they'd pass the
5kSLOC mark.

~~~
throwawayaway
I get about 1.5-2x for c vs c++. Genuinely surprised that erlang took so many
lines of code.

------
merb
did you use GOMAXPROCS=$YOUR_CORES ? or just the standard in 1.2? Actually
golang is really good in multithreading. Next you also have a Java example,
did you warm it up or not? Actually thats why I hate benchmarks, since mostly
they are screwed since most people who are writing benchmarks actually knowing
just a little of each language. (actually they just want to show that "their"
language is good)

~~~
atilaneves
Back then I tried GOMAXPROCS from 1 to 8 and it didn't make much of a
difference. Later on with a different version of Go (I don't remember which),
increasing GOMAXPROCS to 2 made a difference, but any more than that was
pretty much the same. In any case (and again, the last time I tried), the Go
implementation with GOMAXPROCS=2 was the slowest even though it was using
twice as many threads as the other ones!

As for showing "their"/"my" language: there were implementations from 4
different sources, and I didn't write the benchmarks; the guy who wrote the Go
implementation did (and that's why the benchmark app is in Go).

------
radmuzom
It's interesting to see the lowest throughput provided by the C implementation
for the 1k case. Any idea why?

~~~
atilaneves
No, I didn't dig deeper.

------
Gepsens
2013 ? Would gladly see the same benches with Go 1.5.

------
wehadfun
Is there any hardware that uses MQTT?

~~~
playing_colours
There is. For example, we use it for our hardware product:
[https://www.relayr.io/products-
services/wunderbar/](https://www.relayr.io/products-services/wunderbar/)

------
areusch
update from go 1.2? since there have been many performance improvements since
then...

~~~
tcfunk
It seems this article was posted in 2013. Not sure where that falls in the Go
release timeline, but I'm guessing the version wasn't was far behind then as
it seems now.

Edit: After a quick search, it looks like Go 1.2 was released only 4 days
prior to this article being posted.

~~~
atilaneves
I run Arch Linux, all the versions were whatever was newest at the time.

