

Dataflow Programming: Handling Huge Data Loads Without Adding Complexity - bergie
http://drdobbs.com/go-parallel/article/231400148?pgno=2

======
Sukotto
OP links to page 2 of the article. Here's page 1 <http://drdobbs.com/go-
parallel/article/231400148?pgno=1>

------
alecco
Beware of deceiving graphs. That system scales with many cores compared to
_itself_. Erlang isn't easy to program.

Depending on the case, you might be better off moving the critical part of the
data processing to C or C++, perhaps using a relevant library. On many
benchmarks Erlang is 10x slower than C. And more importantly for huge data,
Erlang consumes 3x to 30x more memory.

[http://shootout.alioth.debian.org/u64q/benchmark.php?test=al...](http://shootout.alioth.debian.org/u64q/benchmark.php?test=all&lang=hipe&lang2=gcc)

Edit:

Relative to C/C++, Scala is somewhat better in cpu (still slower) but memory-
wise it's even worse:

[http://shootout.alioth.debian.org/u64q/benchmark.php?test=al...](http://shootout.alioth.debian.org/u64q/benchmark.php?test=all&lang=scala&lang2=gcc)

~~~
zokier
Dataflow as a pattern is not constrained to Scala or Erlang specifically. In
fact, the "Further reading" article[1] is about dataflow programming in C++.

[1]: <http://drdobbs.com/cpp/224202533>

~~~
alecco
Interesting. But then why didn't the author benchmark that too? Pretty graphs
like that can confuse people not making a deep analysis of what's going on.
There are no disclaimers or notes about it.

~~~
igouy
Figure 2 is a very ordinary line graph, and the text tells you exactly what
it's intended to show - "good scalability as the compute resources increased".

When I make mistakes reading, it is most often my fault for not paying
attention and not reading carefully.

~~~
alecco
The point of the whole article is "proven" with that test/graph. It neither
includes a simple scalar version nor versions on different languages. And they
are recommending specific languages (Scala/Erlang).

