
Java 7 Fork-Join Calamity - alrex021
http://coopsoft.com/ar/CalamityArticle.html
======
warmfuzzykitten
This article smells strongly of ad hominem attack. The professor who "hasn’t
walked in the shoes of application programmers" is, of course, Doug Lea, the
driving force behind the current java.util.concurrent package. The performance
comparison smells of cherry-picking: parallel add over a Long array, really?
The complexity criticism sounds a bit like "I don't understand this code! It
must be bad!" He accuses the code of bit-twiddling. Oh my.

~~~
jamesaguilar
I don't see any place in the article where it says, "The author is dumb,
therefore the code is bad," or anything remotely like that. The article does
observe that the code is great from an academic but not professional
standpoint, but it gives reasons why that is the case. There's nothing ad
hominem about the article.

If anything, your comment is more of an ad hominem -- rather, the positive
version of the same, an argument from authority -- with the implication that
F/J is probably good since Doug Lea wrote it.

> The complexity criticism sounds a bit like "I don't understand this code! It
> must be bad!"

All complexity criticisms sound this way. Some are true and some are not, and
we must attempt to separate which is which. But dismissing the criticism
because of how it sounds without actually investigating the claims yourself is
not good. "This sounds like X and is therefore dumb" is not a valid
refutation.

It seems like you are trying to undermine some minor points in the article and
claim that the whole thing is bunk as a result. What about addressing the
major points instead. Can F/J scale across thousands of cores? Is work-
stealing task-management actually valid in Java? Is the performance actually
good? Is monitoring and stall detection available? Would a better approach use
the Unsafe extension less? Etc. Otherwise your comment will not add much
except for misleading weak-minded people into thinking the article is
baseless.

~~~
sparky
As scott_s notes below, I'm not sure I agree with the premise that core
libraries need to be understood _much less modified) by the average
programmer. I'd rather they be correct and highly optimized, and I see no
evidence in the article to suggest that F/J is not. Perhaps this will be
viewed as another appeal to authority, but I've looked at the F/J code
(briefly) and parts of java.util.concurrent (in more depth), and they look to
be written in a similar style. Not the easiest stuff to parse, no, but people
seem to be getting along with java.util.concurrent just fine, despite the fact
that all the 'ivory tower' claims in the article apply just as well to that
code.

Moving past that, there are some serious misunderstandings displayed in the
article that make me nervous about accepting the rest of it at face value. For
example:

> _Only the Operating System (O/S) or a pseudo O/S can manage Tasks (whether
> they’re called tasks or processes is all the same thing.) The best an
> application framework can do is manage Threads.*

WTF? Tasks and processes are not the same thing in this context (yes, they are
interchangeable when talking about a kernel scheduler). In the context of
programming models, tasks are just reasonably independent units of work,
usually thought of as within the same overall application. For examples of
application frameworks that get along just fine managing Tasks, see TBB,
OpenMP 3.0, Cilk and variants, Microsoft TPL, etc.

> _The F/J Framework structure precludes scaling. The entanglement of client
> call/server processing, the spare threads necessary to help with the join()
> waiting problem (above), as well as the work stealing code (threads need to
> serially search for work among all deques) only works well on a small number
> of processors. This design can never scale to hundreds or thousands of
> processors. The overhead would kill the benefits of parallelization._

Work-stealing is _designed_ to be more scalable than a centralized task queue
by reducing contention. The idea is that, statistically, you have to serially
search a very small number of neighbor queues before finding some work to
steal. There are scenarios where a (logically) centralized queue wins, which
are well described in the literature, but equating work-stealing with non-
scalability betrays a lack of understanding of how these things actually.work.

This paragraph also echoes a theme throughout the article, which is that F/J
does not fit into the author's mental model that any parallelism framework
should be "enterprise-grade" and should work for a huge farm of machines. I
argue that it's just fine for solutions to exist, and indeed to be present in
the JDK, which prioritize simplicity and speed over logging, fault tolerance,
and monitoring, and which remove the overhead of a strongly separated client-
server model. Furthermore, is there evidence that the enterprise features
couldn't be layered on top of the proposed framework? That would seem to be
the best of both worlds.

Apart from these factual concerns, the _huge_ degree of repetition (honestly,
this article could've been 1/4 the length) and the complete lack of evidence
for any of the performance and scalability claims (the table shown should not
convince anyone) are red flags for me. The fact that the author happens to be
highly commercially motivated to spread FUD is icing on the cake, but isn't
necessary to figure out that these arguments should be taken with a huge grain
of salt.

That said, there are probably some good suggestions in here for how to improve
F/J. I think a lot of them are better suited for a larger piece of
"parallelism middleware", like what the author sells, than to a language's
standard library.

------
wmf
Is the author of this article also the author of a competing fork-join
framework? Although it shouldn't invalidate the technical criticisms, an
undeclared conflict of interest is a good way to sabotage your credibility.

~~~
scott_s
Yes: <http://www.coopsoft.com/JavaProduct.html>

I hadn't realized that. I had assumed he was referring to others' work in this
article. It's very bad form to compare your work to that of others without
identifying that the work is indeed yours.

------
alrex021
My personal disappointment is the lack of expressiveness.

Here is a C# LINQ example:

    
    
      using System.Linq;
      files.AsParallel().ForAll(file => ProcessXML(file));
    

To achieve something similar in Java 7, one could start on perhaps with
improving the java.util.Collection interface, or appropriate abstract
children, by introducing the asParallel method that the Collection
implementations could implement. One obvious problem I see is the lack of
lambda support in Java 7, so one would have to pass an anonymous class into
forXXX(..) instead which is a bit more clunky.

~~~
warmfuzzykitten
That falls under project lambda (and corresponding work on the collection
classes), which won't appear until Java 8. I agree, it would have been better
sooner.

~~~
wmf
It doesn't seem like a good idea to ship a "bad" parallel framework in Java 7
and then ship the "good" one two years later in Java 8.

------
alatkins
While I'm not necessarily disagreeing with everything in this article, there
are an awful lot of assertions made that could really use some citations and
illustrative examples to back them up.

The fact that the author is associated with a fork/join competitor and yet
never acknowledges it also inflates my skepticism by a long way.

------
cwinters
Given the author's reported experience I'm surprised there aren't more posts
from him on the mailing list
([http://altair.cs.oswego.edu/mailman/listinfo/concurrency-
int...](http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest))
where this library, and all the other juc changes, have been discussed for
years.

------
malkia
Coming from C background, there is OpenMP, with with a bunch of #pragma omp
parallel for (and others) can easily turn suitable processes into data-
parallel ones (for example lots of image processing could be sped up this
way). On top of this, if you don't link with OpenMP it stays the old way
(serializable).

------
pnathan
I found the condescension towards "Average Java programmers" offputting.

~~~
scott_s
I also found those two sentences unclear. I can't figure out if he meant that
average Java programmers can't maintain and extend the Fork/Join framework, or
that they can't use it. If the first, I don't expect average programmers of
any language to be capable of maintaining or extending core libraries.

