
135M messages a second between processes in Java (2013) - matteuan
http://psy-lob-saw.blogspot.com/2013/04/lock-free-ipc-queue.html
======
carsongross
The JVM is a treasure just sitting there waiting to be rediscovered.

It really is a shame that there is so much noise and unnecessary complexity
around using it.

~~~
wtbob
> The JVM is a treasure just sitting there waiting to be rediscovered.

It's the result of man-centuries (man-millennia?) of work, yes, and as a
result is really, really impressive. But imagine if that much work had been
put into something fundamentally great, like Smalltalk or Lisp, rather than
something fundamentally okay like Java.

> It really is a shame that there is so much noise and unnecessary complexity
> around using it.

Very true. Java itself is not a terrible language (hence why I wrote
'fundamentally okay' above); it's the arcane, rococo, lunatic levels of ritual
which surround it. Every time I look at Java code I'm reminded of some insane
fantastic court in which one has to plead with the Minister of Small Affairs,
make a sacrifice to the God of Bureaucracy, pay for a Token of Token-Paying,
appeal to the good graces of the Wise Undersecretary of Vice-Small Affairs and
finally spend a weekend climbing an Escher Staircase before an audience with
the King, all just to get a scrap of toilet paper.

~~~
cageface
If Smalltalk and Lisp are so magical then why didn't they win?

~~~
robotresearcher
Because Worse is Better applies here as usual. Java was fast enough and
familiar enough and the libraries were good enough and the language safe
enough and Sun supported it well enough in the enterprise. Nothing else had
the same combination of okayness. I never enjoyed Java but it had my half-
grudging respect.

~~~
cageface
I'd agree with you with the caveat that "worse" in that paper means that
something might be less elegant or rigorous on a theoretical level but still
the more practical choice.

And now that we have good, modern high level languages with robust but
flexible type systems I'd choose Haskell or Ocaml or maybe Swift over
Smalltalk or Lisp if I wanted to stick my neck out technically on a project. I
consider dynamically typed languages to be significantly "worse" in an
absolute sense for building non-trivial software.

~~~
wtbob
> And now that we have good, modern high level languages with robust but
> flexible type systems I'd choose Haskell or Ocaml or maybe Swift over
> Smalltalk or Lisp if I wanted to stick my neck out technically on a project.
> I consider dynamically typed languages to be significantly "worse" in an
> absolute sense for building non-trivial software.

I don't know if there are any typed SmallTalks, but Common Lisp has optional
static typing: you can declare types and the compiler will enforce them. This
seems to me the best of both worlds: the freedom to interact with your program
dynamically, with the safety of strong static types.

~~~
wtetzner
But with Common Lisp, you don't get the safety of strong static types. If the
compiler can prove that there's a type error, it will throw an error. That's
not the same as throwing a type error when the compiler can't prove that there
_isn 't_ a type error, as a sound type system would.

Even worse, Common Lisp will use your type annotations to optimize code, which
can change the behavior of your program. It's better to think of them as type
hints for optimization than to think of them as optional static types.

~~~
junke
That's true, because doing so in Lisp is undecidable and because you allow
function definitions to change (that's why type checking works better with
inline and local functions).

The type of analysis applied to Lisp is adapted to its type system: you have
ranges of values, unions, etc. In particular SBCL can exploit some
relationships between runtime values and types (I detailed it recently, so I
prefer not to repeat myself:
[https://news.ycombinator.com/item?id=12222404](https://news.ycombinator.com/item?id=12222404))

As far as I am concerned, it is already quite powerful to detect the type-
related errors that I tend to do. The part that cannot be decided during
compilation is left to be checked at runtime, which is strong dynamic safety.
Instead of manipulating untyped bits, you have all you need to never corrupt
the state of your program. The good news is that if implementors make progress
with type inference, you don't have to change the language: more checks can
move from runtime to compile-time (the hard stuff would be the check the
"satisfy" type ;-).

With some types left to be checked at runtime, you end up in a situation that
is no worse than statically typed programs which throw exceptions for other
kinds of errors. Sometimes, those errors arise because programmers implement a
dynamic things, like ad-hoc tags to identify objects at runtime (so this is
basically a dynamic type system in disguise). Other errors related to safety
are tied to temporal issues, deadlocks, permissions, priority, security, etc.
You can't always encode them with types. There are formal tools for dealing
with them, but at this point, whether you use them generate safe code in Lisp
or OCaml is not very important (and generating Lisp code is straightforward).

> Even worse, Common Lisp will use your type annotations to optimize code,
> which can change the behavior of your program.

This is a strange concern. The modified behavior belongs to the set of
behaviors that would be possible without checking types, not a new, faulty
behavior that contradicts what the code says.

------
jpalomaki
There's also more light weight alternatives to "XML heavy traditional
frameworks" on Java side. Check for example Play[1], which makes the "write
some code, test in browser, write more, test" cycle possible.

[https://www.playframework.com](https://www.playframework.com)

------
atemerev
Java in HFT environment is wonderfully perverse and beautifully abused in so
many ways.

~~~
oppositelock
Yeah, it's amazing how fast it goes when you give it tons of RAM and turn off
GC.

~~~
bitmapbrother
Or you could just use Azul Systems pauseless JVM and not have to worry about
the GC ever.

------
srtjstjsj
Is there any reason someone would want to run multiple Java processes
communicating at high speed? It seems useless, since the massive overhead of
each process makes IPC useless in a high-performance environment.

~~~
noselasd
I'm not sure why you think there is massive overhead, can you explain a bit
more about that ?

~~~
ketralnis
Not the parent but the biggest overheads are going to be startup time (which
is easy to work around by just bringing up the procs and keeping them there)
and memory overhead. There's some extra memory used for keeping all of the
code or other shared resources around but let's assume that's negligible.

Because the JVM never gives memory back to the OS, you have to give it the
maximum amount of memory that it could ever want, plus enough to last you
between GC runs. Often this means that you have to give it _at least_ twice
the memory it actually needs. If you have more than one process, you end up
having to multiply that extra "slack" memory by the number of processes that
you have, which means that you have all of this wasted memory sitting around
all of the time. "Memory is cheap" is a common attitude but in my experience
in web stuff it's the only resource that's anywhere close to 100% used on an
app server. So things with high memory overhead mean that you need more
machines or bigger ones.

Of course, whether that matters depends on the actual numbers of how much
memory is needed, how much is "wasted", whether that's actually your
bottleneck, etc.

~~~
kasey_junk
In my experience on the web its all IO bound.

If you are using multiple JVMs and high performance IPC libraries you have
largely already solved the memory bound problem on the JVM with standard
techniques like arena allocation.

------
xchip
Moving blocks of memory around is something I am expecting java to do quite
fast, just because the VM has intrinsic instructions to do that. You could do
the same thing in any other language.

~~~
kasey_junk
The thing to remember though Is that these libraries are type safe, fast &
general purpose.

I bemoan the lack of a similar library in go for instance because the lack of
generics.

You can certainly do the same thing in C as well, but I can't easily mix it
with code that has automatic memory management.

As for dynamic languages, can you actually do this? I'm not saying its
impossible, I've just never seen it.

~~~
srtjstjsj
> these libraries are type safe,

not in this blog post:

    
    
        UnsafeAccess.unsafe
        UnsafeDirectByteBuffer.getAddress

~~~
kasey_junk
Those are wrapped in type safe methods though (all type safety cross process
has this step regardless of language/lib).

In nontrivial examples you tend to wrap more than Java longs but the
techniques are the same.

EDIT: I realize now that I was bringing a lot of context with me that is not
obvious just from the blog post. Martin Thompson and Todd Montgomery who are
mentioned in this post (and Richard Warburton who isn't). Are the main authors
of the real-logic libraries, which are extremely well known in JVM performance
circles. Aeron in particular is a great IPC lib. Other libraries that are
similar and in the same circle are Open-HFT Chronicle and the LMAX Disruptor.
For that matter the author of this blog post @nitsanw is pretty well known in
that arena.

------
option_greek
With all those memory accesses marked as unsafe, I'm wondering why not just
use C++.

------
bogomipz
Can someone elaborate on this:

"IPC, what's the problem? Inter Process Communication is an old problem and
there are many ways to solve it (which I will not discuss here)."

How is IPC an old problem and how was it solved?

~~~
tlb
It was a big focus of the Mach kernel in the early 90s. It was a microkernel,
meaning that they moved as much functionality as possible out of the kernel
and into user processes. For example, to access the file system each call
(open, read, write) was really a message to a file system server process. So
IPC performance mattered a lot.

Here's an example some heroic attempts to make it fast, getting down to about
50 uS on a 1991 CPU:
[http://homes.cs.washington.edu/~bershad/Papers/p175-bershad....](http://homes.cs.washington.edu/~bershad/Papers/p175-bershad.pdf)

~~~
bogomipz
Thanks for the link.

------
leeoniya
(2013)

~~~
biot
Has this metric been surpassed since?

~~~
srtjstjsj
Of course. Since this is a hardware-bound metric, newer faster harder can
transmit more messages.

A more timeless metric would be something like "30-instruction message-passing
on x86"

~~~
biot
Rephrased: given the same hardware, has anyone found a technique that results
in greater throughput?

~~~
geocar
Yes, absolutely. I use KDB to get billions-of-transactions-per-second, but
then an interpreted language is clearly better than the JVM.

Put grossly, the "technique" isn't well defined; Even things like "message"
aren't well defined, and at the end of the day, are we really talking about a
new-and-improved memcpy[1]? Or are we talking about something that's less-slow
than other things (and if so, what?)

Not saying this isn't interesting, just that I don't _know_ if it's
interesting. Something I want to know is when Java programmers struggle to get
into 200k/sec HTTP requests, can I point them at this?

[1]: [http://www.codeproject.com/Articles/1110153/Apex-memmove-
the...](http://www.codeproject.com/Articles/1110153/Apex-memmove-the-fastest-
memcpy-memmove-on-x-x-EVE)

