

Asynchronous Loggers for Low-Latency Logging - justin_hancock
http://logging.apache.org/log4j/2.x/manual/async.html#Performance

======
remkop
Main LMAX Disruptor site: <http://lmax-exchange.github.io/disruptor/>

Related Google Group: <http://groups.google.com/group/lmax-disruptor>

API is not as easy as a queue, but the results speak for themselves...

~~~
galaktor
Also, using disruptor (efficiently) forces a certain threading model onto your
app (at least downstream of the ring).

~~~
remkop
The threading model is part of the "Mechanical Sympathy". Multiple threads can
write into a ring buffer, _without locks_. On the other end, coming out of the
ring buffer you can have one thread (which means that the biz logic never
needs to lock anything and can be blazingly fast), or multiple threads doing
something with the event independently (at LMAX they journal to disk and
replicate to backup machine). These threads can process the event in parallel.
And you can "pipeline" or stage threads, so after both journaling and
replication are complete (and not earlier than that) the event is passed to
the business logic.

Then if the business logic needs to do something slow, like network I/O, disk
I/O, database access etc, you would set up one or more other Disruptors for
that. Instead of doing the slow task in the biz logic thread, you'd post a
task for that on the helper Disruptor. That way the business logic can stay
single-threaded, uncontended and blazingly fast.

When the slow task (disk I/O or whatever) is done, it will post a new event
onto the first Disruptor, so the business logic can pick up the result and
process that.

So it's all asynchronous, all event-driven, but pretty flexible.

In Log4j 2.0 there is just one Disruptor that takes the log message, puts it
in the ring buffer, and passes control back to the application. Coming out of
the ring buffer the event is passed to the normal log4j loggers, parent
loggers and appenders.

It is not completely zero-garbage, but all ring buffer slots are created up
front and re-used during the life of the process. Coming into the ring buffer,
log event parameters are copied into the ring buffer slot without creating new
LogEvent objects. Slots in the ring buffer implement the LogEvent interface,
so coming out of the ring buffer, the slot itself is passed downstream, again
avoiding object creation. Other parts of log4j are not that garbage-conscious
so there is still room for improvement in this area.

~~~
krenoten
That's a great description of how it works - looking forward to reading more
of your comments on HN in the future.

------
capkutay
I wish "Mechanical Sympathy" was defined/explored more in computer science.
Googling mechanical sympathy just leads me to a blog...although it's an
extremely interesting blog and tied to the idea of mechanical sympathy.

Also, LMax Disruptor looks really cool on the surface (can't wait to dive in).
However, I haven't found too much literature that talks about its possible
tradeoffs. Glad this story was posted.

~~~
benjaminwootton
Check Martins talks on the topic too - <http://www.infoq.com/author/Martin-
Thompson>. Very interesting indeed.

