
LibPhenom, a high-performance C eventing framework from Facebook - jamesgpearce
http://facebook.github.io/libphenom/
======
astrodust
Does anyone know how this compares to the standard libevent or the quirky
allegedly faster libev
([http://software.schmorp.de/pkg/libev.html](http://software.schmorp.de/pkg/libev.html))?

~~~
wezfurlong
We're a bit faster than libevent in terms of dispatch throughput; some
benchmarks in this commit message:
[https://github.com/facebook/libphenom/commit/41b6106f04fe62c...](https://github.com/facebook/libphenom/commit/41b6106f04fe62cb1d86c9682ded55ab3be980a9)

The `tests/bench/iopipes.t` "test" allows you to play with some concurrency
parameters to try this for yourself on your hardware.

We haven't compared against libev.

We've added some more APIs (buffers and sockets) since those benchmarks were
done and we don't have numbers to share around those yet.

One key difference between libevent, libev and libuv is that libphenom is
inherently multithreaded in its IO dispatcher and timeout implementation.

If you're dispatching purely CPU bound jobs, we get very close to linear
scaling with the number of cores:
[https://github.com/facebook/libphenom/commit/c2753c2154a0cff...](https://github.com/facebook/libphenom/commit/c2753c2154a0cffc0bd70e8f28dd0ea884aab4fd)

~~~
mrb
What do you mean by "inherently multithreaded"? I have written a network
application handling 100-500k TCP concurrent connections using libev. It was
multithreaded to distribute the number of connections evenly per thread
(typically from 10k to 100k connections per thread). This is a model that is
perfectly supported by libev. And I observed a nice linear scaling of the
network throughput with the number of cores since my jobs were also purely
CPU-bound. Depending on how CPU intensive my jobs were exactly, my libev code
needed anywhere from 2 to 10 threads to saturate a GbE link.

~~~
wezfurlong
The principal difference between the libphenom event dispatcher and the other
event libraries is that libphenom can wakeup and dispatch IO events to any of
the IO scheduling threads (no thread affinity).

Contrast with the libevent approach of using an application specific scheme to
assign descriptors to an event base associated with a thread (strong thread
affinity).

This makes more of a difference if you have chatty protocols and/or long lived
sessions and no way to rebalance your fd -> event_base mapping.

------
codehero
Why is it every framework handling run loops or communication makes its own
implementation of printf? While printf is ubiquitous, I'd hardly call its
semantics or syntax perfect. Why does everyone have to copy the same mistakes
over and over again?

~~~
escaped_hn
maybe because printf is blocking and will block the event loop.

~~~
codehero
I don't know if that's the reason, but there's no reason formatted output HAS
to:

1) block at the caller, ever

2) have its data parameters pushed onto the stack

3) print all or nothing to a single buffer

4) identify the data type or modifier by the first letter of its English name
(int, long, etc)

5) have its core modified to add functionality

------
capkutay
Can someone explain what eventing frameworks are used for? Are they related to
event messaging or processing frameworks?

~~~
wezfurlong
At a high level, they make it easier to write server (or client) software that
deals with multiple concurrent connections (such as HTTP server software, or
just about any network facing service these days). They do this by abstracting
some of the details away so that you can focus on writing your application
code; instead you declare callbacks that get invoked when you have data
available.

Traditional eventing frameworks focused on non-blocking I/O on a single thread
on the basis that you don't need so many resources to scale up to a large
number of clients when compared to a simple one-thread-per-client model.

There's lot of good material discussing this at
[http://www.kegel.com/c10k.html](http://www.kegel.com/c10k.html)

libphenom is a bit more than just an eventing framework though; we have a
number of APIs that help with putting together the whole application. And we
blur the lines a bit: we also have thread pooling and dispatch support for
cases where your can't build 100% of your application in a non-blocking
fashion.

------
IPGlider
How does this compare to libdispatch
([https://libdispatch.macosforge.org/](https://libdispatch.macosforge.org/))
in functionality and performance? I'm very interested in this.

~~~
wezfurlong
We haven't looked at libdispatch specifically; would welcome your feedback on
how we compare.

We hope we do well here; we've had some nice results:
[https://github.com/facebook/libphenom/commit/c2753c2154a0cff...](https://github.com/facebook/libphenom/commit/c2753c2154a0cffc0bd70e8f28dd0ea884aab4fd)

~~~
nwmcsween
Why do you do -fno-omit-frame-pointer on x86_64 the ABI requires dwarf or is
there something I'm missing?

~~~
wezfurlong
Frame pointers make things easier for tools to get backtraces without
requiring complex dwarf unwinders. The observability is something I value more
than not being able to use that register for other things.

------
iatrou
Looks very promising! It would really help the adoption of such new projects
if there was a bit more effort to present the current status as well as a
roadmap. libphenom strives to be a core component for server applications. It
also has to compete with similar Open Source projects that have been around
for a while, with well known strengths and limitations. A more clear statement
of its current status (is it used in production for Facebook? is it in beta?
are there known caveats?) as well as the future goals would help to build
confidence in the project.

~~~
wezfurlong
Thanks; good feedback. It's not currently in production at Facebook, but like
just about everything we do, we're quickly iterating.

Regarding roadmap, we'll try to keep the issue tracker on Github sync'd up
with the broad goals and milestones, and we welcome feedback there too;
they're a bit more dynamic and easier to update in real time than the docs and
website materials.

------
kev009
Nice to see Concurrency Kit here. I wonder if they should have
extended/patched libuv though.

~~~
marshray
I hadn't heard of CK
([http://concurrencykit.org/](http://concurrencykit.org/)) before.

Do you know of anything else that's using it?

