

Bruce – Unix domain socket to Kafka daemon - fintler
https://github.com/tagged/bruce

======
kodablah
We have implemented a similar pattern with
[https://github.com/wavii/darner](https://github.com/wavii/darner) and a
daemon to take the messages and move them to AMQP out of band. It is a healthy
pattern to send asynchronous messages to a local daemon before sending them
out. Takes the external server out of the path of failure.

------
burke
We do a similar thing using SysV Message Queues. Other than being
operationally weird due to their age, SysVMQs are actually a really great fit
for this problem.

------
AceJohnny2
I hadn't heard of Apache Kafka before. As branding goes, it's hard to do
worse.

In my mind, Kafka is synonym with complexity, being obscure or byzantine, and
as opposite to "useful" as I can imagine. See also: nightmare.

~~~
th0ma5
Maybe they were trying to channel the idea of the run on sentence which is
somewhat more taboo in English than in German, and leads to the Kafka labored
style, which he purposefully did. If I try to think of a positive angle, it is
that the system can "keep going" perhaps? Anyway, all this stuff is great
though.

~~~
dragonwriter
Run-on sentences aren't taboo, they are just grammatically incorrect. _Long_
sentences may be taboo, but that's a different thing.

------
endersshadow
Interesting. I really like the idea of using the UNIX sockets to be,
essentially, a message queue before going to Kafka. This may be used on my
next project.

~~~
dspeterson
There is a dedicated thread for monitoring the socket so clients don't block
trying to write. In case of transient load spike or slowness in Kafka, the
bulk of the queueing should occur inside the daemon, where you can see queue
lengths via web interface. God luck with your project :-)

