
Interactive Queries in Apache Kafka Streams - krallistic
https://blog.codecentric.de/en/2017/03/interactive-queries-in-apache-kafka-streams/
======
gopalv
Esper[1] was the go-to solution for these problems, in the last generation -
the reason there are alternatives is primarily because that was GPLv2.

And even from the same stable as Kafka, there's Samza[2]

[1] - [http://www.espertech.com/esper/](http://www.espertech.com/esper/) [2] -
[https://speakerdeck.com/ept/turning-the-database-inside-
out-...](https://speakerdeck.com/ept/turning-the-database-inside-out-with-
apache-samza)

------
lekker
I've been eyeing Kafka for a long time now - tried to set it up on a smaller
DO box and had memory issues. Is it worth using this for smaller operations
(<100k requests per day)? The stream processing seems extremely useful for my
use case. Is there something else that may be a better fit?

~~~
koolba
If they're evenly distributed, 100K requests per day are barely 1/sec. It's
peak throughput for which you should be planning. At 1/sec you can use just
about anything.

That aside, if you're not sure if you should be using it I'm going to guess
the answer is, "No you shouldn't". It's very cool tech but most likely
overkill for whatever you're doing. Also, until recently securing it was a
particular pain due to lack of TLS and authentication (I think this may have
been resolved now).

~~~
lekker
All right, that makes sense. It is likely that we'll see a peak in the 10s of
thousands at a time, if that. Probably more as we move into the future.

I want to avoid deploying a piece of software now that we'll need to swap out
in a year since it does not provide what we need anymore, though I suppose
that is a natural part of scaling.

What would you recommend as an alternative to Kafka that does adequate real-
time processing? The real-time-ness of it is important for our use-case. Maybe
even some kind of time-series database system could work.

~~~
krallistic
If you really need "real-time-ness" TSDB will probably not work. You could go
with sparkstreaming or flink but for that amount that will be overkill. Kafka
scales quite nicely and is relativly easy to operate (compared to spark etc).
But thats only if you really need to scale, for your current usage even kafka
is a bit overkill.

~~~
lekker
All right, thank you. I'll look into the other options you mentioned

------
proyb2
Have anyone using the C version in librdKafka and how was your experience?
[https://github.com/edenhill/librdkafka](https://github.com/edenhill/librdkafka)

~~~
fooyc
Confluent relies on it for their non-Java clients (Python, Go, .Net at least).

~~~
math
Confluent also provide commercial support for it and employ Magnus who
developed it.

I've been working closely with librdkafka for a few months now and find it
fast and reliable. Many companies rely on it in production. In addition to the
Confluent clients, Blizzard has also developed a client that wraps librdkafka
for nodejs (which they use in production).

Disclaimer: I work at Confluent.

