
It’s Okay to Store Data in Apache Kafka - boredandroid
https://www.confluent.io/blog/okay-store-data-apache-kafka/
======
Confiks
Is there anyone here running Kafka in production as primary data store for an
web application backend? Did you write a very custom integration (or indeed a
ES architecture), or did it just fit into existing (MVC) frameworks?

I appreciated the mid-article note that "treating Kafka as a primary store
does mean raising the bar for how you run it". Databases and filesystems are
silently scary.

~~~
papaf
I imagine that you have to use an Event Sourcing architecture because if you
use Kafka as a primary datastore you need to add a lot of complexity outside
of Kafka to make it usable.

For instance, you cannot query Kafka and this is a basic requirement of most
data stores. To get queryable data, you could build up state from events in
some external component and then query that external component.

From what I understand, this building of state from events is an example of an
Event Sourcing architecture.

I think event sourcing as a primary datastore is a cool idea. Its fun to play
with such ideas on small projects and then scale up. The downside is that
Kafka is a distributed monster that is hard to run locally (you need multiple
docker containers for a basic setup). I'd be very interested to see if anybody
else answers your question.

~~~
hactually
Yeah, i had to mess around heaps to get it working nicely on my Mac. Hope this
helps someone!

[https://gist.github.com/17twenty/d619f922ab209f750630824f7c6...](https://gist.github.com/17twenty/d619f922ab209f750630824f7c6836e3)

