
KafkaHQ - rickette
https://github.com/tchiotludo/kafkahq
======
wiradikusuma
If you're considering Kafka, also check Pulsar
([https://pulsar.apache.org/](https://pulsar.apache.org/)). Also the
dashboard: [https://github.com/apache/pulsar-
manager](https://github.com/apache/pulsar-manager)

~~~
arnon
Why should I consider Pulsar instead of Kafka?

~~~
manigandham
Separates storage from brokers for better scaling and performance. Millions of
topics without a problem and built-in tenant/namespace/topic hierarchy.
Kubernetes-native. Per-message acknowledgement instead of just an offset.
Ephemeral pub/sub or persistent data. Built-in functions/lambda platform.
Long-term/tiered storage into S3/object storage. Geo-replication across
clusters.

~~~
philote
I'd looked into Kafka before and really liked the partitioned topics and
specifying partition keys. This makes sure only one consumer handles all
messages for a specific key. Is there a way to do this in Pulsar? I see they
have partitioned topics but it seems to only for performance reasons, but I
don't see how you'd set up a partition key or make sure a specific key would
always go to the same consumer.

EDIT: Also wanted to note that, for Python, there is some support for async
clients in Kafka (aiokafka, asynckafka) but I don't see any async libs for
Pulsar yet. Maybe I'm missing something.

~~~
manigandham
Yes, you can tag messages with a 'key' for compaction (maintaining the latest
message for a given key) and there's now a beta feature for key-ordered
consumers on the same topic: [https://pulsar.apache.org/docs/en/concepts-
messaging/#key_sh...](https://pulsar.apache.org/docs/en/concepts-
messaging/#key_shared)

You can also just use separate topics since they're very lightweight and can
be nested under the tenant/namespace hierarchy.

------
pul
We're using AWS's managed kafka (MSK) which doesn't have a dashboard, so we're
using KafkaHQ. I wouldn't say it's perfect, but does the job pretty well. In
retrospect, I think we should have gone with Confluent cloud, which has all
this functionality + a schema registry + KSQL + Kafka connect built in. It's
also pay-as-you-go, instead of >500 euro per month.

------
hotcrossbunny
[https://operatr.io](https://operatr.io) also expanding into this space

------
mau
We are currently using Kafka Manager ([https://github.com/yahoo/kafka-
manager](https://github.com/yahoo/kafka-manager)). It seems Kafka HQ have the
same features (it is not clear from the doc if you can actually manage the
partition arrangement or only view it) plus the possibility to view the actual
messages that are streaming in Kafka that might be very useful.

~~~
tchiotludo
View message inside Kafka was my main goal when building KafkaHQ !
Unfortunately, you can't, for now, reassign the partitions, but maybe I will
can add the feature when this one will be ready :
[https://cwiki.apache.org/confluence/display/KAFKA/KIP-455%3A...](https://cwiki.apache.org/confluence/display/KAFKA/KIP-455%3A+Create+an+Administrative+API+for+Replica+Reassignment)
(targeted for Kafka 2.4, but I don't think this was release)

------
tspann
SMM is another option and has some really cool features for alerting as well.
Runs in any cloud or on premise.

[https://blog.cloudera.com/smm-1-2-released-with-powerful-
new...](https://blog.cloudera.com/smm-1-2-released-with-powerful-new-alerting-
and-topic-lifecycle-management-features-with-schema-registry-integration/)

------
xref
How does this compare to Kafdrop which hit the HN front page a couple weeks
ago? Looks like Kafdrop may be read-only?

[https://news.ycombinator.com/item?id=21788863](https://news.ycombinator.com/item?id=21788863)

------
cfontes
I use Kafka with SSL/SASL on K8S and last time I checked it's not easy or fast
to put any of this dashboards to work with it.

I think I will try again, maybe it's easier now.

~~~
tchiotludo
I confirm, this works ! Behind KafkaHQ is the official Kafka java client, so
you can't connect to every cluster with any kind of security.

------
ryanong
I used Kafka at my last company. One of the places that it failed us was
trying to use it as a queue. Don’t do that.

~~~
continuations
How did it fail you as a queue?

Did you use it successfully as something else?

