Cool, would be nice to see more competition for Conduktor. I have worked with a few “Kafka is a database” people in my career and even if you agree with them, it’s still a terrible one from a tooling perspective. Having to debug Kafka issues on topics with almost any volume of data is slow and frustrating, either with a GUI or command line/REPL tools. Your only hope (if you absolutely have to wrangle Kafka in the core of a project) is heavy testing with the test driver from the start and hoping to avoid getting into too many knots. It’s interesting to note that Conduktor’s more enterprisey features are in this area now.
> Having to debug Kafka issues on topics with almost any volume of data is slow and frustrating, either with a GUI or command line/REPL tools.
Agreed. The scripting feature in Franz is born out of my frustration with some of this stuff and seems like a promising route for quickly visualizing and interacting with what’s happening on a topic (though it’s early days and I’ve got lots of work to do until it becomes truly useful).
It works well with master data [1], i.e. semi-static data of limited volume that is widely needed throughout systems. Think physical store locations and such.
Services can either store it in their own database and still get an update stream, or services can read the complete set on boot and keep it in memory. It can avoid the need for RPC/REST calls so centralized services that keep track off this kind of data, preventing cascading failures if those fail.
Thanks! Yes, I plan to eventually sell it. I’m debating with myself between doing a shareware model where it’s perpetually free with the occasional nudges to buy a license or a more straight forward trial plus $30-50 license. Probably the latter would be more sustainable.
Sorry about that! My audio recording skills are subpar. Yes, likely my next video will be a walkthrough of some of this code. I’ll do my best with the audio on that one.
At the same time, RedPanda provides a native command-line tool named "rpk" which works on macOS as well and which can manage topics and consumer groups. Which makes me think that not only would RedPanda users select this, but perhaps it's also a better generic CLI to talk to Kafka?
what kind of authentication mechanisms does it work with? We have an oauth and ldap backed cluster. This tool would be a nice drop in tool for diagnosing issues.
I'd like to second this request. All of our Kafka clusters are in MSK with IAM auth. I've been desperately looking for another management tool to replace Conduktor (conduktor.io).
