> Maybe in hyper-scaled applications that need to be ultraperformant (e.g., realtime trading, massive streaming platforms) is where Kafka comes into play.
Kafka is used because the Java folks don't want to learn something new due to job security, even though there are faster and compatible alternatives that exist today.
Rather use Redpanda, than continue to use Kafka and then complain about how resource intensive it is alongside zookeeper and all the circus the comes with it and make AWS smile as you're losing hundreds of thousands a month.
Very few people want to re-create their production AWS MSK cluster from scratch. And that's the only way currently. MSK can usually upgrade Kafka brokers with minor performance degradation but not for this particular change.
Kafka is used because the Java folks don't want to learn something new due to job security, even though there are faster and compatible alternatives that exist today.
Rather use Redpanda, than continue to use Kafka and then complain about how resource intensive it is alongside zookeeper and all the circus the comes with it and make AWS smile as you're losing hundreds of thousands a month.