Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Does it support Protobuf deserialization via Schema Registry? This is basically where every other tool falls apart. Kafka UI just added support very recently but kcat falls apart.


Hi! it validates if the message was (or not) generated for schema registry, it checks if the message has the schema registry magic bytes (this bytes has the schema id). so, it deserializes messages with or without schema registry.

This is the drawback: you have to generate the descriptor first.

1) download the schema from schema registry: http :8081/schemas/ids/<my-id>/schema

2) generate the descriptor with the schema: protoc --include_imports --descriptor_set_out=my-descriptor.desc --proto_path=. my-schema.proto

3) use kaskade: kaskade consumer -b my-kafka:9092 -x auto.offset.reset=earliest -k string -v protobuf -t my-protobuf-topic -p descriptor=my-descriptor.desc -p value=mypackage.MyMessage


Perfect! I had tried this last night before your comment (I saw these instructions in your README) but I was trying to include `-s url=` and `-s basic.auth...` which appears to force you into using `-k avro` or `-v avro`. But, once I removed the Schema Registry config options, I finally got the TUI showing up and was able to read messages! Thank you very much. Nearly all of my topics are Pb-encoded so I'll have to throw something together to get all the descriptors and perhaps abstract a bit. I love seeing the different uses of Textual coming out and this is no different; this is a big improvement from kcat IMO.


Hey ddoolin, It's so nice to read this, yes! kaskade ignores the schema registry metadata for json and protobuf messages generated for schema registry, you do not need to provide url (only in case of using avro).

The only catch (in the protobuf case) is to generate the descriptors with protoc, but the good thing is generally we all have them, so maybe is not a big problem.


It'd be cool if you supported the Buf Schema Registry's Reflection API: https://buf.build/docs/bsr/reflection/overview

Then, the descriptors could be delivered on demand!


Working on it


about the first step: if you already have the .proto file you do not need to download it from schema registry




Consider applying for YC's Fall 2025 batch! Applications are open till Aug 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: