Great stuff! I'm starting to work a lot on the satellite space (https://www.sourcemeta.com), building a binary serialization format around JSON called JSON BinPack (https://jsonbinpack.sourcemeta.com) that is extremely space-efficient to pack more documents in the same Iridium uplink/downlink operation (up to 74% more compact than Protocol Buffers. See reproducible benchmark here: https://arxiv.org/abs/2211.12799).
It is still a heavy work in progress, but if anybody here is suffering from expensive Iridium bills, I would love to connect and discuss to make sure JSON BinPack is built the right way!
Hey Juan, this looks promising - using a pre-shared schema would allow you to reduce payload sizes in the same way using pre-shared dictionaries makes compression more efficient (such as with zstd or brotli). Having worked on exactly this space in the past (sending and receiving messages from a microcontroller that eventually go through an Iridium SBD modem), some feedback:
After spending a few minutes poking around your landing page, I'm not sure how JSON BinPack works. I see a JSON Schema, a JSON document and then a payload, but I'd be interested in the low level details of how I get from a schema to a payload. When I'm on a microcontroller, I'm going to care quite a bit about the code that has to run to make that, and also the code that's receiving it. Is this something I could hand-jam on the micro, and then decode with a nice schema on the receiving side? Understanding the path from code and data to serialized payload would be important to me.
One thing that was nice about using ProtocolBuffers for this was that I had libraries at hand to encode/decode messages in several different languages (which were used) - what is the roadmap or current language support?
I can understand how ProtocolBuffers handles schema evolution, but I'm still not sure how JSON Schema evolution would work with JSON BinPack. An example would move mountains.
Finally, if I were digging into this and found it had switched from Apache to AGPL, and required a commercial license, it would be a hard sell vs all the permissively licensed alternatives. At the end of the day, Iridium SBD messages are like 270 bytes and even hand rolling a binary format is pretty manageable. I think I could swing budget for support, consulting, maintenance or some other service. But if I were evaluating this back when I needed a serialization format, and ran into AGPL, I would bounce almost immediately.
Thanks for the very valuable and honest feedback. All really good points, mainly how lacking the website is in terms of documentation at the moment. Making notes and plan to address all of it!
It is still a heavy work in progress, but if anybody here is suffering from expensive Iridium bills, I would love to connect and discuss to make sure JSON BinPack is built the right way!