Hacker News new | past | comments | ask | show | jobs | submit login

(I'm the PM for this at Cloudflare & the blog author)

We looked at a number of existing options — trying to strike a balance between languages we run at scale (tooling, ops, security), memory safety, and long-term maintainability as a growing company.

We ended up writing an MQTT broker from scratch in Rust, with our own Durable Objects[1] for consistency and co-ordination across our hundreds of edge locations (and tens of thousands of machines). The MQTT spec itself[2] is very approachable, which made this undertaking easier still.

One of the increasingly cool parts of Cloudflare is that we use more and more of what our customers use: our Broker service sits behind Spectrum[3], and we use Workers + DO for all of the metadata/session handling. The next set of new products will invariably start to build on Pub/Sub itself, too :-)

[1]: https://developers.cloudflare.com/workers/learning/using-dur... [2]: https://docs.oasis-open.org/mqtt/mqtt/v5.0/mqtt-v5.0.html [3]: https://developers.cloudflare.com/spectrum/




The MQTT spec itself[2] is very approachable, which made this undertaking easier still.

Indeed. I've found writing an MQTT client is a good tutorial project when learning a new language.


Some time ago I was at a talk about verifying different MQTT brokers so they fulfill the spec. They presented several inconsistencies they found (in OSS brokers) and submitted fixes for.

I looked up if there's some info online and found that paper (unfortunately paywalled): https://graz.pure.elsevier.com/en/publications/learning-base...


The F-Secure folks also have https://github.com/F-Secure/mqtt_fuzz - although it doesn't support authenticated brokers as of yet.


Thanks, make sense.

Actually, I asked because it surprised me that D1 - sqlite - was not written the same way.

[1] https://blog.cloudflare.com/introducing-d1/


D1 _does_ use Durable Objects under the hood! Like Pub/Sub, it's ultimately an implementation detail, albeit probably interesting to this audience.


>>> And, since we're building on the redundant storage of Durable Objects, your database can physically move locations as needed, resulting in self-healing from even the most catastrophic problems in seconds.

True. I missed that part !

Thanks.


Do you plan to opensource the implementation some time?


No concrete plans at this time, especially since there are many competing implementations.

If we open-sourced we’d need to do it right (proper governance, triage, support, etc) vs just putting into a public repo and calling it a day.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: