Hacker News new | past | comments | ask | show | jobs | submit login
Mosquitto: An open-source MQTT broker (github.com/eclipse)
272 points by ducktective 3 months ago | hide | past | favorite | 138 comments



I've learned a lot about MQTT while setting up Home Assistant, and Mosquitto is a pretty important part of my Home Assistant installation. I use it to control all of my Zigbee devices via "zigbee2mqtt" (buttons, switch modules, etc.), and also a smart kettle running Tasmota [1]. I also added a little patch to Tasmota [2] so that my kettle responds with `418 I'm a teapot` [3]

[1] https://tasmota.github.io/docs

[2] https://github.com/ndbroadbent/Tasmota/commit/bbcf57faffbf5b...

[3] https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/418


Prime use-case for HTTP 418!


People always want that for teapots. I have a device that uses it as an indication it’s internal data shouldn’t be trusted (some unknown error).

I have it respond with 418 as “I taste colors and the government spies on you with the ground in your house wiring”.

I like it for that more than actual tea pots which is far too on the nose.


What kind of device might have internal data that cannot be trusted?


Cool stuff and inspiring. Have a website or GitHub about your home automation projects? I checked out your blog, and the raspberry pi microwave was pretty sweet.


MQTT is becoming the defacto interface for home automation. As an example, Home Assistant used to have plugins for zwave and zigbee but they are now replaced by an MQTT broker that sits between Home Assistant and the program driving the wireless dongle.

Since I'm already running Mosquitto to interface with zwave devices, I decided to also use it to monitor my home lab. Servers push sensor statuses into Mosquitto and Home Assistant is used for the UI and alerting. That was a fun week-end project.


> Home Assistant used to have plugins for zwave and zigbee but they are now replaced by an MQTT broker that sits between Home Assistant and the program driving the wireless dongle

And now, they've moved on from that to having straight websocket communication between those two programs, eliminating MQTT altogether.

I use MQTT for a lot of stuff but it adds a layer of delay between devices that is particularly noticeable with flipping a light switch in the Home Assistant web ui.


This may be a use case issue. I have a zigbee door contact that communicates to HA through Z2MQTT which turns on the light when the door opens. It turns on so fast that my wife didn't believe the light wasn't already on until I made her open the door from the inside. It is instant for all intents and purposes. So this delay may be relevant to the setup, more than MQTT itself.


Are you sure it's MQTT adding the delay? From my experience that part is pretty much instantaneous, but processing though Home Assistant, and in my case zigbee2mqtt does add some delay.


No I'm not certain but I can certainly say that the whole setup is much more responsive after taking MQTT out of the loop.


That's interesting to hear, I would say that most of the latency comes from HA REST API call rather than MQTT (at least if MQTT is local to the zigbee2mqtt installation) but now I want to do some experiments in my setup, even if unfortunately HA native Zigbee HW support is way more limited than Z2M


Is that the MQTT part causing the problem, though? I have a bunch of Z-Wave devices on my network (using openHAB, which drives the Z-Wave hub dongle directly), and response times range from under a half second to several seconds, depending on phase of the moon. I expected that since Z-Wave is a mesh network, pushing commands out to devices that are farther away from the controller could take longer.


ZHA is still the preferred way for Zigbee with Home Assistant, isn't it?


Many people use https://www.zigbee2mqtt.io/ now too, it seems people are moving in this direction



I've been happy with the ZHA implementation and haven't tried z2mqtt. I am much happier with zwavejs than the old implementation, though.

I do run MQTT for a few wifi devices and communication between node red and custom client code on my laptop/desktop.


I started with ZHA but had problems with some devices (e.g. a 4-way switch panel that would only show as a single button.) I've had a much better experience with zigbee2mqtt, and it seems a bit more powerful


ZHA is great and fast, I prefer it instead of zigbee2mgtt. I have 100+ zigbee devices and 80 z-wave.


I observed this as well and was just starting to dabble in MQTT with my ESP projects.

I have yet to find a good resource for outlining autodiscovery. Most of the projects I have come across just hard code some pub paths in the homeassistant namespace.


The official docs[1] had everything I needed to know in order to make discovery work for my own hardware; essentially, you're just pushing a config stanza in JSON to a particular MQTT path. The only thing that's not immediately clear from the docs is that the discovery data should either be pushed in retain mode or resent every few minutes, and even though it's optional, you should include a unique_id so that the device can be tweaked in the web UI.

[1] https://www.home-assistant.io/docs/mqtt/discovery/


After being in the trenches on this type of thing: unique_id is officially optional, but HIGHLY suggested.

Personally, I like to use the MAC since it means code reuse, but whatever it is make sure it’s there and unique.


Thanks, that's incredibly helpful.

I don't know what it is about the hassio docs but I have a hard time groking them and quite often find myself looking elsewhere for clarification.


>MQTT is becoming the defacto interface for home automation.

The problem I have with MQTT is also a feature. The payload format isn’t defined, there is no request response structure, encryption of payload requires custom encapsulation, etc. You are on your own for everything.

This is made better in MQTT v5, but AWS still doesn’t support that.

If I was starting over today, I really like OSCORE and CoAP derivatives for their completeness.


This should have been my first comment, but there you go.

Hello, I'm Roger, the Mosquitto project lead. Thanks for the kind comments, I'm happy to answer questions!

I work for cedalo.com developing Mosquitto and other MQTT projects. We offer Mosquitto based support, please get in touch if that's of interest to you.


I just wanted to thank you for it. Over the years it has had a big impact on all types of projects, from industrial heavy ones to raspberry-based tinkering experiments :D


Thank you for your work!


I've recently transitioned my home "IOT" network (and servers) to a Wireguard-only network. It's wonderful. All the routing accessibility through firewalls or with dynamic IPs just disappeared. All services are exposed only on the wireguard interface, meaning security and connectivity are solved (to the 80/20 level) with just Wireguard.

That kind of removed the need for a static broker entirely since now every node has a static IP on my wireguard network and are accessible no matter what network they are actually on. Pinging my mobile phone from an air quality sensor was an eye-opening experience.


Did you ever notice any issues with significant increased softirqs?

I started going down a similar path but a node that is acting as a gateway/bridge for UDP traffic routing to an outbound wireguard tunnel is now spending a constant ~20% CPU time on NET_RX softirqs with 10Mbps and 5~30Kpps. Almost no NET_TX.

It's specifically the combination of UDP-over-wg that seems to cause this. It's otherwise handling 1Gbps and way more packets just fine.

(I'm aware MQTT is TCP but hey, worth a shot)


what kind of IOT devices are we talking about? linux embedded or is wireguard available for something like nodemcu?


Linux embedded. (Pis mostly)


Hey, tell us more, please :)

I'v hoped to find in your profile maybe more info/blog, but instead: > Group supervisor at NASA Jet Propulsion Laboratory working in autonomous vehicles and sensing for space exploration.

Oh well, a little out of scope and I see you blog about other things. Anyway - you mean I don't need static IP to form a network of distant IoT devices? This would be some really useful property.


> Group supervisor at NASA Jet Propulsion Laboratory working in autonomous vehicles and sensing for space exploration.

Yes, at JPL they let engineers cycle in and out of management. (Serving institution / projects in phases)

> Anyway - you mean I don't need static IP to form a network of distant IoT devices? This would be some really useful property.

Yes, wireguard will "keep alive" a connection to peers by sending udp packets. This fools stateful sufficiently to allow traffic to migrate across networks easily. Kind of how push notifications magically follow you.

On top of those connections, wireguard establishes its own IP addresses for each peer. If you set up wireguard, senging a packet over IP that is one of your wireguard peers will get tunnelled through UDP to the peer at its location and real IP. Thus, you know the node by the WG IP, but it gets delivered seamlessly via whatever means necessary.

VPNs have done this forever. Wireguard makes it trivial and portable and fast.


I think every upvote on your comment is a request for a blog post or runbook. I would certainly be interested in that.


I'll just pile on my own accolades. We use MQTT to connect handhelds with in the field ag irrigation controllers. There's probably about 10,000 valves on farms and other ag projects throughout, that are ultimately scheduled and monitored through MQTT. We've even been able to build a full request/event protocol on top of it.

The protocol is simple enough, that when I wasn't happy with not many full solutions existing in iOS, I just rolled our own in a day or two.

Currently adding an Elixir REST bridge to the whole thing. It's been pretty cool all around.


That sounds really nice, I love interesting projects in areas outside of my experience. If you're using Mosquitto I'd love to hear more about it - my contact details are at the bottom of https://github.com/eclipse/mosquitto/ :)


I started home automation in my late 40-ies, as an amateur developer.

Despite having casually developed for 25 years at that point, I found it amazing how much I learned quickly when setting up Home Assistant and pyscript (and obviously mqtt).

This drove me to use a bus (mqtt for instance) for non-home automation code just because it is so useful.

To friends who wonder how to help their kids get interested in coding (which is not easy at the beginning because converting farenheit to celsius gets boring after some time), I tell them to buy a few zigbee devices and have them automate stuff in pyscript. I am happy to see that some are on the way to graduate in CS :)


I've never used Mosquitto, but I have done a fair amount with Verne.mq [1], and I have to say that MQTT is downright pleasant to use in a lot of cases. I've not done a ton with it in "real world" situations, but I have used it for multiple hackathons, and I'm always impressed how little of a headache it is to build a decent "live" application with almost no effort.

If your frontend web project calls for any kind of messaging, I definitely recommend looking into trying MQTT before you jump straight into WebSockets. There's a good chance MQTT does what you need, scales better, can communicate over WebSockets, and will make your life easier .

[1] https://vernemq.com/


I have used ZMQ for this with great success. The one-line python calls to send data between nodes are really nice.


ZeroMQ is fantastic. I've been using at work for a few years now, but I generally enjoy the tech anyway, their documentation[0] and "The Guide"[1] is also really nice, with examples for all of the different language bindings.

From C or Golang (or a bunch of other language bindings) you can set up some nice pub/sub or req/res with fan in/fan out and all sorts of cool routing with low-level, high speed sockets with the added power that ZMQ provides, such as the queueing, high-watermarks etc. You can also enable encryption easily with CurveZMQ which is a refreshing change from TLS.

[0]https://zeromq.org/get-started/

[1]https://zguide.zeromq.org/


I almost feel like the guide should be mandatory reading for anyone learning network programming. It gives extremely digestible summaries of most of the network patterns, when and why you'd use them, and then how they're done in ZeroMQ. Even if you end the guide thinking that ZeroMQ is stupid (which you probably wouldn't), you'd still probably benefit from having read the guide.


I love ZeroMQ, but I didn't think that had any first-class support for communicating in the browser (could be wrong).

I agree it's super nice though, even with a single process.


I learned about them on my previous job. It's rather easy to cluster and the Web-Hooks are super easy to get started with. I don't know Erlang myself, but looking at the code and comparing it to some of the Java solutions, it seemed well structured and easier to understand/modify if I wanted to. The only "Issue" I have with it right now, is the "custom OpenSource license" they have.


The VerneMQ bugtracker scares me. Especially this one: https://github.com/vernemq/vernemq/issues/1663. I'm running one instance but I'm on the lookout for an alternative that can more reliably save messages if a subscription client goes down.


I wish I could say that I chose Verne.mq because I weighed all the options and it was the best for my project but...I really chose it because it was written in Erlang and I like Erlang.

I won't pretend to have done an audit of the best servers in terms of bugs or features.


> I definitely recommend looking into trying MQTT before you jump straight into WebSockets.

What is a nice way to use mqtt in the front-end instead of websockets? What can you recommend?


Eclipse Paho JavaScript Client[0] for the front-end and personally I prefer RabbitMQ Web MQTT Plugin[1] for the back-end[1].

Take note Paho MQTT Client is available for several programming languages(Py,Rust,Go,C,C#,etc)[2].

[0]: https://www.eclipse.org/paho/index.php?page=clients/js/index... [1]: https://www.rabbitmq.com/web-mqtt.html [2]: https://www.eclipse.org/paho/index.php?page=downloads.php


Beat me to it!

I was about to suggest the Eclipse Paho library. One of the very few things in the developer world that has "just worked" for me the first time I tried it.


Yeah, it's pretty cool. I am tasked with developing a new API and communication method for a bunch of networked devices that run embedded Linux. Initially I focused on MQTT, but have since decided to define an API in OpenAPI and implement an HTTP/REST-style interface because that's well understood and easily supported by third parties, whose APIs we might want to integrate with or provide a translation layer for.

But the devices need to alert a controller application to status changes... which so far I supposed they would do by POSTing REST-style messages back to that application. Do people combine REST & MQTT?


> Do people combine REST & MQTT?

I'm not a REST+MQTT expert, but some people do combine them, why not?

One of the project I was working on (IoT, smart home), the mobile app received the current status from the REST API, then subscribed to changes via MQTT. Having MQTT was great for live updates on the mobile app, and for communicating with the IoT devices. HTTP was great for integrations with Google Home, Alexa, and we could test it easily as there are many backend frameworks focusing on CRUD REST HTTP backend apps.

Of course, if you actually have a problem you want to solve, be sure the to do your own research, there are more than one way to skin a cat, and there are so many services and platforms that could be "good enough" for your use case.


Generally people choose MQTT because it is lightweight and very well tested. I personally use it because its light one source, scalable, and very flexible. Have also chosen to use CoAP rather than REST for ad hoc data requests because for constrained devices it is much lighter on bandwidth, simpler to use than typical REST over TCP+HTTP with which most people are familiar.

In IOT its hard to go past MQTT for the usual signals paired with CoAP for ad hoc. When I want to reduce dependencies or just being lazy even CoAP gets replaced with building a request/respond mechanism using a MQTT queue so a hub pubs a message to the devices command queue and device reads that queue and responds to it. How often device checks the command queue depends on whether or not device can run command queue monitoring asynchronously or must do a scheduled round-robin of the queues.

The Eclipse Paho libs have been great for me using across different OS and bare metal, Arduino, FreeRTOS.


One thing to note is that MQTT is session-oriented, so you have to send periodic pings to the server to keep your session active. Which is obviously a deviation from stateless REST.


The REST paper explicitly mentions that REST and HTTP only try to solve the common cases, and solve those well. If something is not covered well then you’re explicitly advised to leave it to another protocol. You could for example have a GET response that (due to HATEOAS) suggests a websocket url including a topic to subscribe to.


For someone that has worked with RabbitMQ, is there a guide to the differences and appropriate usage of MQTT brokers with some real use cases explained?


You are aware RabbitMQ support MQTT[0] and MQTT over WebSockets[1]?

As for comparing it to RabbitMQ you should compare the AMQP protocol vs MQTT protocol, it seems MQTT with its lightweight design "won" IoT market over and should be rather compared against protocols like CoAP.

As for guides I have always found resources rather scarce and mostly learned by adding features to mqtt libraries or writing my own prototype games using it.

[0]: https://www.rabbitmq.com/mqtt.html [1]: https://www.rabbitmq.com/web-mqtt.html


Unfortunately, Rabbit's implementation of MQTT has some pretty serious deficiencies which you may find ends up being a deal-braker (e.g. https://github.com/rabbitmq/rabbitmq-server/issues/2556).


We run a hosted version starting at $5/month [0]. We used to have a free version but it was too popular.

[0] https://www.cloudmqtt.com/plans.html


We use this for work and it's a pretty slick product. Please add support for MQTT 5.0, or shared subscriptions!


In case someone's looking for a free MQTT service with the only constraints being client connection limits and bandwidth, check out HiveMQ Cloud: https://hivemq.cloud


We use cloudmqtt at work and we've been really impressed by the price and quality of the product. You made it not worth hosting our own instance!


The free version was a great resource while developing MQTT clients a few years ago, sorry it was too expensive.


If you are a library developer/maintainer we'd be happy to sponsor you with free subscriptions, contact us at support@cloudmqtt.com.


The node-red [0] project is a great complement to MQTT use-cases. [1]

"Node-RED is a programming tool for wiring together hardware devices, APIs and online services in new and interesting ways. It provides a browser-based editor that makes it easy to wire together flows using the wide range of nodes in the palette that can be deployed to its runtime in a single-click."

[0]: https://nodered.org/

[1]: https://cookbook.nodered.org/mqtt/connect-to-broker


Node-RED is one of those polarizing apps. People either love it or hate it with a passion.

I am in the latter camp. I mean, it does what it says on the tin, but I could never understand how laboriously "wiring" up a bunch of boxes and filling in parameters is any easier than whipping up a few lines of Python.


Well, if you are not a code based developer, eg. you are used to program industrial control systems, they are often developed the same way, as with certified/proofed tools, you can guarantee that the code has no uncaught error paths or simple syntax errors.

It also has a lot appeal to people who do not develop on a daily basis, eg. I'm pretty confident I could teach my dad to use nodered vs writing python in a day.

I myself just use it (after years of being on the second side) because I noticed, that it was somehow way faster to get stuff working and keep it maintained


Isn't it because the boxes have in-place documentation and the wires visualize the workflow? Making it easy & fast for the occasional user and let them avoid the pitfalls of unmastered syntax.


I'm in your camp too, but would it be fair to say that Node-RED is not designed for us?


I'm a big fan of NATS. It a small binary which can be scaled as much as you want and it replaces Kalfka, lightweight MQTT brokers, etc and has clients for most languages.


https://nats.io/ https://github.com/nats-io/nats-server

Is it suitable for embedded applications? (very low resources, intermittent network availability, etc)


The NATS protocol [0] looks pretty simple, but is text based unlike MQTT which uses a size optimized binary protocol.

[0]: https://docs.nats.io/nats-protocol/nats-protocol


NATS actually also has support for MQTT[0]. If you already have a deployment for queues, microservices, etc, using the NATS mqtt endpoint seems like a very good option.

[0] https://docs.nats.io/nats-server/configuration/mqtt


Anyone have experience scaling with more than 256 devices?

I use MQTTS at work and for my own projects. It is a very thin protocol (low code footprint), especially combined with Wi-Fi where those stacks typically eat up most of your MCU's flash if you're using a single-chip solution with TLS. I've used it on a max of 256 devices on a single BSSID, but I would be curious to know how it scales.

EDIT: Specifically, when all devices are subbed to a noisy channel. In some cases we had to stagger pub with random jitter to prevent both Wi-Fi and edge processing issues.


All Dyson smart products use MQTT. As I recall it works fine from a performance point of view (it's a very lightweight protocol), but personally I thought it was a pretty flawed choice because the problem of choosing/discovering paths is just completely ignored, and it shares the same problem as all event bus style systems - events can get lost, it's very difficult to debug and learn, etc.


> events can get lost,

Well, yes, but QoS=1 or 2 with a reasonably large queue I haven't seen loss issues.

What kind of loss problems are you running into (that I should be worried about)?


Yeah I don't mean due to the protocol, I mean due to application bugs where something adds a message to some queue and the thing that is meant to process it doesn't for some reason, and then nothing will ever realise.

Compared to callbacks / RPC for example, where you have a function call the either has to succeed or it has to fail.

It's not a deal breaker but I don't like it.


If your MQTT client allows control over this aspect of the protocol, you can just not ACK QOS > 0 messages that failed to process. Next time you connect you'll get the non-ACK'ed messages first.


You can use a subset of MQTT for ingress data and store it in Kafka for instance. Check out https://www.drogue.io and https://sandbox.drogue.cloud for an open source project that does that


I can't comment on MCUs or WiFi BSSIDs, but on the server side I have an Intel Atom N2800 (think low performance) with nearly 4000 clients sending out ~50k messages/s with an average payload size of 80 bytes. The overwhelming majority of those are not using TLS. About a quarter using MQTT over websockets.


I’ve only seen papers from the automobile industry where they used MQTT but fed it into Kafka to scale.


Perhaps someone here that's familiar with MQTT (and variants) can shed light on how to handle this case: an IoT device that periodically comes to life, queries the broker for the last known state of topic 'xyz', then sends a message regarding another topic 'abc' and goes back to sleep. i.e. it's not awake to receive updates to the 'xyz' topic, but needs to know the last update when it awakes.


With mqtt you may do that with retained messages. The emitter of a message mark the message as to be retained. The server will retain it. Now when your mqtt client will subscribe tout the topic, the server will send the last retained message for this topic.


If you need a highly available broker you can consider https://github.com/streamnative/mop which adds a MQTT protocol handler to Apache Pulsar.

I recently contributed auth support and TLS support also just landed. There is perhaps a few non-standards compliant things that need to be worked out but it's in fairly good shape.


I really like Bevywise MQTTRoute[1] as it is good one for home automation solutions. It allows us to create our dashboard[2] specific to our use case with pre-built widgets. The easy & flexible integration with home assistant[3] and the mobile app[4] permits us to automate our home devices.

[1] https://www.bevywise.com/mqtt-broker/

[2] https://www.bevywise.com/mqtt-dashboard/

[3] https://www.bevywise.com/blog/mqtt-route-integration-with-ho...

[4] https://play.google.com/store/apps/details?id=com.Bevywise.m...


I like MQTT and we use Mosquitto in our product but I always wonder if MQTT pushes TCP into areas where UDP would be a better fit.


Like your other reply said, MQTT-SN can use UDP so may be a better fit in some situations. It's a shame that MQTT-SN hasn't seen wider use, but there isn't that much support at the moment. The protocol is being standardised at the moment, and there has been an MQTT to MQTT-SN gateway as part of the Eclipse Paho project for a while now. The Mosquitto project has the aim of adding support for MQTT-SN - it's just not as high a priority as other things at the moment unfortunately though. If it's something that's important to you, please get in touch.

Roger


That is where MQTT-SN comes in. I have used it over NB-IOT and then a gateway to full MQTT over mTLS to AWS IoT.


MQTT/Mosquitto relays is a great solution for passing data across network boundaries when inbound is limited.

I use it with Webhooks deployed to Internet VM that pass the data to broker installed on VM.

Another broker in home network connects to Internet broker and relays messages to local clients.

I don’t have to open any ports into my home network for webhooks to work with other services.


MQTT is awesome. In deepstream.io we bridged the node MQTT protocol to deepstream which allows us to mix and match protocols. MQQT for IoT devices (and to be honest any device considering the large amount of libraries) and the deepstream JS client for node/browsers.

This pretty much allowed clustering out of the box + automatically saves messages with quality assurance set in an actual database. Plus a bit more authentication and permission tools.

https://deepstream.io/blog/20190909-release-deepstream-4.2/

Deepstream itself is under maintenance right now, but I find the concept of mixing protocols together (similar to what ably.io provides) to hit a sweet spot for development across multiple different environments/devices


I've been wanting a GCM/FCM alternative, could mqtt be used for notifications like that? I understand its usually used for IOT devices and stuff so I assume its suitable for not using loads of power


Thats what Facebook Messenger uses. Downside to this is that only FCM can work around android's power saving mechanisms and deliver instantly... unless you're Facebook and able to convince vendors to patch the firmware for your servers too ;)


Yep. Another real concrete example of an unwanted monopoly. We have issues with fcm, we are sending about 10m per day for millions of users. About 1 to 3% never arrives. Ends up in the void. Google doesn't want to help. We are not Facebook. But no alternative. A pity


97% is actually a really good delivery rate. From my experience working on several push-based products, ~80% is the industry standard.


I'm developing a mqtt client/broker and the protocol is all but simple to implement. I didn't found any implementation, mosquitto included, that was really lossless.


Is that true if you set QoS to a value other than 0 in your application?

https://www.hivemq.com/blog/mqtt-essentials-part-6-mqtt-qual...


No, even when you set the QoS to 2, "ExactlyOnce", you may still hit edge cases where you loose data, without knowing it, especially on enterprise networks, or if there is an high throughtput.


The protocol is simple (I've written my own implementation in php), what do you mean with lossless?


That when you use QoS 2 "ExactlyOnce" (or 1), you will never loose a message.


Paho-mqtt implements QoS and also has an option to store unsent messages to disk for later sending.


The issues I spotted are how the protocol is implemented, not how the packets are stored.


+1 on paho-mqtt for python and mosquitto. All of my fun projects such as XBee3 sensors and HA + Z-Wave go through MQTT. It has been rock solid and a joy to develop against.


It will work fine for your small scale IoT, but when they said "for IoT" they never meant it for high scale.


"Really lossless" really boils down to a client having handled the message completely, which is outside the remit of a transport protocol.


Yes, but no. If your protocol or your implementations have holes, you may loose messages because of that, and in a lot of applications, it's unacceptable to have the transport protocol dropping packets.


I've worked quite a bit with MQTT - can you tell us a little more about what you mean by "really lossless"?


To never loose a packet on QoS 1/2.


Could you expand on the difficulties you are facing? From the outside the protocol looks quite simple.


The Packet ID logic is not enough restricted.

For exemple, the emitter is not restricted in the ID it should use for it's packet, it can be any unsigned 16 bit number.

In contrast, a lot of protocol the ID must be incremental, and the receiver can detect that a package is missing (which result in smaller network overhead, the client can ack, or re-ask for a dropped packet, which result in a single roundtrip when there is no issue, instead of having the double round trip that mqtt have). It means that, the receiver can't know if it missed a packet. Because you don't have to waits for the acks to propagate the message, it mean the message delivering is unordoned. It also mean that if the same message keep dropping, the protocol should continue to work. Now if you implemented your mqtt with an i++, this where you can start to have dataloss, because now that you sent 65k messages you finished you start to need actually free packet id.

And it cause a lot more issues that what I said there, that I hope I covered every case in my implementation: https://github.com/signature-opensource/CK-MQTT/blob/develop...

Then, this is just the PacketID Logic. MQTT 3 doesn't do enough things, while MQTT5 do too much things. MQTT 3 Have design issues which MQTT 5 acknowledged it, and tried to fix it with duck tapes. One issue for exemple is the credentials being sent after the last will packet, which lead to a huge DOS issue, because now an attacker can attack you with your maximum last will packet, that you will want to save to disk, then wait to timeout without ever sending credentials. MQTT 5 is not really a breaking change as this behavior still exist, but you can enforce a credentials packet for the connexion.

All this highlight that MQTT could be a far smaller, faster, simpler protocol if correctly done (and a protocol like this probably exist, I didn't looked at a lot of popular protocols for now, only a dozen of hardware custom ones).

If I knew all the above before starting to implement my MQTT client, I would had want with another protocol.


I take it mqtt was developed for embedded systems, microcontrollers etc to be lightweight. Can the broker also run in such and environment, or is that just the clients?


I'm running Mosquitto as the central messaging broker on an embedded Linux product right now. It's super lightweight.

Running the broker on the target has been a really great thing during development. I can run validation systems on other machines and inject messages over the wire, or I can actually run the UI on my dev machine and let the backend respond from a live target.

I also have a development team in Asia that can't get to the newest hardware, so I fire up a system here and open a port they can access. They love it.

That said, the bandwidth in MQTT is lightweight but you still need a TCP/IP stack on your system. That's typically more substantial of a footprint than Mosquitto will be at the end of the day.


The lowest spec device I've come across Mosquitto being used is a 180MHz MIPS processor on 32MB RAM, running Linux.

Someone has recently taken the RSMB broker (from IBM originally) and ported it to the ESP32: https://github.com/DynamicDevices/picobroker I can't comment on what the limitations of that are though.


Mosquitto is C/C++ without any crazy dependencies so you should be able to do that. I don't immediately see why you would do this though? Any specific use cases?

I haven't done this myself yet but I've seen a couple of production IoT setups with a local MQTT broker running on a RaspberryPi configured to relay all messages to a central broker running in the cloud.


Pi is a far cry from "Microcontrollers".

MQTT requires a TCP stack, which limits to the higher end chips.

But, for embedded linux, you're probably golden.


MQTT is available via LWIP on STM32 parts that don't have an MMU.

I've tried it on the STM32F746 (Cortex-M7) and it works well, although there are other issues with networking on CubeMX that are open.


If you don't need it to scale crazily, yes. E.g. microcontrollers in nodes and then a local gateway as an embedded Linux system would not surprise me to see.


I really like Mosquitto, I'd recommend it for quickly getting started hacking together IoT projects on the hobby / side project scale. One of our community (QuestDB) contributors put together a docker compose example that I thought was pretty nice for quick prototyping, could be convenient for someone here:

https://github.com/shantanoo-desai/questitto


We recently used mosquitto/mqtt as the broker for a timekeeping system for a student engineering event at a track and was mirrored to the web/livestream for viewing pleasure of everyone else at home (since it was during the pandemic). It worked with minimal load and without problems even when the connection was shoddy due to extremely badly configured routers by vodafone (filling up nat tables is still a thing in 2021 apperantly).


This project is exceptionally useful not only in itself but as a tool you can use during development. Software that can be controlled entirely from the command line without configuration files (unless you choose to) are very convenient.


We’re using MQTT to have different sensors publish their measurements and different user groups can subscribe to the relevant ones. I think it’s a great way to separate stuff from each other or send meaningful data away together.


Unfortunately the websockets implementation only allows IPv6, and in my tests from a js app sometimes took seconds to start if it worked at all. I’m currently looking for an alternative broker due to those issues.


As others have pointed out, that's not the case. Something you do have to be careful of is the compile options for libwebsockets and what version you're using. Recent-ish version of libwebsockets in particular disable by default some features Mosquitto relies on, so some installations may suffer.

Version 2.1 of Mosquitto will no longer need libwebsockets (although it can still be selected at compile time) and so websockets support will be less dependant on how other libraries are compiled.


> Unfortunately the websockets implementation only allows IPv6

Huh? Pretty sure that's not true.


>socket_domain [ ipv4 | ipv6 ]

>By default, a listener will attempt to listen on all supported IP protocol versions.

>If you do not have an IPv4 or IPv6 interface you may wish to disable support for either of those protocol versions. In particular, note that due to the limitations of the websockets library, it will only ever attempt to open IPv6 sockets if IPv6 support is compiled in, and so will fail if IPv6 is not available.

From the manual. Huh, seems like I might’ve misunderstood this paragraph. But I was only ever able to access the broker if the browser was using an ipv6 address. I’ll have to test again I suppose. Still, the performance and reliability was not good.


Right, that just says if you don't have IPv6 support compiled in, you can't use IPv6. IPv4 is always available, and has worked wonderfully well in my experience.


Thanks for pointing that out. I’ll have to try again. Maybe ipv4 got disabled in the configuration somehow by accident.


I used Mosquitto years ago for a few years and never had any issues with it...


Use Mosquitto on Windows server at home to transfer data from my energy smart meter to home assistant. Pretty rock solid and easy to diagnose issues via mqtt tools.


Anyone have experience comparing MQTT with NATS, they seem to cover similar scenarios.


Would be looking for this as well. I love using NATS, would like to see a comparison of both.


is mosquitto clusterable nowadays? on its own or via k8s?

for: * fault tolerance * fail over * vertical scalability?

does anyone know this? if not is there a another opensource mqtt broker which does that?


VerneMQ [0] could be interesting for you, they claim that:

> VerneMQ is a high-performance, distributed MQTT broker. It scales horizontally and vertically on commodity hardware to support a high number of concurrent publishers and consumers while maintaining low latency and fault tolerance.

I'm not sure about their licensing at the moment [1], to me it's a bit confusing.

[0] https://vernemq.com/

[1] https://vernemq.com/blog/2019/11/26/vernemq-end-user-license...


Mosquitto is not, we use RabbitMQ which runs on Erlang/OTP and has mqtt plug-ins. Mosquitto is single threaded. It's fast for sure but doesn't scale out. They can interoperate though as Mosquitto has configurable bridging.


Recently wanted to also create a cluster of mqtt brokers and found this medium tutorial stringing Docker Swarm, RabbitMQ, Consul and HAProxy together very helpful[0].

I was not happy with the static ip's for haproxy and opted to register the mqtt/tls services in consul and use the consul dns resolver with hapraxy.

[0]: https://medium.com/hepsiburadatech/implementing-highly-avail...


I'm running a cluster of Aedes backed by redis.

https://github.com/moscajs/aedes

I was inspired by this post by Bolt

https://medium.com/bolt-labs/developing-software-for-bolt-sc...


I work for Cedalo AG developing Mosquitto. Cedalo will be offering a version of Mosquitto with failover support as a paid for feature soon.


There are broker implementations that support clustering. Emqx comes to mind.


There are lots of brokers and Eclipse Amlen is coming soon: https://www.eclipse.org/amlen/

(disclosure: I work on Amlen)


Have a look at Apache Artemis. Well architected multi-protocol broker with clustering.


I think it is time to use Kafka for home automation messages.





Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: