13 ms·

MQTT turns 25

289 points by andypiper 2y ago

BrandoElFollito 2y ago
I discovered MQTT with Home Assistant (a home automation solution, no, sorry, THE home automation solution). It helped me to understand pubsub philosophy and I ended up using it for my personal development - to the point where I was misusing it (which is the typical path with technologies one discovers) It is a great protocol for home automation but quickly gets annoying for more complex scenarios.
dloranc 2y ago
What do you think about LwM2M in comparison with MQTT?
- vv_ 2y ago
  It is not directly comparable because LwM2M was originally developed for device management, while MQTT is primarily used for transferring telemetry or other opaque data. My personal gripe with LwM2M is that it uses CoAP (Constrained Application Protocol) as its primary transport protocol. Since CoAP is based on UDP, you’re forced to rely on DTLS for authentication and encryption. Managing sessions with DTLS can be quite frustrating, and unlike TCP, there’s no reliable indication of when a session is closed by your peer. Another issue is that it doesn't strictly follow the client-initiated communication pattern and can receive unsolicited messages from the server. I have found this to be very unreliable in real-world conditions for Device Management use cases.
Uptrenda 2y ago
The cool thing about MQTT is there's so many servers for it that are completely public -- and intentionally so. You can use it for real time messages but I think it also has quality modes that allow for some persistence. Some of these servers have stat pages that show how much data they transfer. Usually phenomenal amounts of data. It's pleasing to me as someone doing a lot of stuff with open source networking to have such an establish protocol (instead of say -- 10 crappy ones.) MQTT works and it works well.
- roland35 2y ago
  Have any examples of public servers?
  - Scarjit 2y ago
    HiveMQ has a public one: https://broker.hivemq.com/ https://broker.hivemq.com/
  - dikei 2y ago
    I only know this one https://www.mqtt-dashboard.com/ https://www.mqtt-dashboard.com/ But a quick Google search will get you some others.
  - Risse 2y ago
    Helsinki Regional Transport (HSL) has a public server with realtime position of all of their vehicles (buses, trains). https://digitransit.fi/en/developers/apis/4-realtime-api/vehicle-positions/high-frequency-positioning/ https://digitransit.fi/en/developers/apis/4-realtime-api/veh...
bouvin 2y ago
I have been using MQTT for years in my IoT courses – it has proven to be a very versatile tool, and it is convenient that it is also supported over web sockets.
kitd 2y ago
About 15 years ago, Andy Stanford Clark's house became something of a news item, in the days before tweeting IoT devices were a thing: https://www.bbc.co.uk/blogs/technology/2009/06/things_that_tweet.html https://www.bbc.co.uk/blogs/technology/2009/06/things_that_t... The core protocol, devised in the days when 1 byte over a satellite link cost $1, is incredibly efficient and simple to implement.
baruchthescribe 2y ago
Reposting my comment from five years ago: I once had a client where the only port available for me to use on their firewalls was for MQTT (1883) because that's how we were getting sensor data from them. They would not open anything else for us no matter how we implored them so I wrote a live TCP wrapper over MQTT to get around it. It was a local multithreaded TCP daemon that listened for outbound requests on a certain port, wrapped them in MQTT and then published them using a unique topic. The server daemon would detect these topics and unwrap them before forwarding to our server processes. So the client machine thought it was making a live TCP connection to our server but in the middle was a funky invisible MQTT wrapper. It was really elegant once it worked but my goodness was it a pain to debug - a couple of months before I got the whole thing right because of all the blind alleys I went down.
- ericvsmith 2y ago
  I’ve worked at places where subverting firewall rules like this would get you fired. I now refer to such situations where I’m forced to twiddle my thumbs while no work gets done as “letting the process work”.
  - exe34 2y ago
    I always kick this kind of decision upwards: we can do it this way and break this rule, so I need director level approval, or we can wait until the customer blinks.
    - n_plus_1_acc 2y ago
      Yes. Ask your boss for approval and tell them otherwise nothing will get done.
- ocschwar 2y ago
  So you basically implemented an MQTT-Sockets protocol. Which you could then use to connect to a WebSockets server on the other side.
  - 2y ago
joezydeco 2y ago
I had a lot of fun on an embedded systems project recently where I used MQTT as an interprocess messaging system. Broker and clients on the same machine. If I needed to sniff or debug something it was as easy as putting the device on the network and using MQTT Explorer to record or inject messages. I could even let coworkers that were working remotely work with the system by opening the port outside the LAN.
- klysm 2y ago
  Haven’t seen this done much but it does seem to have some desirable properties. My biggest concern when using it as a system component has been durability guarantees. I don’t have a lot of trust on broker implementations not to lose data.
  - logifail 2y ago
    > My biggest concern when using it as a system component has been durability guarantees. I don’t have a lot of trust on broker implementations not to lose data (If we ignore retained messages) MQTT brokers are not designed to store data, so there's no need to be concerned about durability because there simply isn't any...(!) It is (just) a messaging protocol.
  - joezydeco 2y ago
    MQTT, by design, isn't going to have the confirmation mechanisms and safeguards that you could get from other methods or even sockets. But once you release yourself from that paranoia, it's pretty liberating. The nature of my project was that a missed message wouldn't be fatal, and message retention was good enough protection for a process that crashed and restarted.
- roland35 2y ago
  Would zeromq be an alternative here?
  - joezydeco 2y ago
    It's an alternative, but there are some aspects to having the broker that I liked. If there was a bug I could halt the system, attach/subscribe an external dev machine, and grab all retained messages from the broker. I don't believe ZMQ allows that.
miljen 2y ago
Fun fact: at the same time, the most famous C++ library, Boost, is reviewing the `async-mqtt5` implementation (https://github.com/mireo/async-mqtt5 https://github.com/mireo/async-mqtt5) to be included in Boost as Boost.MQTT: https://lists.boost.org/Archives/boost/2024/10/index.php https://lists.boost.org/Archives/boost/2024/10/index.php
- formerly_proven 2y ago
  Is Boost still something people reach for in newer projects? Anecdotally most adoption I've seen happened in the 00s and very early 2010s (pre everyone mandating C++0x/C++11), I only rarely see it around these days. edit: boost.org is a blast from the past, still looks exactly like I remember it from like 2008. Down to the "Get Boost" shopped on an emergency off button!
  - miljen 2y ago
    That is an interesting question. Here you can see the actual popularity of each library within Boost: https://grafikrobot.github.io/boost_lib_stats/ https://grafikrobot.github.io/boost_lib_stats/. Also, see the comments at https://www.reddit.com/r/cpp/comments/130bzj8/has_boost_lost_its_charm/ https://www.reddit.com/r/cpp/comments/130bzj8/has_boost_lost.... My company has been heavily utilizing Boost.Asio and Boost.Beast, so we've definitely reached for Boost in newer projects as well.
  - spacechild1 2y ago
    AFAICT Boost.Asio is still the go-to networking library. (Though Asio is also available as a standalone outside of Boost, I didn't know that until recently.) Apart from that, Boost has quite a few goodies such as Boost.Json, Boost.Program_options, Boost.Interprocess, Boost.Lockfree, Boost.Unordered, Boost.Dynamic_Bitset, etc. Some libraries, like Boost.Atomic, Boost.Thread, Boost.Chrono and Boost.Filesystem, simply became obsolete with modern C++ versions. In fact, they served as the blueprint for the corresponding C++ standard libraries. Personally, I have been wary of using Boost in the past because it's such huge library, but CMake integration is actually quite good these days and I found it pretty easy to use. Documentation is also quite good IMO.
this_user 2y ago
Having recently used MQTT for a project, I can't say that I'm a huge fan. There are a lot of options in the protocol where it's not immediately clear what they do and why they are important, or in what combination they need to be used to make sure things work as intended, and the documentation is often not great at explaining. Part of this may also be on the Eclipse Mosquitto Python client that I had been using. I took me several days to figure out that the client was running into a race condition on a slower system that resulted in subscriptions to topics silently being ignored which caused the associated callbacks to malfunction despite following their documentation 100%. Overall, this has been one the messiest experienced I have ever had with a protocol, certainly one that isn't that old.
- jorl17 2y ago
  My experience using MQTT has been through the paho python MQTT library, and, while we definitely have managed to get a lot done, it has been a terrible experience. Everything about it rubs me the wrong way. The API design, the poor documentation, even the way they seem to not adhere to typical python conventions. It looks like it's easy to start using it, but then the breadth of protocols and implementations start getting to you. At a point in time, I remember the only reliable way we had of knowing if we had successfully connected to a server was to try to subscribe to a topic twice and catch a specific error code on the on_connect message (which actually was documented as a success code at the time). I know how crazy it sounds and maybe there was a better way to do it, but, if there was, I don't think it was that easy to find. It's easy to complain, though. I'm very grateful so many people have worked to create this library. If they hadn't, I wouldn't have built what we've built with it. I really admire people who take on these huge projects.
- DannyBee 2y ago
  So your client just sucks here. My experience with the eclipse clients (paho, etc) is similar to yours in both python and C++, - it makes it overly complicated and seems very low level. It also is somewhat buggy because of the architecture. I believe (perhaps wrongly), it's all basically maintained by one person or close to it (only one person has contributed to the C++ client in the past 6 months, for example, and nobody has contributed in the past 3), so i kind of understand how it's gotten this way over time. I filed a simple PR (1 word change) to fix an obvious bug and it took 2 years to review it and accept it with no changes. Again not a complaint, just trying to portray how the state feels - a lot of libraries, bugs, and work, and few overworked people helping get it all done. I moved to other clients and the experience is much much better in python, rust, C#, and C++. Most of them have a good combination of high and low level API's, so if you just want to send a message on a topic, you don't have to worry about acks, retries, etc. But if you need control, you can get it. Honestly, i worry at this point that keeping paho/etc alive in this state is doing more harm than good - if they were officially dead it would at least force the issue. Right now you end up with users who have an experience like yours, and then either give up on MQTT or assume they are doing it wrong :)
vv_ 2y ago
There aren’t many telemetry transfer protocols suitable for IoT besides MQTT (especially for resource constrained systems). It is incredibly simple to implement, has numerous libraries available in various programming languages, is supported by major cloud providers and is designed to operate over TCP (meaning you avoid DTLS). We have shipped many products that use MQTT for telemetry transfer to various different vendor servers and it always just worked. It was never a pain-point in stark contrast to CoAP or some proprietary protocols.
8fingerlouie 2y ago
It's such a great little protocol, and besides being "small enough" for hobby projects, it also scales enough to be used for something like Facebook Messenger [1] [1]: https://engineering.fb.com/2011/08/12/android/building-facebook-messenger/ https://engineering.fb.com/2011/08/12/android/building-faceb...
ErneX 2y ago
Most of my home automations depend on it, it’s been great honestly. Cheers to mqtt!
ronameels 2y ago
MQTT is being used a lot more in recent years inside of factories for sharing data between machines. Historically it's been used in Oil & Gas for SCADA (getting data from remote well sites). 10+ years ago we added it to Kepware (OPC server) and streamed tag values to "the cloud". I was at a conference giving a presentation on it when Arlen Nipper, one of the creators of MQTT, came up after the presentation and said I did a "decent job". It was humbling :). Fast forward to today, and we have a new company (HighByte) modeling factory data at the edge and sending it via MQTT, SparkplugB (protocol over MQTT), direct to S3, Azure Blob, etc, etc. All that to say, MQTT is a big driver in Industry 4.0, and it's cool to see it so heavily used all these years later.
- fartfeatures 2y ago
  Is MQTT being used in places where previously Modbus would have been used?
  - ronameels 2y ago
    In some cases yes, controllers are adding in MQTT support directly or SparkplugB. It works in simple cases, but the lack of transactions & ability to control what and how often you're reading from the client side, can create challenges.
  - InDubioProRubio 2y ago
    There is also MQTT integrating bus-hardware- you basically plug in as bus-hardware replacement which allows to send data directly from the bus to the broker. The PLC never know what hit it. https://www.ifm.com/de/de/product/AL1332 https://www.ifm.com/de/de/product/AL1332 Pretty wild times. PS: You can web GET/POST set and get ios with these. Robot experience that tragic irony POST {NOOO} and he just dropped it like its hot.
- eska 2y ago
  I also work in that industry and am quite puzzled by the technological inferiority, scope creep and NIH syndrome of the OPC foundation’s standards. I wish they just used sparkplug b and implemented their specs on top of that for semantics. Then again the async stuff they’re doing now is so over-engineered and terrible as well.. I took part in their teams meetings some time, only to find they had never read the mqtt spec (less than 50 easy to read pages) and didn’t understand what the headers are for for example (they wanted to put stuff in there that actually belongs in the payload). Some microsoft guy also took offense at my suggestion to first look at what competitors are doing with mqtt, because he wanted to create something new rather than copy. My company will just treat opc ua at the very edge and isolate it from our tech as much as possible on my suggestion.
NortySpock 2y ago
My most recent foray into MQTT was using the MQTT 3.1.1 compatibility mode in NATS, which was pretty easy to set up once I got the permissions sorted out. Works great for my smart home needs while also letting me explore latency and throughput load testing if I want to have a little fun.
- artooro 2y ago
  I'm using NATS as well, natively when possible but bridged to MQTT for devices that need it.
gorbypark 2y ago
I think my first project that's still "in production" and used daily was taking an existing SVG map of a water distribution system (snowmaking and fire suppression pipelines/pumps/valves/etc at a major ski resort). I made a MQTT topic for each pump, valve and section of pipe along with some state for each (water direction, pump or valve on/off, pressures, etc). It's a website, so I used mqtt.js and jQuery to update the colours and fills of the SVG to represent the various states. It's statically hosted and there's a mqtt broker that's been running in a container, untouched for nearly 10 years now. Mqtt.js is running over a web socket, so if the state is updated, it just magically updates for everyone else.
- remram 2y ago
  Holy shit Docker is 11 years old.
  - deleted 2y ago
    [deleted]
  - gorbypark 2y ago
    It was early days, yeah. The other part of the project that's been depreciated now was running node 0.4, if I am not mistaken. The watermap was part of a bigger asset tracking system, node + jade templating (before jade became pug, even). The watermap was actually embedded in a jade template, but when I depreciated the other system, I ended up just copy and pasting the output from the jade file into an index.html, and everything just kept working :D
- brian-gilmore 2y ago
  aaah I love me some AJAM ;)
InDubioProRubio 2y ago
A heap of dead "replacements" at the feets - what more can a communication standard want.
sunshine-o 2y ago
I never understood why we don't use MQTT for simple systems monitoring, with a push model. I mean it was almost designed for this use case. Also if using NATS things become very simple to setup.
Evidlo 2y ago
I remember one of the annoying things I found about actually using MQTT was that many clients didn't implement any sort of connect/disconnect callback and expected you to handle reconnection logic (e.g. exponential backoff) yourself. Also it didn't support RPC style requests for some reason. Also since MQTT is TCP-based you can't natively speak the protocol from a webbrowser without the broker also listening on a websocket. Ended up switching to WAMP-proto [0], which checked all my boxes and just did everything over websockets by default. 0: https://wamp-proto.org/ https://wamp-proto.org/
- pjbk 2y ago
  MQTTv5 now has RPC. You can specify a response topic in the request message, and also add 'correlation' data to track the context of the call.
kerblang 2y ago
I don't buy the b.s. around MQTT being so lightweight/efficient/etc. It's just using TCP/IP. Maybe by the standards of the day that was comparatively special, but I've never seen any real evidence for the claim besides this constant bragging. I guess it's nice that MQTT is a standard and so you have a way to connect to off-the-shelf devices that are enabled. I still think there are better options for pub/sub and message queues, esp. if you need to failover on the consumer side.
- bdamm 2y ago
  What better options? The neat thing about MQTT, and that almost all other pub/sub implementations that I've seen get wrong, is that the core data structure in MQTT is the subscribing client, not the queues & topics. That means you can map as large an address space as you want into the topic tree. The topic tree can be trillions and trillions of endpoints, even an embedded server could have a topic tree with one endpoint for every ipv6 address, if you wanted. So the topic tree is rich, your subscriptions can be as selective as you need, and the server can be fast on a low footprint.
  - mmcnl 2y ago
    Exactly. Even shorter summary: MQTT broker is stateless thus easy to manage and hard to break. This makes MQTT reliable.
- xnyan 2y ago
  > I still think there are better options for pub/sub and message queues, esp. if you need to failover on the consumer side. Sincerely, what are these options?
  - kerblang 2y ago
    The various message-queueing products, Kafka, et al. Sorry for late response and not responding sooner...
thesnide 2y ago
I'm wondering if someone tried https://mqtt-udp.readthedocs.io/en/latest/ https://mqtt-udp.readthedocs.io/en/latest/ Sounds like a very good idea on paper, which can even be brokered if needed

289 points by andypiper 2y ago

I discovered MQTT with Home Assistant (a home automation solution, no, sorry, THE home automation solution). It helped me to understand pubsub philosophy and I ended up using it for my personal development - to the point where I was misusing it (which is the typical path with technologies one discovers) It is a great protocol for home automation but quickly gets annoying for more complex scenarios.

dloranc 2y ago

What do you think about LwM2M in comparison with MQTT?

vv_ 2y ago
It is not directly comparable because LwM2M was originally developed for device management, while MQTT is primarily used for transferring telemetry or other opaque data. My personal gripe with LwM2M is that it uses CoAP (Constrained Application Protocol) as its primary transport protocol. Since CoAP is based on UDP, you’re forced to rely on DTLS for authentication and encryption. Managing sessions with DTLS can be quite frustrating, and unlike TCP, there’s no reliable indication of when a session is closed by your peer. Another issue is that it doesn't strictly follow the client-initiated communication pattern and can receive unsolicited messages from the server. I have found this to be very unreliable in real-world conditions for Device Management use cases.

Uptrenda 2y ago

The cool thing about MQTT is there's so many servers for it that are completely public -- and intentionally so. You can use it for real time messages but I think it also has quality modes that allow for some persistence. Some of these servers have stat pages that show how much data they transfer. Usually phenomenal amounts of data. It's pleasing to me as someone doing a lot of stuff with open source networking to have such an establish protocol (instead of say -- 10 crappy ones.) MQTT works and it works well.

roland35 2y ago
Have any examples of public servers?
- Scarjit 2y ago
  HiveMQ has a public one: https://broker.hivemq.com/ https://broker.hivemq.com/
- dikei 2y ago
  I only know this one https://www.mqtt-dashboard.com/ https://www.mqtt-dashboard.com/ But a quick Google search will get you some others.
- Risse 2y ago
  Helsinki Regional Transport (HSL) has a public server with realtime position of all of their vehicles (buses, trains). https://digitransit.fi/en/developers/apis/4-realtime-api/vehicle-positions/high-frequency-positioning/ https://digitransit.fi/en/developers/apis/4-realtime-api/veh...

bouvin 2y ago

I have been using MQTT for years in my IoT courses – it has proven to be a very versatile tool, and it is convenient that it is also supported over web sockets.

kitd 2y ago

About 15 years ago, Andy Stanford Clark's house became something of a news item, in the days before tweeting IoT devices were a thing: https://www.bbc.co.uk/blogs/technology/2009/06/things_that_tweet.html https://www.bbc.co.uk/blogs/technology/2009/06/things_that_t... The core protocol, devised in the days when 1 byte over a satellite link cost $1, is incredibly efficient and simple to implement.

baruchthescribe 2y ago

Reposting my comment from five years ago: I once had a client where the only port available for me to use on their firewalls was for MQTT (1883) because that's how we were getting sensor data from them. They would not open anything else for us no matter how we implored them so I wrote a live TCP wrapper over MQTT to get around it. It was a local multithreaded TCP daemon that listened for outbound requests on a certain port, wrapped them in MQTT and then published them using a unique topic. The server daemon would detect these topics and unwrap them before forwarding to our server processes. So the client machine thought it was making a live TCP connection to our server but in the middle was a funky invisible MQTT wrapper. It was really elegant once it worked but my goodness was it a pain to debug - a couple of months before I got the whole thing right because of all the blind alleys I went down.

ericvsmith 2y ago
I’ve worked at places where subverting firewall rules like this would get you fired. I now refer to such situations where I’m forced to twiddle my thumbs while no work gets done as “letting the process work”.
- exe34 2y ago
  I always kick this kind of decision upwards: we can do it this way and break this rule, so I need director level approval, or we can wait until the customer blinks.
  - n_plus_1_acc 2y ago
    Yes. Ask your boss for approval and tell them otherwise nothing will get done.
ocschwar 2y ago
So you basically implemented an MQTT-Sockets protocol. Which you could then use to connect to a WebSockets server on the other side.
- 2y ago

joezydeco 2y ago

I had a lot of fun on an embedded systems project recently where I used MQTT as an interprocess messaging system. Broker and clients on the same machine. If I needed to sniff or debug something it was as easy as putting the device on the network and using MQTT Explorer to record or inject messages. I could even let coworkers that were working remotely work with the system by opening the port outside the LAN.

klysm 2y ago
Haven’t seen this done much but it does seem to have some desirable properties. My biggest concern when using it as a system component has been durability guarantees. I don’t have a lot of trust on broker implementations not to lose data.
- logifail 2y ago
  > My biggest concern when using it as a system component has been durability guarantees. I don’t have a lot of trust on broker implementations not to lose data (If we ignore retained messages) MQTT brokers are not designed to store data, so there's no need to be concerned about durability because there simply isn't any...(!) It is (just) a messaging protocol.
- joezydeco 2y ago
  MQTT, by design, isn't going to have the confirmation mechanisms and safeguards that you could get from other methods or even sockets. But once you release yourself from that paranoia, it's pretty liberating. The nature of my project was that a missed message wouldn't be fatal, and message retention was good enough protection for a process that crashed and restarted.
roland35 2y ago
Would zeromq be an alternative here?
- joezydeco 2y ago
  It's an alternative, but there are some aspects to having the broker that I liked. If there was a bug I could halt the system, attach/subscribe an external dev machine, and grab all retained messages from the broker. I don't believe ZMQ allows that.

miljen 2y ago

Fun fact: at the same time, the most famous C++ library, Boost, is reviewing the `async-mqtt5` implementation (https://github.com/mireo/async-mqtt5 https://github.com/mireo/async-mqtt5) to be included in Boost as Boost.MQTT: https://lists.boost.org/Archives/boost/2024/10/index.php https://lists.boost.org/Archives/boost/2024/10/index.php

formerly_proven 2y ago
Is Boost still something people reach for in newer projects? Anecdotally most adoption I've seen happened in the 00s and very early 2010s (pre everyone mandating C++0x/C++11), I only rarely see it around these days. edit: boost.org is a blast from the past, still looks exactly like I remember it from like 2008. Down to the "Get Boost" shopped on an emergency off button!
- miljen 2y ago
  That is an interesting question. Here you can see the actual popularity of each library within Boost: https://grafikrobot.github.io/boost_lib_stats/ https://grafikrobot.github.io/boost_lib_stats/. Also, see the comments at https://www.reddit.com/r/cpp/comments/130bzj8/has_boost_lost_its_charm/ https://www.reddit.com/r/cpp/comments/130bzj8/has_boost_lost.... My company has been heavily utilizing Boost.Asio and Boost.Beast, so we've definitely reached for Boost in newer projects as well.
- spacechild1 2y ago
  AFAICT Boost.Asio is still the go-to networking library. (Though Asio is also available as a standalone outside of Boost, I didn't know that until recently.) Apart from that, Boost has quite a few goodies such as Boost.Json, Boost.Program_options, Boost.Interprocess, Boost.Lockfree, Boost.Unordered, Boost.Dynamic_Bitset, etc. Some libraries, like Boost.Atomic, Boost.Thread, Boost.Chrono and Boost.Filesystem, simply became obsolete with modern C++ versions. In fact, they served as the blueprint for the corresponding C++ standard libraries. Personally, I have been wary of using Boost in the past because it's such huge library, but CMake integration is actually quite good these days and I found it pretty easy to use. Documentation is also quite good IMO.

this_user 2y ago

Having recently used MQTT for a project, I can't say that I'm a huge fan. There are a lot of options in the protocol where it's not immediately clear what they do and why they are important, or in what combination they need to be used to make sure things work as intended, and the documentation is often not great at explaining. Part of this may also be on the Eclipse Mosquitto Python client that I had been using. I took me several days to figure out that the client was running into a race condition on a slower system that resulted in subscriptions to topics silently being ignored which caused the associated callbacks to malfunction despite following their documentation 100%. Overall, this has been one the messiest experienced I have ever had with a protocol, certainly one that isn't that old.

jorl17 2y ago
My experience using MQTT has been through the paho python MQTT library, and, while we definitely have managed to get a lot done, it has been a terrible experience. Everything about it rubs me the wrong way. The API design, the poor documentation, even the way they seem to not adhere to typical python conventions. It looks like it's easy to start using it, but then the breadth of protocols and implementations start getting to you. At a point in time, I remember the only reliable way we had of knowing if we had successfully connected to a server was to try to subscribe to a topic twice and catch a specific error code on the on_connect message (which actually was documented as a success code at the time). I know how crazy it sounds and maybe there was a better way to do it, but, if there was, I don't think it was that easy to find. It's easy to complain, though. I'm very grateful so many people have worked to create this library. If they hadn't, I wouldn't have built what we've built with it. I really admire people who take on these huge projects.
DannyBee 2y ago
So your client just sucks here. My experience with the eclipse clients (paho, etc) is similar to yours in both python and C++, - it makes it overly complicated and seems very low level. It also is somewhat buggy because of the architecture. I believe (perhaps wrongly), it's all basically maintained by one person or close to it (only one person has contributed to the C++ client in the past 6 months, for example, and nobody has contributed in the past 3), so i kind of understand how it's gotten this way over time. I filed a simple PR (1 word change) to fix an obvious bug and it took 2 years to review it and accept it with no changes. Again not a complaint, just trying to portray how the state feels - a lot of libraries, bugs, and work, and few overworked people helping get it all done. I moved to other clients and the experience is much much better in python, rust, C#, and C++. Most of them have a good combination of high and low level API's, so if you just want to send a message on a topic, you don't have to worry about acks, retries, etc. But if you need control, you can get it. Honestly, i worry at this point that keeping paho/etc alive in this state is doing more harm than good - if they were officially dead it would at least force the issue. Right now you end up with users who have an experience like yours, and then either give up on MQTT or assume they are doing it wrong :)

vv_ 2y ago

There aren’t many telemetry transfer protocols suitable for IoT besides MQTT (especially for resource constrained systems). It is incredibly simple to implement, has numerous libraries available in various programming languages, is supported by major cloud providers and is designed to operate over TCP (meaning you avoid DTLS). We have shipped many products that use MQTT for telemetry transfer to various different vendor servers and it always just worked. It was never a pain-point in stark contrast to CoAP or some proprietary protocols.

8fingerlouie 2y ago

It's such a great little protocol, and besides being "small enough" for hobby projects, it also scales enough to be used for something like Facebook Messenger [1] [1]: https://engineering.fb.com/2011/08/12/android/building-facebook-messenger/ https://engineering.fb.com/2011/08/12/android/building-faceb...

ErneX 2y ago

Most of my home automations depend on it, it’s been great honestly. Cheers to mqtt!

ronameels 2y ago

MQTT is being used a lot more in recent years inside of factories for sharing data between machines. Historically it's been used in Oil & Gas for SCADA (getting data from remote well sites). 10+ years ago we added it to Kepware (OPC server) and streamed tag values to "the cloud". I was at a conference giving a presentation on it when Arlen Nipper, one of the creators of MQTT, came up after the presentation and said I did a "decent job". It was humbling :). Fast forward to today, and we have a new company (HighByte) modeling factory data at the edge and sending it via MQTT, SparkplugB (protocol over MQTT), direct to S3, Azure Blob, etc, etc. All that to say, MQTT is a big driver in Industry 4.0, and it's cool to see it so heavily used all these years later.

fartfeatures 2y ago
Is MQTT being used in places where previously Modbus would have been used?
- ronameels 2y ago
  In some cases yes, controllers are adding in MQTT support directly or SparkplugB. It works in simple cases, but the lack of transactions & ability to control what and how often you're reading from the client side, can create challenges.
- InDubioProRubio 2y ago
  There is also MQTT integrating bus-hardware- you basically plug in as bus-hardware replacement which allows to send data directly from the bus to the broker. The PLC never know what hit it. https://www.ifm.com/de/de/product/AL1332 https://www.ifm.com/de/de/product/AL1332 Pretty wild times. PS: You can web GET/POST set and get ios with these. Robot experience that tragic irony POST {NOOO} and he just dropped it like its hot.
eska 2y ago
I also work in that industry and am quite puzzled by the technological inferiority, scope creep and NIH syndrome of the OPC foundation’s standards. I wish they just used sparkplug b and implemented their specs on top of that for semantics. Then again the async stuff they’re doing now is so over-engineered and terrible as well.. I took part in their teams meetings some time, only to find they had never read the mqtt spec (less than 50 easy to read pages) and didn’t understand what the headers are for for example (they wanted to put stuff in there that actually belongs in the payload). Some microsoft guy also took offense at my suggestion to first look at what competitors are doing with mqtt, because he wanted to create something new rather than copy. My company will just treat opc ua at the very edge and isolate it from our tech as much as possible on my suggestion.

NortySpock 2y ago

My most recent foray into MQTT was using the MQTT 3.1.1 compatibility mode in NATS, which was pretty easy to set up once I got the permissions sorted out. Works great for my smart home needs while also letting me explore latency and throughput load testing if I want to have a little fun.

artooro 2y ago
I'm using NATS as well, natively when possible but bridged to MQTT for devices that need it.

gorbypark 2y ago

I think my first project that's still "in production" and used daily was taking an existing SVG map of a water distribution system (snowmaking and fire suppression pipelines/pumps/valves/etc at a major ski resort). I made a MQTT topic for each pump, valve and section of pipe along with some state for each (water direction, pump or valve on/off, pressures, etc). It's a website, so I used mqtt.js and jQuery to update the colours and fills of the SVG to represent the various states. It's statically hosted and there's a mqtt broker that's been running in a container, untouched for nearly 10 years now. Mqtt.js is running over a web socket, so if the state is updated, it just magically updates for everyone else.

remram 2y ago
Holy shit Docker is 11 years old.
- deleted 2y ago
  [deleted]
- gorbypark 2y ago
  It was early days, yeah. The other part of the project that's been depreciated now was running node 0.4, if I am not mistaken. The watermap was part of a bigger asset tracking system, node + jade templating (before jade became pug, even). The watermap was actually embedded in a jade template, but when I depreciated the other system, I ended up just copy and pasting the output from the jade file into an index.html, and everything just kept working :D
brian-gilmore 2y ago
aaah I love me some AJAM ;)

InDubioProRubio 2y ago

A heap of dead "replacements" at the feets - what more can a communication standard want.

sunshine-o 2y ago

I never understood why we don't use MQTT for simple systems monitoring, with a push model. I mean it was almost designed for this use case. Also if using NATS things become very simple to setup.

Evidlo 2y ago

I remember one of the annoying things I found about actually using MQTT was that many clients didn't implement any sort of connect/disconnect callback and expected you to handle reconnection logic (e.g. exponential backoff) yourself. Also it didn't support RPC style requests for some reason. Also since MQTT is TCP-based you can't natively speak the protocol from a webbrowser without the broker also listening on a websocket. Ended up switching to WAMP-proto [0], which checked all my boxes and just did everything over websockets by default. 0: https://wamp-proto.org/ https://wamp-proto.org/

pjbk 2y ago
MQTTv5 now has RPC. You can specify a response topic in the request message, and also add 'correlation' data to track the context of the call.

kerblang 2y ago

I don't buy the b.s. around MQTT being so lightweight/efficient/etc. It's just using TCP/IP. Maybe by the standards of the day that was comparatively special, but I've never seen any real evidence for the claim besides this constant bragging. I guess it's nice that MQTT is a standard and so you have a way to connect to off-the-shelf devices that are enabled. I still think there are better options for pub/sub and message queues, esp. if you need to failover on the consumer side.

bdamm 2y ago
What better options? The neat thing about MQTT, and that almost all other pub/sub implementations that I've seen get wrong, is that the core data structure in MQTT is the subscribing client, not the queues & topics. That means you can map as large an address space as you want into the topic tree. The topic tree can be trillions and trillions of endpoints, even an embedded server could have a topic tree with one endpoint for every ipv6 address, if you wanted. So the topic tree is rich, your subscriptions can be as selective as you need, and the server can be fast on a low footprint.
- mmcnl 2y ago
  Exactly. Even shorter summary: MQTT broker is stateless thus easy to manage and hard to break. This makes MQTT reliable.
xnyan 2y ago
> I still think there are better options for pub/sub and message queues, esp. if you need to failover on the consumer side. Sincerely, what are these options?
- kerblang 2y ago
  The various message-queueing products, Kafka, et al. Sorry for late response and not responding sooner...

thesnide 2y ago

I'm wondering if someone tried https://mqtt-udp.readthedocs.io/en/latest/ https://mqtt-udp.readthedocs.io/en/latest/ Sounds like a very good idea on paper, which can even be brokered if needed