
50 Million Concurrent PubSub Connection Infrastructure @ Hotstar - piyushgupta27
https://blog.hotstar.com/building-pubsub-for-50m-concurrent-socket-connections-5506e3c3dabf
======
polskibus
I'm curious - can MQTT or some enhancement to it achieve persistence and
exactly-once-delivery semantics (or at-least-once semantics) ? I am looking
for a message queue with guaranteed delivery that can be embedded into
existing platform and packaged with it (so that there is no external
dependency for messaging, like Kafka, etc.).

I investigated NATS Streaming but found some information that discouraged me
from using it.

~~~
piyushgupta27
Yes. MQTT is just a protocol and supports three types of Quality of Service
(QoS): \- QoS 0: Fire and forget \- QoS 1: At least once \- QoS 2: At most
once

You can choose to send messages via any one of these QoS which suits your use-
case. Apart from this, standard MQTT implementations also involve a "retain"
flag for a topic and a message which means that the brokers will maintain the
last sent message on the given topic. When new clients subscribe on this topic
with "retain" flag set as true, they will instantly receive the last retained
message.

Apart from these, there are more extensions that you can build to add
persistence to your system. This depends on the specific MQTT implementations
which you might have to find out.

~~~
polskibus
Thanks a lot. Do you know of any real world mqtt deployments that deploy QoS 1
or 2 with success on non trivial scale? Can you recommend a broker impl. that
is well tested in those scenarios?

~~~
piyushgupta27
There are some big players that use MQTT although i am not sure about the QoS
levels. I know WhatsApp used MQTT, although i am not sure if they still do it
not or what QoS they used. My bet would be they used QoS 1.

There are multiple brokers for MQTT: Mosquitto, HiveMQ (Java), VerneMQ
(Erlang), EMQ: the one we used (Erlang) etc. You can find out more brokers
that are available as well.

------
theprogrammerin
Wow! This is good stuff, I would be waiting for the unicast blog using MQTT

~~~
piyushgupta27
Sure. We have already launched the Unicast functionality in Beta and are
monitoring it closely. We will be sharing the Unicast post as well soon.

------
acloudguru
Amazing stuff! Thanks for sharing.

~~~
piyushgupta27
Thanks! The blog outlines our journey for the last 8 months. We ended up
sending more than 250Billion+ messages over a Live Cricketing tournament in
India. (VIVO IPL 2019)

