
The realtime magic behind Moot and how we deliver all your instant notifications - courtneycouch0
https://moot.it/blog/technology/real-timey-wimey-stuff.html
======
Ecio78
Ehm.. tens of millions of realtime events but no "www." alias?:

404 Not Found

Code: NoSuchWebsiteConfiguration

Message: The specified bucket does not have a website configuration

BucketName: www.moot.it

RequestId: xxxx

HostId: xxxxx

I didn't know what "Moot" was so I tried www.moot.it

~~~
courtneycouch0
Hmm I think you must have hit it while we were doing a deploy. Looks like the
www alias points right to S3 instead of our CDN. The CDN would have hidden the
bucket reconfiguration. Oops.

Will be updating the alias to point to the CDN. Thanks for catching that.

~~~
Ecio78
You're welcome!

(less welcome whoever downvoted me :))

------
lcasela
You guys should really change your name.

~~~
alanctgardner2
You're right, the ents will be pissed ;)

It's a really good name for forum software - better than Discourse for a few
reasons:

\- one syllable, one easy phonetic spelling \- archaic, so it won't be
confused in everyday speech \- it doesn't sound like it might do something
else. Discourse could be online learning, golf venue rankings, etc.

~~~
mhurron
Ignoring the 4chan aspect, Moot is archaic and won't be confused in everyday
speech? Is English not your first language?

[http://dictionary.reference.com/browse/moot](http://dictionary.reference.com/browse/moot)

One definition is of archaic usage, but the others, not so much.

~~~
alanctgardner2
It's used in one idiomatic expression, "the point is moot", which is almost
always misused to mean 'irrelevant' rather than 'debatable'. The funny thing
about idioms is that many are atomic: native speakers don't bother to dissect
the individual words, they use the whole phrase because it signals something
specific to other fluent speakers. When you learn a second language (French,
in my case), these expressions are interesting because they usually do have an
outmoded meaning, but if you ask someone they have a very hard time
translating or explaining. They're also a giant pain unless you learn through
immersion, because they're habitual and not usually systematic in any way.

The "archaic" usage at that link is actually very much alive, it's a technical
term used in law school for mock trials [1]. I would argue law students
actually understand the meaning of the term better than average, native
English speakers, making the technical term less archaic.

1\.
[http://en.wikipedia.org/wiki/Moot_court](http://en.wikipedia.org/wiki/Moot_court)

~~~
courtneycouch0
This.

Also the word overwhelmingly is used to describe a meeting to discuss a topic
throughout Europe.

It's not an unusual statement in Europe to have a moot or attend a moot. We're
primarily a Finnish company so the name for us is meaningful.

------
RyanZAG
If you want something similar to this and you're on the JVM, you can use open
source Atmosphere[1] which handles this flow exactly (websocket with fallback,
multiple 'tags', etc).

[1]
[https://github.com/Atmosphere/atmosphere](https://github.com/Atmosphere/atmosphere)

~~~
Cyranix
I integrated Atmosphere into a Dropwizard-based project (not on GitHub yet,
hoping to get it out there in a week or two -- look for Cyranix/mi5 if
interested). Although I did get it to work, it wasn't a smooth experience.
There seem to be many ways of accomplishing the same thing which are just
different enough to trip you up in subtle ways (at least if you're only a
moderately experienced dev like myself), and the documentation is a bit of a
mess (scattered, not well translated from French, etc. -- not entirely, but
enough to be a pain). Any JavaScript devs interested in this sort of tech
could make a much-appreciated contribution to the project by making a client-
side script that doesn't depend on jQuery.

------
jfager
"This is then picked up by every JSON-RPC server and they will intersect those
two tags, sending the event {hello: 'world'} to any connection that matches."

Do you see this eventually becoming your bottleneck (every rpc server has to
handle every event)? If so, what's the plan for reducing the event fan-out?

~~~
courtneycouch0
Yes we do see this as becoming a bottleneck. We are already working on a
solution for that. I mention in the "What's Next" section about a bit more
advanced routing using ZeroMQ instead of Redis.

The problem is a difficult one to say the least. We're nowhere near the
bottleneck now, but we do want to resolve this long before it is close to
becoming an issue.

~~~
egeozcan
Could you please elaborate on how using ZeroMQ would solve your problem? I
have a surprisingly similar setup and I'm also worried that this is going to
be a bottleneck.

~~~
courtneycouch0
ZeroMQ doesn't solve the problem. ZeroMQ is flexible enough that we can build
a solution. We're working on distributed routing and registration using ZeroMQ
and Redis as it's data store.

The RPC servers register the tags and channels we distribute based on that
data. Any ZeroMQ server can process and deliver any event to the end RPC
server.

We'll probably do a detailed writeup when this is done. We plan on using this
for all the infrastructure wiring when it's done.

~~~
outside1234
Please do a writeup when you've done this - would love to read about this -
very relevant to my work as well.

------
pearjuice
When you registered the domain name and selected branding, did you know moot
is a well-known internet handle?

~~~
saurik
[https://news.ycombinator.com/item?id=4806787](https://news.ycombinator.com/item?id=4806787)

~~~
pearjuice
No further imformation/action after that?

~~~
saurik
It came up again halfway between then and now in comments here; I do not know
of any resolution.

[https://news.ycombinator.com/item?id=5486786](https://news.ycombinator.com/item?id=5486786)

------
fooyc

        > tens of millions of realtime events daily
    

Does it means hundreds per second ?

~~~
courtneycouch0
During peak hours thousands per second, off hours not many at all.

------
Kequc
I followed it the whole way through until I got to the diagram of the way Moot
handles channels compared to static channels. What does the diagram
illustrate? It seems to be that the Moot version doesn't have central input.
But it would all need to be filtered through something central, otherwise it
is completely possible to miss a location the payload was intended to be
delivered to.

~~~
courtneycouch0
That central location is Redis pub/sub. Multiple publishers publish to Redis
and many JSON-RPC instances listen and then process the notifications to
distribute.

The diagram is more meant to illustrate that the notifications don't just
blanket deliver to all subscribers rather than be an exactly technically
accurate.

~~~
mmmooo
Realistically, it sounds like the 2nd diagram should have the middle section
of the first diagram. As in, a single message/circle, gets "split" (or more
accurately dup'd) and somewhat simultaneously delivered to the start of every
'channel'. The way it is now makes the appearance that a message isn't sent
down a channel where the logic gate will not eventually deliver it. But that
doesn't seem to be the case.

~~~
JLehtinen
Yeah, yeah, I know: I'm just a graphic designer without a deep knowledge of
the technology... But what happens in the diagram is still pretty close to
what you describe, to be fair: messages duped and delivered simultaneously.
Absolute accuracy would have made for a pretty complex illustration. Point
taken anyway, I'll try harder next time!

Btw, did you notice it has blue heads? I thought those were cool.

~~~
Kequc
Why are they blue do they not understand the diagram they are being used in
either?

------
chinmoy
Do you have any plans to open source this for people who wants to install Moot
on their own server? As a forum junkie, I'm excited about Moot, but I hope you
do understand that to be successful as a platform you have to give the option
to self-host :)

~~~
courtneycouch0
Our infrastructure is built as a service. I totally understand that many
people want to run their forum on their own network and that's totally ok.
There's plenty of products out there servicing that market.

If you were to compare us to blogging companies we're more inline with tumblr
than wordpress. We're a service that makes it simple for anyone to add forums
and commenting.

Our service obviously isn't for everyone, but we feel like there is a large
market for simple to set up and manage forums that can integrate with little
to no technical knowledge or expertise.

------
javajosh
I've built something very similar using Vert.x, which is sort of like Node on
the JVM. But instead of Redis, we have full on state machines at the network
turn publishing on the event bus. It's pretty sweet, and I hope to open source
it soon.

------
speeq
Are you going to open-source your json-rpc library? If so, is there a specific
timeframe you have in mind? I'd really love to use this technology in my own
projects.

~~~
courtneycouch0
Yes we intent to open source all of this. No timeframe in mind. We need to get
through our beta roadmap before packaging up the various pieces we plan on
open sourcing.

~~~
speeq
Thank you so much, that's amazing! :)

------
sergiotapia
On Moot: I registered my account because I wanted to integrate Moot on my new
website I'm building and when trying to integrate it to a view in Rails:

    
    
        `a class="moot" href="#{request.original_url}"> Awesome Startup`
    

I'm getting an message on the page and no moot forums.

" (moot logo) startup does not exist."

Isn't this like Disqus where I can have commenting threads based on the page?

~~~
tipiirai
Yes. You can have commenting threads based on the page. The value on the href
attribute takes a form

https\://moot.it/i/[forumname]/[an_unique_path]

You'll get the [forumname] from the setup
([https://moot.it/setup](https://moot.it/setup)) and you need to pick an
unique path for each of the commenting area. The paths can be designed to fit
your site's structure. For example

/blog/2013/august:my-awesome-entry /blog/2013/june:another-blog-post

You can list all comments from any path such as

/blog/2013

The "/i/" part means SEO indexed content. Our servers automatically generate a
search engine optimized version of all the comments that are structured
according to the path structure. Here is the SEO content for our forums for
example

[https://moot.it/i/moot](https://moot.it/i/moot)

And here are all comments from our release blog entries

[https://moot.it/i/moot/blog/release](https://moot.it/i/moot/blog/release)

The way we handle SEO is one of the differences from Disqus.

Here is more information about the paths:

[https://moot.it/docs/embedding.html#paths](https://moot.it/docs/embedding.html#paths)

------
sesaya
Interesting, but the inability to delete posts and host it myself make it
useless for me. I want freedom, damnit.

~~~
JLehtinen
Reasons for why Moot is a hosted solution:
[https://moot.it/docs/technology.html#distributed-
service](https://moot.it/docs/technology.html#distributed-service)

------
swah
I wanted to follow you on Twitter so I get updates about your product, but you
don't seem to have one :)

~~~
courtneycouch0
@letsmoot :)

