
Deepstream.io – Open Realtime Server - wolframhempel
https://deepstream.io/#1.0
======
kabes
I really like that the documentation is quite honest and unbiased. Like with
the integrations:
[https://deepstream.io/tutorials/integrations](https://deepstream.io/tutorials/integrations)
it always mentions downsides next to the upsides.

~~~
benjaminjt
This is a really good point actually. I particularly liked the overview of
other realtime libraries[1] for the honesty factor, and it really helped me
understand how it relates to more familiar frameworks too.

[1] [https://deepstream.io/blog/realtime-framework-
overview/](https://deepstream.io/blog/realtime-framework-overview/)

------
asadlionpk
This is great! I wish somebody would write a Firebase API compatible wrapper
around this so escaping Firebase lock-in would be possible without altering
code.

I might start it myself when I find some time.

~~~
cheriot
Or even just integration testing Firebase without hitting a remote service!

~~~
asadlionpk
that might be overkill. Here is a better solution:
[https://github.com/urish/firebase-server](https://github.com/urish/firebase-
server)

~~~
cheriot
I looked into it. firebase-server doesn't work with recent firebase clients
and it's not a priority for the author.

------
MasterScrat
From a quick look at it it looks like a self-hosted Firebase.

Would that be a good approximation?

~~~
wolframhempel
yup, that's not too far off. It also gives you Pub/Sub and Request/Response
though. Also Firebase uses one huge data-structure for data-sync whereas
deepstream breaks data down into smaller objects called "records" with
individual lifecycles

~~~
duncanawoods
Huh. I visited your site and reading the tag line "a fast, secure and scalable
websocket & tcp server", I assumed it was just a socket server and had no idea
it was actually a data-store.

~~~
MasterScrat
Yeah I mostly understood that from the example.

You really need to make the explanation clearer ^^

~~~
wolframhempel
Very much agree. Would you have any recommendation?

~~~
MasterScrat
I would show what it does step by step: one box with data-sync, one with real-
time data sharing, etc... with an animation for each.

Your animation "built for a new generation of apps" is awesome but it doesn't
not show clearly what role deepstream plays in it.

I don't think it is possible to explain everything clearly in one sentence so
I wouldn't start with a tagline like that.

~~~
rambos
+1 something like this is defintley needed

------
partycoder
While it looks interesting, I honestly don't feel like building a system like
this on top of node. Not because of the library itself, which seems properly
implemented (hard to see libraries using proper formatting and JSDoc these
days).

node is a simple, friendly technology, until you to profile it. Then it's not
so friendly. With a realtime system you require a great deal of profiling and
quite honestly the tooling is not there.

"Oh but what about Flamegraphs and DTrace?"... it is nothing compared to what
you have in Java. And node tooling is better on obscure operating systems like
IllumOS.

If I had to implement a system like this I would opt for Scala + Akka, or
Elixir/Erlang. If it's for simplicity/budget reasons, try PubNub.

~~~
dragonne
What tools do you use to profile a Scala + Akka service? IME they are quite
opaque to traditional Java tools like VisualVM. What am I missing?

~~~
partycoder
Scala can be profiled with VisualVM.

You can monitor Akka with kamon.io, or using the tool offered by the authors.

------
reactor
Genuine question, how does this different from
[https://github.com/centrifugal/centrifugo](https://github.com/centrifugal/centrifugo)
?

~~~
wolframhempel
I'm afraid I don't know too much about centrifugo, but it looks like a pub-sub
server you can connect to Redis. deepstream on the other hand provides
datasync (stateful, persistent JSON objects you can subscribe and listen to),
request-response and pubsub in a scalable way with granular permissions and
authentication strategies and an ecosystem of connectors that make it work
with different databases, caches and message buses.

~~~
reactor
Thanks, any plan for the android client?

~~~
wolframhempel
sorry, not quite following. There is an android client here:

[https://deepstream.io/install/android/](https://deepstream.io/install/android/)

~~~
reactor
Looks quite interesting, I was evaluating few real-time frameworks (esp
socket.io) for the past couple of days . Could you pls shed some lights on how
easy to use deepstream to setup a one-to-one in-app chat/messaging setup for
android? I guess its the event's are the way to go.

For eg: client.event.subscribe('channel1', eventCallback)
client.event.emit('channel1', 'hello how are you?')

How scalable is that? can it support thousands of channels without any hiccups
on a single medium digital ocean instance?

~~~
wolframhempel
It scales very far. We've conducted benchmarks lately that showed that a
single deepstream on an AWS t2.medium instance can comfortably handle
160.000~200.000 messages a second and a cluster of six ~ 4 Billion an hour.
Please find the detailed results here

[https://deepstream.io/info/performance/four-billion-
messages...](https://deepstream.io/info/performance/four-billion-messages-per-
hour/)

For your chat though you need to consider whether your messages will be
persistent and mutable (changeable after they've been posted) or are just one-
off messages. In the first case using a list
([https://deepstream.io/tutorials/core/datasync-
lists/](https://deepstream.io/tutorials/core/datasync-lists/)) with references
to records ([https://deepstream.io/tutorials/core/datasync-
records/](https://deepstream.io/tutorials/core/datasync-records/)) is the way
to go, in the second case pub/sub
([https://deepstream.io/tutorials/core/pubsub-
events/](https://deepstream.io/tutorials/core/pubsub-events/)) might be better

~~~
stemuk
Do lists offer pagination for lazy-loading?

------
akanet
Do you provide a version of Firebase's onDisconnect?
[https://firebase.google.com/docs/reference/js/firebase.datab...](https://firebase.google.com/docs/reference/js/firebase.database.OnDisconnect)

It's very hard to correctly manage distributed state without a guarantee
similar to this.

~~~
wolframhempel
Yes, there are granular connection states and associated events. Messages are
queued during connection loss. Please find more information here.

[https://deepstream.io/docs/client-
js/client/#connectionstate...](https://deepstream.io/docs/client-
js/client/#connectionstatechanged)

[https://deepstream.io/docs/common/constants/#connection-
stat...](https://deepstream.io/docs/common/constants/#connection-state)

~~~
akanet
Sorry, I should have been clearer. I don't mean "does Deepstream tell the user
when it's disconnected", I mean:

Does the Deepstream client have a facility (like Firebase) to instruct the
remote server to perform an action in the event the client disconnects in the
future and the client is unable to get _any further message_ to the server?

~~~
wolframhempel
ehm...this is not Horizon. But its a good question none the less. No, it does
currently not have that facility, but it would certainly be a good addition

~~~
akanet
Hah, sorry, too many tabs open lately. I'll edit. Thanks for the answer.

------
samblr
Could you please shed some light on concurrent connections benchmarks ? Im
looking for a scalable solution - I am trying out socket.io and I read from
others there are some performance issues as it scales.

edit: added socket.io part

------
oakesm9
How does this compare to something like ShareDB
([https://github.com/share/sharedb](https://github.com/share/sharedb))? It
seems on the face of it that the data part of it is pretty similar (albeit
Deepstream is more fully featured with auth etc.), however, there is no
mention of how Deepstream handles conflict resolution if two clients are
modifying the state.

Internally ShareDB uses Operational Transformation with special operations
which are designed for use in JSON documents.

Does anyone know how this works in Deepstream?

~~~
wolframhempel
There's a choice of merge algorithms as well as the ability to hook in your
own.

[https://deepstream.io/tutorials/core/handling-data-
conflicts...](https://deepstream.io/tutorials/core/handling-data-conflicts/)

~~~
oakesm9
Thanks! I hadn't found that before.

Sounds like having those merge algorithms/hooks and splitting up the document
into multiple "records" might actually be enough for my use case.

------
marknadal
Wow, deepstream looks really good (and I'm a competitor,
[http://gun.js.org/](http://gun.js.org/) ). Your guy's product seems to have
significantly matured over the last year. I'm very impressed! Up for chatting?
mark AT gunDB DOT io (I still like your guys old website better though with
the server/client graphic animation).

~~~
wolframhempel
Ooh, thanks so much. Just heading out for the inevitable "we've made the
frontpage drink", but let's get in touch tomorrow. The diagram is still there,
but buried in [https://deepstream.io/tutorials/core/cluster-
messaging/](https://deepstream.io/tutorials/core/cluster-messaging/)

~~~
marknadal
Yeah that one! I love that graphic! Your profile doesn't have your email
address in it, so you'll have to ping me. Tomorrow sounds good. Congrats on
the launch!

------
pubnub
I love NodeJS. So many NPM's to be had. I want to open source our ANSI-C
Realtime Server - maybe someday. Our [https://www.pubnub.com/blocks-
catalog/](https://www.pubnub.com/blocks-catalog/) BLOCKS engine actually
starting in Java. That was a M-I-S-T-A-K-E-E-E-E. It's better now.

------
makkesk8
I like it but coming from socket.io I have a hard time with it. For instance,
Where/how would I put my server logic? The "Node API" as its called isn't
really helpful or not documented enough. But then again, I could have
misunderstood what deepstream is all about or how you are supposed to be using
it.

~~~
wolframhempel
It's a different beast from socket.io :-) your server logic is just another
client, written in java or node. There's a lot of functionality that let's you
listen in on what clients request etc. Please find more details at
[https://deepstream.io/tutorials/core/active-data-
providers/](https://deepstream.io/tutorials/core/active-data-providers/)

------
firdaus
Can you state your stance on AGPL[1] on the website like MongoDB and RethinkDB
do to put people's minds at ease.

[1]
[https://github.com/deepstreamIO/deepstream.io/issues/298](https://github.com/deepstreamIO/deepstream.io/issues/298)

------
dschiptsov
Realtime? In nodejs?

It must be a webscale realtime, or at least reactive realtime..

~~~
hackerboos
If you're willing to put the work in to load balance then it can probably
scale quite well.

I'm always baffled why people choose JS over Erlang/Elixir which is designed
to do this and comes with the tools/libs required to scale.

~~~
elcct
Exactly word "probably" is a deal breaker in this case. I am also curious how
someone was thinking JS will be a good idea for such project. But hey, there
are people building cars out of plastic bottles...

~~~
wolframhempel
We are working with Alex,creator of µWebSockets
([https://github.com/uWebSockets/uWebSockets](https://github.com/uWebSockets/uWebSockets))
on moving all connectivity and networking aspects to low level C

~~~
wolframhempel
yes, that's one major aspect of it. engine.io adds is own protocol and isn't
exactly the most performance tuned library

~~~
wolframhempel
Sorry, it doesn't let me reply for some reason :-( Development is in progress,
there will be figures soon

~~~
stemuk
It seems like you have to wait some seconds for the server to process the
comment, if you refresh it should let you comment again. Anyway, looking
forward to the data release, through a blog post I suppose?

~~~
wyldfire
That feature of HN is by design. It's there to force you to think about your
replies instead of engage in knee-jerk flamewars. :)

------
stemuk
Are there any plans for an iOS/Swift client? So far it seems only js and Java
clients are available...

~~~
wolframhempel
Yup, we're in the process of finalizing it. There's a related issue at
[https://github.com/deepstreamIO/deepstream.io/issues/68](https://github.com/deepstreamIO/deepstream.io/issues/68)
with links to the beta artefacts.

~~~
stemuk
Nice, really appreciate the progress here! However, is there any reason the
artifacts aren't on github? It may be more convenient when it comes to
issues/changelogs etc.

~~~
wolframhempel
agreed, they will be soon. At the moment the core is cross-compiled from the
Java-Client using j2obc, hence the underlying repo is the java one. We are in
the process of evaluating how well this worked.

~~~
stemuk
:D I am actually trying to do something similar with the js client. I recently
tried to convert it to Dart using jsparser, since i am working with
[http://flutter.io](http://flutter.io), a hybrid app development project from
Google. It worked out just fine for now, currently testing it to get the weak
spots fixed.

------
phelm
How does this compare to RethinkDB / Horizon ?

~~~
wolframhempel
Please find a detailed answer here:

[https://deepstream.io/tutorials/integrations/db-
rethinkdb/](https://deepstream.io/tutorials/integrations/db-rethinkdb/)

------
wolframhempel
oops, thanks so much, fixed it

------
aw3c2
Can't read the text, please review the contrast part of
[https://www.w3.org/TR/WCAG20/](https://www.w3.org/TR/WCAG20/)

~~~
stemuk
This might help you out for now (if you are using Firefox):
[https://support.mozilla.org/en-US/kb/accessibility-
features-...](https://support.mozilla.org/en-US/kb/accessibility-features-
firefox-make-firefox-and-we)

------
lpinca
I'm confused. 1.1 has been released 13 days ago
[https://github.com/deepstreamIO/deepstream.io/releases/tag/v...](https://github.com/deepstreamIO/deepstream.io/releases/tag/v1.1.0)

~~~
wolframhempel
yes, but we wanted to give it an initial test phase before announcing it to
make sure we didn't miss anything major

~~~
andrewwhartion
Isn't that what a release candidate is for?

------
120bits
Page Not Found. When I clicked on Getting started

[https://deepstream.io/core/getting-started-
quickstart/](https://deepstream.io/core/getting-started-quickstart/)

Edit: Clicked on tutorials and then Getting started.

