
Show HN: WebSocket-first development - jasonl99
https://github.com/jasonl99/card_game
======
Matthias247
I did a lot of websocket first development, since I ported various plain TCP
based communication frameworks to websockets. You can do lots of cool stuff
there, from plain request/response protocols that are faster than HTTP/1 to
advanced stuff like publish/subscribe mechanisms and even protocols that keep
the clients state automatically synchronized to the server side.

For advanced use-cases that's certainly a way to explore. For the average user
I'm not sure whether I would recommend to go the route. You lose a lot of
stuff that HTTP gives you out of the box, e.g. a request/response abstraction,
dozens of frameworks and mechanisms for authorization, logging, etc. HTTP is
also easily load-balanceable, while websockets are not. HTTP/2 in principal
makes small request/response exchanges as fast as you could implement them
with a custom websocket protocol, so there's no more gain. And SSE allow to
push server updates in a form that still fits the HTTP model and frameworks
pretty good. There are still lots of ways in which websockets can improve
reactivity, but you have to invest a lot of engineering effort into it, which
might not be worth it depending on the application.

------
marknadal
This is cool, but I'm a bit confused. Websockets have been around for a long
time now, and there are tons of popular libraries (like socket.io, but more
importantly, you should check out the high performance
[https://github.com/uWebSockets/uWebSockets](https://github.com/uWebSockets/uWebSockets)
instead).

Is all this doing trying to encourage awareness that using websockets is good?
If so, awesome. Else, what is it? Because it does not discuss how Websockets
__lose state __on reload. Your game will be in a completely wrong state.
Managing and maintaining state is the important thing (which is
what[https://github.com/amark/gun](https://github.com/amark/gun) does, which
is of course also "websockets-first"). When you reload your game, it should
return/remain as it was, or else half your players are going to see a half-
broken system!

~~~
jasonl99
Author here...Well, I started down the websockets path because of how awesome
I found crystal to be. I wasn't intending to do this a few months ago, it just
sort of took on a life of its own :)

The framework as I've written takes state into account; in fact it was/is a
design goal. There are three types of server objects currently - WebObject,
StaticBuffer and DynamicBuffer. A WebObject stays instantiated on the server
even when no users on viewing it (at some point it will be garbage collected).

At any time, rendering that object will produce the exact representation as
seen by someone who has viewed it and been updated with websockets. In fact,
the card game is a good example of this. Start a game, draw a few cards, start
another game, and then go back to the first. It'll be in the same state you
left it in, complete with the events that occurred while you were gone. This
was one of the top goals.

DynamicObjects are just arrays of WebObjects in a configurable buffer - think
of a list of sports scores, where each item handles its own update.

StaticObjects are an array of simple strings of html (think of tail to view a
log file).

~~~
marknadal
ahhhh! Okay, yeah then that is super awesome and very impressive. You
definitely should highlight/emphasize that more in the piece then. Cause that
is super exciting, and the ideal way to build/have things work. FRP reactive
streaming stuff, as the buzzwords go.

So next up, how do you handle conflicts? Two users write to the same thing at
the same time? Great work!

------
kahnpro
For everyone here who seems to be working with websockets, how do you deal
with the potential that a browser goes offline and misses messages? How do you
reconcile changed offline state with more recent events on the server? An OT
library?

Do you have a way to cluster websocket servers so that events are propagated
to all clients?

~~~
Matthias247
That's a good and important question. I think the solution depends on your
application requirements. If all you need to do is get the updated current
state in the browser then it's enough if the browser resubscribes after each
connect. It would then get pushed the new current state and can display it. If
you are not only interested in a current state but all state transitions (or
events that describe them) then it gets harder, since you would need to store
all the events somewhere on server side, and only remove them once a browser
has acknowledged that they have been consumed. In such a model the difference
to fetching the events via HTTP would probably not be too big.

~~~
mr_luc
Yeah - if I had to handle requirements more towards the painful end of the
scale I would probably say Agent + Event Store. Ie, ephemeral process(es)
representing the user on the server side, with the ability to cache up to a
certain amount of outgoing messages, respond to user's acks, and also to be
wound down if the client hasn't consumed for too long. The Agent is a natural
place for decisions about how, or if, to attempt to get the user 'caught up.'

------
anilgulecha
I think websockets as you're using it is a stand-in for realtime-first
development.

This territory is well explored -- services like firebase/parse are a
testament to many usecases.

IMO, this isn't per-se a new paradigm in the likes of offline-first, which is
advocating for local application cache/data synced with a remote store, this
offering the benefit of offline apps, with the sync/realtime benefits of
network connected apps.

~~~
k__
> this isn't per-se a new paradigm

If you look how regular websites work, I think it is.

In my last project, people were kinda baffled, why the changes they made,
didn't propagate to the other users instantly.

5 years ago everyone knew you had to refresh a page to see changes. Today
everyone seems to expect these changes to be pushed to them.

It also was a fight to get this whole stuff working with React and Redux. So I
think, if you want to build a good realtime site, you have to think about this
stuff before choosing libraries, if you want to get good results.

For example, GraphQL has subscriptions that can be delivered via Websockets.

~~~
jon-wood
It surprises me getting it to work with React and Redux was a struggle, they
seem like the perfect tools to handle a stream of state updates coming in
realtime. You could practically dispatch client side events straight from the
websocket stream to update local state.

~~~
catshirt
almost commented the same thing earlier. i was surprised how manual adding
multiplayer support was for redux/react.

the solution was still pretty simple, for my app: 1. middleware that sends all
actions to a server 2. websocket client that dispatches actions when they
receive them. this works, but doesn't actually reconcile or keep data on the
server which isn't suitable for most applications.

anyway yeah, i was surprised there was no robust or standard library...

------
ocharles
It's a shame Server-Sent Events aren't universally supported. I find them a
lot simpler than WebSockets - modifications are just normal HTTP calls, and
then you can broadcast the actual result of the transactional call back down
with a SSE. But Edge doesn't support them :(

~~~
drdaeman
Based on
[http://caniuse.com/#feat=eventsource](http://caniuse.com/#feat=eventsource)
only IE/Edge doesn't have it, and there are polyfills for those, e.g.
[https://github.com/Yaffle/EventSource/blob/master/README.md](https://github.com/Yaffle/EventSource/blob/master/README.md)

Or there are more problems?

Want to use SSE for one small project soon (haven't used it before, and don't
want Websockets as they're an overkill for a basic event stream), would
appreciate any info about how things could go wrong.

~~~
taf2
Again imagine if MS focused on user experience and used blink / WebKit -
instead of rolling their own.

We'd have a more standard and open web with fewer set backs. More engineers
working on the same engine. A larger team working on a shared _native_
application for everyone to build on for them to sell ads too. It even makes
good business sense for them now. One can dream right?

~~~
striking
Edge has much better battery life than other browsers on Windows, much like
Safari on Mac. Neither of those are the best browser, but when my choice is
between six or ten hours of battery life, it's easy for me to decide.

I dream that one day Webkit / Blink will actually be efficient.

~~~
nacs
> I dream that one day Webkit / Blink will actually be efficient.

Apple's Safari uses Webkit and does get increased battery life on Macs. Theres
no reason MS couldn't do the same with Webkit and optimize for battery life
using the same engine.

------
JangoSteve
Cool demo! I like that you can see all the events in real-time. I built a
similar demo ([https://os.alfajango.com/websockets-
demo/](https://os.alfajango.com/websockets-demo/) and
[https://github.com/JangoSteve/websockets-
demo](https://github.com/JangoSteve/websockets-demo)) a while ago to showcase
using websockets with a node app for a presentation I was giving at a
JavaScript user group ([https://os.alfajango.com/websockets-
slides/](https://os.alfajango.com/websockets-slides/)).

The presentation goes through a bit of the history and lead-up to websockets
for context.

One of my favorite things about the demo for the presentation was that
everyone in the room (probably about 50 people) was able to connect to the
live demo in real-time and some people started finding XSS vulnerabilities in
it right away, and using the app to send emojis and pictures to others in the
room in real-time (all harmless stuff). It was crazy though how many events
you could see streaming through the app just running from my laptop with so
many people connected and interacting at once (including x,y coordinates for
mouse movements a few times per second per user as well).

The presentation goes through alternatives as well, including long-polling,
short-polling, and server-side events (the slides are laid out in two
dimensions, so pay attention to when you can hit the "down" arrow instead of
the "right" arrow for more slides). It also delves a bit into the actual
websocket spec. It was a while ago though, so it doesn't go into anything
HTTP2-related.

------
slindz
I have two projects on the go right now. Both have gone the web sockets first
approach. I love it.

For me, the project I keep rebuilding to learn new languages/frameworks is
Risk (though it's been quite awhile now).

After spending so much time with Phoenix channels on my main project, I
couldn't resist the fun of reimplementing it over sockets. It'll probably wind
up on GitHub when I'm done.

~~~
Kexoth
But isn't channels the Phoenix abstraction (middleware) for websockets/sockets
per se?

What I mean is why not write a sockets Phoenix channels transports adapter to
be used instead of the default one & continue using the existing codebase? :)

~~~
slindz
Sorry! Reading my comment back, it's no wonder it was tough to follow.

Instead of writing: "I couldn't resist the fun of reimplementing it over
sockets."

I should have wrote: "I couldn't resist building another version of Risk with
Phoenix/channels/sockets because it should be mind-meltingly easier compared
to all of the previous times I'd built it on other platforms"

------
hardwaresofton
Is it wise to get hooked on websockets given that they're disappearing in
HTTP2? I assume lattice is not tied to websocket as the implementation, and
can switch to doing things over webrtc data (which would be even better for
games since webrtc data can use UDP)

~~~
Matthias247
Websockets won't disappear!

It's just the case that you can't upgrade from a HTTP connect/request to a
websocket connection, since at that point the connection is already in HTTP/2
mode and both HTTP/2 and websockets need complete control over the TCP stream.

But: You can still make a HTTP/1 request to the server and upgrade to
websockets. Both servers and clients will speak HTTP/1 in parallel to HTTP/2
for a loooong time - most likely forever.

------
laser
Just a heads up the the crystal language link is mis-routed in the readme.

------
edibleEnergy
Very cool demo. We did a demonstration of PornHub using WebSockets to bypass
ad blockers[1] a few months ago., though that's more WebSocket-fallback
development ;)

[http://blog.bugreplay.com/post/152579164219/pornhubdodgesadb...](http://blog.bugreplay.com/post/152579164219/pornhubdodgesadblockersusingwebsockets)

------
fiatjaf
Can a normal web server (a Heroku dyno, for example) with a normal websocket
library
([https://github.com/gorilla/websocket](https://github.com/gorilla/websocket),
for example) support multiple users connected at the same time?

How many in comparison to the number of normal web clients that do some XHR
call from time to time?

~~~
JangoSteve
It can. I have a demo deployed on Heroku for this app
([https://github.com/JangoSteve/websockets-
demo](https://github.com/JangoSteve/websockets-demo)), and I've seen it handle
maybe about 40 or 50 clients at a time on a single dyno before (and each
client streaming a bunch of events), even though it's intentionally
implemented in the simplest (i.e. inefficient) fashion possible. The request
overhead for normal HTTP request/response goes away, since you only need the
request headers and handshakes once at the beginning for a persistent
connection.

~~~
jasonl99
I'm trying to get this demo working on an extremely inexpensive plain VPS
(running Debian 7), and I'm getting "WebSocket connection to... failed:
Invalid frame header". Is that what you saw when you as a result of hitting
the 40-50 client limit? I am getting the error on a first connection, so it's
probably something different.

~~~
jasonl99
I'm going to answer my own question - I tracked it down to a bug in lattice-
core that occurred when it couldn't parse incoming JSON, and for some reason
it killed the socket server. I fixed it by rescue the JSON parse and all is
good now.

------
arca_vorago
I think the key point is that web sockets are a TCP connection with a http
prenogotiation. If you need UDP style connections (OK to miss state changes as
long as it catches up) WebRTC is where it is at AFAIK.

------
vr46
Looks interesting - NB: Broken link in README that should go to
[https://crystal-lang.org/](https://crystal-lang.org/)

------
nilved
WebSocket first seems to be the wrong way to go about this. How about build
your app to accept stdin and stdout, then expose this as TCP, HTTP or whatever
else?

