
Use streaming JSON to reduce latency on mobile - beau
https://instantdomainsearch.com/articles/streaming_json_jsons/
======
pkulak
I agree that a streaming response is cool, but why the dismissal of returning
valid JSON, streamed? Why invent a new protocol when JSON already exists?
Streaming JSON parsers aren't unicorns, they are horses (sorry).

With this new-line-delimited JSON format all your clients HAVE to know about
your new protocol. They have to stream the response bytes, split on new lines,
unescape new lines in the payload (how are we doing that, btw?), etc. If a
client doesn't care about streaming, it can't just sit on the response and
parse it when it's done coming in. Or, how about if later on you upgrade the
system so that the response is instant and streaming is no longer necessary?
Then you move on to a new API and have to keep supporting this old streaming-
but-not-really endpoint forever.

~~~
greglindahl
I've been using jsonlines for logfiles from about 5 years before it acquired a
name.

With line delimited logfile objects, it's easy to grep for a string of
interest and then only parse the lines that match -- much more efficient than
parsing an entire logfile to pick out 0.01% of the lines.

That's not the usecase talked about in this article, but it is a usecase
that's important to me.

~~~
beau
It's also easy to load them into Google's BigQuery:
[https://cloud.google.com/bigquery/docs/loading-
data#supporte...](https://cloud.google.com/bigquery/docs/loading-
data#supported_data_formats)

~~~
stingraycharles
I was just about to name that as an example. Newline separated JSON is a so
much simpler solution to the problem than streaming JSON parsers. They're
easier to understand, and it's trivial to implement.

From a purity point of view, sure, streaming JSON is more "correct", but I
wouldn't call it the better solution.

------
rhacker
Somewhat related is the JSON Lines "spec":
[http://jsonlines.org/](http://jsonlines.org/)

------
slig
I have nothing but praise to this service. It's fast and efficient, and does
the job without bloat.

For instance, their iOS app weighs 888.8 KB! When it's common for simple apps
to be 50 MB monsters, it's very refreshing to use something that has been
developed with proper care.

~~~
beau
Thanks!

------
zeger
You say that using websockets was less reliable than streaming HTTPS, can you
elaborate why? In my experience websockets are perfect to use for the use case
you described, are there disadvantages?

~~~
beau
I tried this a long time ago. At the time, certain queries were hard for a
server to answer. Other clients connected to the zombie server stalled until
something timed out. With HTTPS, a load balancer can direct new queries to
servers that are responsive.

~~~
thinkloop
I was also curious about this (and thanks for the article, very informative!)

Are you saying that at one point the servers would crash relatively often,
which would leave sockets clients hanging, unless some complicated client-side
code was written - whereas without sockets, a load-balancer could
automatically switch clients to functional servers, without extra coding, and
mitigate the issue? Isn't the problem the crashy servers?

~~~
beau
Sure, but why keep state when you don't need to? Now that HTTP/2 is
ubiquitous, sending new GET requests is no more expensive than sending bytes
through a WebSocket.

------
Figs
As someone who _actually_ regularly uses a slow mobile connection (8 kilobytes
per second!) with somewhat high ping (~90ms to Google), please don't do this
thinking you're making my life significantly better. It barely makes a
difference in performance once loaded, and the initial load time is
_ridiculously_ worse. I'd much rather you make your page work _without
JavaScript_ , kept the design light (as this page has otherwise done), and
make your CSS cacheable.

Right now, it takes over 5 seconds(!!) for this page to load because of all
the freaking JavaScript it has to download! With JS off, the page loads
_almost immediately_. With a keep-alive connection, subsequent loads over
HTTPS are not particularly long, unlike what this article seems to think.
(Hacker News is one of the FASTEST sites I can access, for example. Even on my
crappy connection, pages load nearly instantly.)

Simply letting me type, press enter, and wait 0.1~0.3 seconds for a new page
response would not be a significantly worse experience -- however, due to the
way the site is written, search doesn't work AT ALL with JS disabled.

So, lots of engineering effort (compared to just serving up a new page) for
little to no actual speed improvement, and a more brittle website that breaks
completely on unusual configurations... Yeah. Please don't do this!

~~~
beau
You should get the iOS app: [https://itunes.apple.com/us/app/instant-domain-
search/id1068...](https://itunes.apple.com/us/app/instant-domain-
search/id1068039352?mt=8)

It uses the streaming API, and will work well for you.

~~~
Figs
Thank you for the link, but I have Android, and I'm looking at the performance
by tethering to my PC.

------
erikrothoff
Really interesting stuff! What about simply opening a Websocket connection and
using that to for all requests if connection latency is such an issue?

------
fenwick67
I did this a few years ago before I knew it was a "thing" and felt really
proud that it actually worked.

The use-case was we had a slow database query for basically map pins. The
first ones pins come back in milliseconds, but the last ones would take
seconds. The UI was vastly improved by streaming the data instead of waiting
for it all to finish, and the server code was easy to implement.

A different delimiter would have worked, but newlines are easy to see in a
debugger.

------
tuukkah
I'd like to see this streaming JSON parser incorporated into GraphQL clients:
[http://oboejs.com](http://oboejs.com)

------
MonkeyDan
Any benefit to using this over Server-Sent Events? (other than IE/Edge
support)

------
iamd3vil
I think websockets is a much better use case for this if you don't want the
reconnection overhead. Also since websockets are bidirectional, you can keep
the connection open and send all requests through the connection as well as
receive responses from the connection. Also you can send binary on websockets
if you want to save bandwidth as well. We do this at work and it works pretty
nicely.

------
bshacklett
How is this handled from a UI perspective? As more applications are built
around the idea of streaming data, I've found that UI elements tend to jump
around, and I find myself clicking/tapping the wrong item more and more,
because the item which had been under my thumb/cursor has jumped away just
before I could activate it.

~~~
beau
This is a good point. De-bounce, static results ordering, and static elements
with placeholders help. Instant Domain Search has room to improve here.

------
Osiris
Isn't this pretty similar to how you would use WebSocket frames to transfer
individual JSON elements when a client is subscribed?

At one job I had several years ago we came up with the same idea and use \n
separated JSON elements as a streaming response. We also tossed around the
idea of using WebSockets to stream large responses between services.

------
JensRantil
Let's talk about reliability: The network is unrealiable; Firewalls might be
broken, packets are dropped, IP-addresses may change, cellphones lose
connection in subway tunnels. Simply calling "streaming reliable" without even
defining what the "reliability" is protecting again, makes "reliable" an
overstatement.

IMHO the most reliable way to get data from point A to point B is likely by
having a client actively polling for data, using a strict socket timeout. Data
should be at-least once delivered. If JSONS should be called anything remotely
"reliable" as periodically polling, at least it should have a strict timeout
(not mentioned in the article) for receiving the next newline & it should
handle replaying of non-acked messages. Otherwise I would call it far from
"reliable".

~~~
beau
Each request takes a second or two to complete. If you lose the connection,
the client can send the same request again (with exponential back off).

------
jannes
Does anyone know a JSON parser that parses an ArrayBuffer instead of strings?
[1]

JSON.parse() only accepts strings.

The library that the article recommends also uses XMLHttpRequest with strings.
[2]

The reason I'm asking is the maximum string length in 32-bit Chrome.

[1]: [https://developer.mozilla.org/en-
US/docs/Web/API/XMLHttpRequ...](https://developer.mozilla.org/en-
US/docs/Web/API/XMLHttpRequest/responseType)

[2]:
[https://github.com/eBay/jsonpipe/blob/master/lib/net/xhr.js](https://github.com/eBay/jsonpipe/blob/master/lib/net/xhr.js)

~~~
matharmin
If you want to parse more than 4GB of JSON in a browser, ArrayBuffer versus
string isn't the most important criteria - you shouldn't parse a 4GB
ArrayBuffer as a single chunk either. You can always convert between an
ArrayBuffer and a string, but you'll want to compare parsers according to the
ability to stream, as well as performance and memory efficiency.

------
sajal83
If the concern is HTTPS overhead, why not use HTTP/2 and send multiple
requests?

I think streaming would be useful only if the responses are stateful and it's
hard to share it across requests.

~~~
beau
Even with HTTP/2, sending a GET request is not free. Most of the benefit is
that we can show the user results as they come in. Each query gets over 50
responses in random order from DNS, in-memory indexes of zone files, slower
fuzzy searches over other data, and so on. Why wait to show them the .com
result while .ca resolves?

------
delaaxe
Feels to me like the response content type shouldn't be "application/json"
anymore (it is what's returned on that first example).

~~~
beau
IIRC, earlier versions of WebKit wouldn't emit data from each chunk when I
sent a custom content type. Or maybe there was some conflict with gzip, I
forget. Now that those browsers are gone, maybe ndjson? jsons?

------
wybiral
Is there an advantage to using chunked streams like this over using WebSockets
like this:
[https://github.com/wybiral/hookah](https://github.com/wybiral/hookah) (this
will take newlines from a programs stdout and send them over WebSockets, even
aggregating multiple streams into one).

------
gumby
I feel like I'm missing something here. Isn't that the point of using a JSON
SAX parser instead of a DOM parser?

------
cwt137
How is this different than SSE
[https://html.spec.whatwg.org/multipage/comms.html#server-
sen...](https://html.spec.whatwg.org/multipage/comms.html#server-sent-events)
? Or, why would someone choose this over SSE?

------
nategri
Why not just gzip the JSON? Should make complicated JSON around an order of
magnitude smaller, and be more portable to boot.

~~~
beau
I do gzip the streamed JSON. Try:

$ curl -H "Accept-Encoding: gzip" \--trace -
"[https://instantdomainsearch.com/services/vanity/apple?hash=8...](https://instantdomainsearch.com/services/vanity/apple?hash=866016287")

------
osrec
Why would you use this over web sockets?

------
stringham
Should have (2017) in the title.

