

Ask HN: A matter of protocols - maximveksler

Assuming one wished to sent data from his brilliantly developed mobile sdk to his still brilliant but less resource constrained backend, efficiently.<p>Would he be using capnproto or msgpack or protobuf or flat buffers or thrift or maybe cbor or rust or sbe?<p>So much awesomeness, so little love for poor old json. oh my :)<p>Which one will it be? We’re swift &#x2F; objc on the SDK frontend, go on the backend. Nothing but net in between.
======
Blackthorn
Choose what makes sense for the long-term and uses (storage, querying,
extensibility, whatnot). I see little reason to use a schemaless thing like
JSON anymore. If I was putting together a new project I'd go with capn proto.
If I was putting together a new project _and_ wanted a universal RPC stack,
I'd go with Thrift (eventually capn proto when it gets more languages w/ RPC
support). That's just my opinion though, others will vary. The important thing
to remember about your choice is that it's going to be really difficult if not
impossible to change later on down the line, so choose wisely.

~~~
maximveksler
That’s a good and very viable option to consideration, and is a factor for
when you start focusing on the direction but I think it’s unwise to being with
narrowing down the options based on such constrains as IMHO this does not lead
to the optimal solution but instead to a compromise.

// (btw, totally OT) If you’d like to know where I’ve learned about this
reasoning, kindly introduce self to [http://hpmor.com](http://hpmor.com)
(chapter 35-45, can’t remember more specific than that).

------
vendakka
Transport Layer

I'd use HTTP with SSL on port 443. The ports 443 and 80 are whitelisted by a
number of corporate firewalls and the like. I've run into problems in the past
caused by blocked ports. In addition there is a large amount of existing
software which already understands HTTP for things such as load balancing,
caching, etc.

Encoding

I would recommend using JSON to begin with. It is extremely flexible and
coupled with the compression abilities you usually find built into HTTP
systems it will also be compact. I've used Protobufs (at Google) and Thrift
(at my previous job) before, but find that the primary use case ended up being
code generation and serialization using their respective description
languages. This might seem absurd, but in my experience, since these don't
allow you to embed the schema in the message their main advantage was reduced
to having a very compact representation with some automatic schema evolution
capabilities thrown in.

JSON might lead to some additional initial typing but provides very good
performance in most languages. Storage of JSON messages can also be very
compact if you store the compressed form. In addition most JSON libraries
allow you to be backwards compatible when decoding messages (i.e adding new
fields, etc).

When compressing, gzip can be expensive in terms of CPU cycles so consider
using Snappy[1] from Google.

I've also heard both good and bad things about Avro but I haven't used it
personally.

Most people claim JSON is schema-less but this misses the point a little bit.
The JSON schema is the code you write detailing the various structs/objects
that the JSON objects map to. Merely because there isn't a specific language
used to lay out the schema does not immediately mean you cannot have a JSON
schema. There just isn't a standard so use the library that works best for
your language.

[1] [https://code.google.com/p/snappy/](https://code.google.com/p/snappy/)

~~~
maximveksler
right, forgot about Avro.

s/“^/or avro”/

------
angersock
Why not just use JSON until you've got a compelling reason not to?

~~~
maximveksler
because we will be sending lots of small data bits, in chunks and one of our
main features is 0% impact on performance. Now, clearly 0 is impossible but
infinity close to 0 is a good goal to have. JSON is textual protocol, which
carries the scheme inside every message, not the most effective use of network
data traffic.

