
The rise and rise of JSON (2017) - henridf
https://twobithistory.org/2017/09/21/the-rise-and-rise-of-json.html
======
sidstling
I write a lot of smaller service agents, connecting legacy systems or feeding
them with data in the public sector of Denmark. Most of those are done with
XML and it’s often quite terrible.

We write a lot of C# but one of our tools is an old adobe lifecycle server,
and .Net XML isn’t the same as Adobe XML. I mean, technically it’s just XML,
but the two techs expect your XML elements to be build a certain way and break
when they aren’t. So in order to make the two techs talk with each other
through SOAP, we had to write a custom library to translate the .Net xml
before we send it. And that’s just one in a long line of similar stories.

Hell, SOAP calls en general take longer time to get running than any JSON rest
service call I’ve ever set up. Even when the XML interprets the same on both
ends, though I’m not sure why that is exactly.

So it’s fair to say that I love JSON. The article outlines it as being easier
to read, and there is certainly that, but for me it’s the efficiency.

Working with JSON never takes longer than it should.

~~~
bouke
You’re comparing SOAP, a message protocol, with JSON, a data format. SOAP is
so much more, it specifies how objects are defined, how they should be
serialized and delivered to a server (WSDL). Note that JSON doesn’t have
anything like this that has seriously taken off. There’s a few attempts, but
they aren’t in great use or have tooling support. So as a result everyone
working with JSON implements their own de-/serializer, resulting in all sorts
of errors.

SOAP on the other hand takes care of this and even has a formalized date type
and various number types.

Your complaint is about two vendors creating incompatible
inplementations/extensions. I’ve run into this as well and causes a lot of
issues with compliant implementations that cannot work together. I think this
has been a big reason why eventually SOAP will die.

~~~
donmatito
I think parent's point was that with JSON you are, in most langages, a
JSON.parse(payload) away from a message protocol.

I am interfacing our startup's services with legacy (insurers) systems. We
don't use the same programming langage. The docs are missing. Yet, we somehow
manage to set up a functional API with minimal pain using JSON.

When I compare that, to the amount of documentation a SOAP service requires
(for another partner), I find JSON much more usable

------
asavinov
If we ignore syntactic aspects then the success of JSON is due to one
fundamental reason. JSON relies on two structural elements [] and {}, which
have very clear semantics of _sets_ (collections) and _tuples_ (combinations),
respectively. It is simple, general, natural and consistent.

In comparison, XML also provides two structural elements but they do not have
such a nice duality and clarity of interpretation. Here is a very typical
example which is more than an anti-pattern:

    
    
        <books> 
            <book id="1"><title>XML is evil</title></book> 
            <book id="2"><title>XML considered harmful</title></book> 
        </books> 
    

Here we see that semantically <books> is a collection while <book> is a tuple
(where <title> is an attribute). Yet, they use one and the same structural
construct. XML does not enforce the dual semantics of sets and tuples, and
therefore XML files are frequently a conceptual mess. The above example is
essentially _wrong_ but XML does not care.

~~~
WorkLifeBalance
XML without schema are a mess.

XML with schema and validtion can be a joy to use compared to the untyped JSON
world, but it's the lack of having to specify type which helps JSON remain
popular.

Given a choice between strong or weak typing and weakly typed or un-typed
systems will prevail in popularity.

Looking at javascript itself and the type coercion rules it is easy to dismiss
it as unworkable mess which leads to expensive bugs in production, but in the
real world the ability to mostly ignore types lets people get things done in a
more hackish / amatuer way which eases the learning curve and helps
popularity.

~~~
asavinov
Typing (enforcing constraints) is an important aspect. But even without typing
XML has one fundamental flaw. You are not able to (correctly) represent tuples
with attributes which are sets. In XML, tuple attributes are properties, for
example:

    
    
        <book id="1" title="XML"> -- object or object
    

Now let us assume that I want to have a list of authors as a property:

    
    
        <book id="1" title="XML" authors=["Me", "My Friend"]> -- NOT SUPPORTED
    

Therefore, we use a _workaround_ (a crime actually):

    
    
        <book id="1" title="XML"> 
            <author>Me</author>
            <author>My friend</author>
        </book> 
    

Now a book is a collection where authors are members (IS-IN relationship).
This is not what we wanted. Our goal was to have an attribute "authors" which
is a collection.

~~~
dragonwriter
Pretty much any time I've seen something like this, relevant inner element of
the book is:

    
    
      <authors>
        <author>...
        <author>...
      </authors>

~~~
asavinov
It is again a workaround because <authors> is a member of a collection - it is
not a tuple attribute. Which suggests that we cannot enforce this separation
and an application has to understand itself which element is an object
property and which element is a member of a collection.

~~~
Mikhail_Edoshin
You're trying to impose an arbitrary model on XML.

XML grew from markup languages. It's as if we took some text, parsed it, and
then store the resulting syntax tree along with the text. Here elements are
non-terminals of the grammar and element attributes are additional augmenting
properties of those non-terminals.

The _text_ , of course, does not have to human-readable, it can very well be a
sequence of anything, for example, of bits, bytes, words, etc. In XML we'd
have to represent these with special elements, something like <byte value="00"
/> or <int32 value="12345" />, but once we do this, we can use XML to enclose
them into additional non-terminals that tell us what these bytes mean and let
us use automated tools to manipulate them.

XML can represent objects, but in its own way: we have to first sort of
_serialize_ our object into a sequence and once we have this sequence, we can
use XML. The model you seem to be talking about an abstract model of abstract
objects _in memory_. Although memory is technically sequential, we normally
ignore this and treat objects as nodes in some graph. This is not the domain
of XML; it has to be a sequence to begin with. XML has a concept of IDs and
references to IDs and thus can represent graphs reasonably well, but it must
be a _serialized_ graph.

So XML is basically a language to express the underlying grammatical structure
of an arbitrary sequence. That structure is a tree, but it is based on a
sequence nonetheless. It's not quite what abstract objects are in programming;
but if you think of files, for example, files are sequences and thus are
totally the domain of XML.

------
niftich
JSON maps well and intuitively to common datatypes: with a list and a hashmap,
you can model a lot. Conversely, structs needing to be passed around in
application code will probably be made up of stuff that maps well to JSON.
XML's greater-than-blindingly-obvious impedance mismatch is a barrier to entry
when an alternative, dead simple format exists.

The XML ecosystem nonetheless has parts that are desirable for certain use-
cases, as the reinvention of schema validation for JSON, awkward namespacing,
and efforts to dabble in hypermedia linking show. The XML ecosystem didn't
require these enhancements and extensions to be used, but implementations in
the wild often did, which was further off-putting to newcomers. This helped
JSON gain ground back when it (and the ecosystem around it) was truly minimal
and simple.

It also embodied the hacker, kludgey ethos and mythos, because without
elaborate schemas and code generators, you'd pluck out only the fields you
cared about and disregard the rest.

These factors led to JSON's rise at an opportune time, giving it credence and
exposure, and within a few years every hip cool API was emitting JSON and
using cool methods like PUT and DELETE and throwing around terms like REST
despite serving linkless structs as 'application/json'.

The prevalence and staying power of CRUD HTTP APIs exchanging loosely typed
JSON payloads shows that there's real benefits to semi-human-readable, self-
documenting payloads that can be parsed into either rigid native datatypes, or
loose collection types with ease. JSON fits into a world where stuff can be
deliberately or inadvertently half-assed and still work.

------
Schwolop
This article had a footnote linking to a snide Douglas Crockford retort to a
blog post that was dismissive of JSON. The other discussion comments on that
page are a fascinating history of the arguments for and against, through the
lens of 2006.

[https://scripting.wordpress.com/2006/12/20/scripting-news-
fo...](https://scripting.wordpress.com/2006/12/20/scripting-news-
for-12202006/#comment-26383)

~~~
joatmon-snoo
Whoa, thanks for linking that. It's really interesting seeing all this talk
about sandbox escapes.

This is a particularly interesting quote:

> One huge practical advantage to JSON over XML in the web browser is that you
> can load JavaScript from any site, while you’re restricted in which sites
> you can load XML from. I have no idea why browsers insist on limiting where
> you can load non-executable XML from, when they don’t bother to limit where
> you can load executable JavaScript from, but there you go: that’s how it is,
> and there’s nothing anybody can do about it. JSON nicely works around that
> limitation, instead of holding its breath and waiting until the world comes
> around to being fair. Surely you can appreciate that practical advantage.

------
DonHopkins
>Though one might think that a contest between data interchange formats would
be unlikely to engender death threats, Winer wrote:

>"No doubt I can write a routine to parse [JSON], but look at how deep they
went to re-invent, XML itself wasn’t good enough for them, for some reason
(I’d love to hear the reason). Who did this travesty? Let’s find a tree and
string them up. Now."

In retrospect, he should have said: "Let's find a tree and stringify it."

------
denormalfloat
The success of JSON is due to one thing: Browser support. As soon as it's use
on a server, or a mobile device, or practically anywhere else almost all the
benefits of JSON go away.

Can you inspect the payload easily? Well, yeah, if you unwrap the bytes into a
string or open up Wireshark. But on any moderately optimized system, the JSON
is going to be gzip encoded and whitespace compressed, making in quite
unpleasant to look through. By that point a pretty printer is needed, and you
might as well have picked any format that has a pretty printer.

Can you add new fields easily? Yeah, but try changing a field name, or its
type, or remove a field. This is guaranteed to break some client or server
somewhere eventually. I've seen code that uses at least 5 permutations of
"Zipcode", "zipcode", "zip_code", etc. I also wouldn't dare try to change it
for fear of accidentally breaking our payments processing system.

JSON trades ease of starting up, for suffering when the code base gets big.
It's an incredibly alluring trap, as TFA's graph shows.

~~~
humbleMouse
Why are you using wireshark to inspect json payloads?

Your other complaints stem from poor system design and lack of shared data
contracts. They could be complaints about any data format.

~~~
foota
I would argue that the fact that json doesn't have schema support built in
makes it more difficult to handle those issues though.

~~~
humbleMouse
Avro json schemas and jackson json annotation for POJOS makes json object
schemas easy to implement. Versioned object repos make great shared schemas
and are easier to manage than xsds. (imo)

~~~
lmm
Anything would be better than XSD (though I don't think JVM-only is a good
idea), but with an object you still don't necessarily know what kind of
changes are going to be forward compatible or not. The only thing I've had any
success with is something like Thrift or Protocol Buffers, where the canonical
definition of your objects is a terse, Algol-like syntax but one that's been
designed explicitly with version compatibility in mind, and you know exactly
what changes do or don't break compatibility.

------
ToFab123
So, when do we get a typescript for JSON? You mean a schema like XML has?

It looks like typesafe programming languages are on the rise, yet, when it
comes to the format used for data exchange we go from typesafe to
unstructured. I find that odd.

~~~
ubernostrum
The counterpoint here is that a lot of the success of JSON was in presenting
an alternative to the then-state-of-the-art, which was basically "here's a
small mountain of spec documents explaining how you need at least fifty
megabytes of metadata spread out over forty-three separate namespaces' worth
of elements to safely send a single string key to someone else with XML".

And at the time people insisted JSON could never replace that, because JSON
was so underspecified and unsafe. You could _maybe_ get away with it for a
couple requests to your puny baby child's "Fisher-Price My First Webservice"
toy program, but JSON would instantly and completely fall over the instant any
real-world problem presented itself.

Yet here we are all those years later, in a world which largely runs on JSON
as a data interchange format.

Or, more concisely, as I wrote in 2006 in response to the Dave Winer piece
someone else linked in another comment:

 _And now here are these kids with their startup companies and their weblogs
who are getting data exchange and even things that kind of look like APIs out
of… JavaScript arrays? The XML guys are sitting up on the mountaintop like the
Grinch, with his pile of stolen presents, wondering how Christmas still
managed to happen: it came without specs! It came without hype! It came
without angle brackets, envelopes or types!_

~~~
donmatito
Or, "worse is better"

~~~
acdha
Worse is highly subjective: the XML world spent millions of dollars layering
on huge amounts of complexity but never made a corresponding investment in
having common tools not suck. Things like validation and schemas still have a
horrible user experience and then you have things like libxml2 meaning that
most of the world can’t use XSLT or XPath features added a decade ago.
Similarly, it’s been notoriously hard to find things in the XML world which
have quality documentation, examples, or working validators — doubly
unfortunate given the complexity of what they expect users to do.

If you have a job to do, JSON is just plain better on almost every count. I
remember being excited by XML 1.0 back in the 90s but the fall in popularity
is something lots of people worked hard to accomplish by building so many new,
more complex things rather than fixing the foundation. The assumption that
users will have to learn your standard is almost always wrong.

------
pingec
I use XSD a lot for XML validation in .net world and better intellisense when
writing XML by hand. I was exploring the json world but I could not find a
json schema technology that would be as expressive and the tooling around it
is much inferior IME.

I feel like the technology around schemas (json, xml or anything else) is
underrated and not developed enough. And if I wanted to support both json and
xml it would be such a pain to maintain, would be nice to have a tool that
could translate xml schema to json schema and back.

~~~
senozhatsky
> would be nice to have a tool that could translate xml schema to json schema
> and back.

Hmm, after a quick search I found a bunch of json-to-xml and xml-to-json
converters, including libraries. E.g. JSON-Java library [0]. May be same
things exist for .Net. Is this what you meant?

[0] [https://github.com/stleary/JSON-java](https://github.com/stleary/JSON-
java)

-ss

~~~
bouke
I think parent was talking about the document definition (schema) instead. So
not to rely on the document instance for the translation definition.

~~~
senozhatsky
Ah, got it. Thanks. I wasn't entirely sure, admittely not a very experienced
json user. Does term "schema" apply to json in the same way it applies to XML?

/* Well... so... One can convert JSON to XML and apply XML schema
verification. That's partly a joke :) */

-ss

------
nebulous1
I think the the only people who could be confused as to the reason for the
rise of JSON are people who have never used XML

------
vertere
> _By being at the intersection, [JSON] turns out to be the thing that
> everybody can agree on, and so it 's really easy to pass data back and
> forth. Prior data interchange formats tended to try to be the union of all
> the languages, and that turns out to be horrendously complex and really
> difficult to deal with. JSON by being so simple actually became really easy
> to use._

\- Douglas Crockford ('discoverer' of JSON)

[https://youtu.be/-C-JoyNuQJs?t=730](https://youtu.be/-C-JoyNuQJs?t=730)

~~~
sametmax
What amazes me is that I regularly train people in IT that have never used
JSON themself and are not sure what's it good for.

So something that is quite everywhere now, is simple, easy, and has existed
for a long time, is still far from being a given.

It definitely puts in perspective people that are telling me that my webpack
configuration from last year is obsolete, that I should ditch everything and
learn Elm, that nix is the future of package management or that webassembly is
going to change everything next week.

------
saagarjha
Of course, JSON has its own share of issues too, largely as a result of it
trying to be as simple as possible. Many JSON parsers support nonstandard
features such as comments and trailing commas, and in the process, create
mutually incompatible "dialects". XML might be verbose and complicated, but I
think JSON unfortunately is a bit _too_ far in the other direction.

~~~
function_seven
If a JSON parser ignores the JSON spec, is it the spec that went too far in
the other direction? Or is that parser just badly-written? (Or misguidedly
lenient?)

You could say the same for an XML parser that allows a naked '&' between tags,
or continues parsing a document that omits a closing tag somewhere.

Your complaint is probably still valid, but I'd say it's better directed at
the culture than the specs.

------
cool-RR
" _JSON’s dominance is surprising when you consider that as recently as 2005
the web world was salivating over the potential of “Asynchronous JavaScript
and XML” and not “Asynchronous JavaScript and JSON.” "_

"As recently as 2005," heh. In the frontend world, you might as well say "as
recently as the Paleolithic age."

------
hguhghuff
It’s a great thing that JSON adequate.

It’s a great pity that JSON isn't much better.

It would be great if JSON had comments, more thorough typing and determinism
so that data structures could be compared. Apart from that it’s streets ahead
of XML.

~~~
wetpaws
It's exactly as good as it needs to be. Trying to 'improve' JSON will turn it
into xml again.

~~~
tlocke
Surely it's a problem that in JSON, duplicate keys are allowed?

[https://stackoverflow.com/questions/21832701/does-json-
synta...](https://stackoverflow.com/questions/21832701/does-json-syntax-allow-
duplicate-keys-in-an-object)

------
ktpsns
One aspect about JSON is the lack of comments. This makes it mostly unsuitable
as a configuration language and in fact simple "JSON for humans" such as
[https://hjson.org/](https://hjson.org/) and
[https://json5.org/](https://json5.org/) exist. In my opinion, this is a
design flaw and if JSON would be a bit more human-oriented, it could be much
more widespread in non-web contexts.

~~~
fenwick67
I actually like that JSON has no comments.

With comments in a data transmission format you will end up with things like
"conditional comments" in IE.

I agree there are certainly many better configuration languages.

~~~
ktpsns
I agree, but you can do context switchs also without having comments. For
instance, the JSON Schema defines (Source: [https://json-
schema.org/latest/json-schema-core.html#rfc.sec...](https://json-
schema.org/latest/json-schema-core.html#rfc.section.9))

    
    
      {
         '$comment': "string that can hold whatever you want"
      }
    

and it is obvious that you can embed any other language in a string.

------
blauditore
It's funny how people often compare JSON and XML, but don't mention one huge
difference: XML is "more powerful" because it supports attributes on nodes,
while JSON doesn't. So if you want to encode data structure with metadata on
nodes (like objects with a type), you'll have to introduce a non-standard
workaround.

Of course that's often not an issue, but I've run into it in the past. :)

~~~
ubernostrum
[http://www.flightlab.com/~joe/sgml/faq-
not.txt](http://www.flightlab.com/~joe/sgml/faq-not.txt)

 _Q. I 'm designing my first DTD. Should I use elements or attributes to store
data?_

 _A. Of course. What else would you use?_

 _Q. What is the usual way to choose between using attributes and children?_

 _[Update from David Carlisle:]_

 _In England, the normal method is to flip a coin in the air and call heads it
's attributes, tails it's children._

 _Unfortunately I 'm not sure what to suggest in other countries where you
can't guarantee having the Queen's head on one side of every coin._

~~~
tannhaeuser
Hilarious, but the deeper rule is: when there isn't a concept of rendering a
document to the user (where attributes hold rendering properties not directly
presented to a user) the concept of a markup language doesn't make sense in
the first place. So if you find yourself in a situation where you need to
decide on encoding data as elements vs attributes, you probably shouldn't use
SGML/XML at all.

But then there is the wealth of XML standard data representations for
financial/fiscal/e-commerce, medical, etc. data. You're deciding to use them
to have your interfaces being implemented once and for all, and be able to
switch eg. payment providers by merely specifying new endpoints and
credentials. JSON doesn't really compare to XML/XSD in this space.

------
lkuty
XHTML is still alive as the section 13 "The XML syntax" in the "HTML Living
Standard" shows us. Moreover XML comes with a host of useful technologies :
schemas, XSLT, XQuery and namespaces. It is quite complex compared to JSON but
more powerful when you consider the whole ecosystem. In the end, it depends on
the need we have. Quick, small, simple XML is possible and easy.

------
amaccuish
I feel like the best takeaway here is that XML is a document format, and JSON
is for programmatic interchange, e.g. APIs. That's at least how I operate, and
I use both XML and JSON for various things. My APIs are all JSON, but if I'm
serialising some data to disk, or for export etc, it's usually XML.

------
Zamicol
JSON is one of the best things to ever come out of the CS disciplines.

It is one of the most perfect technologies I deal with.

~~~
gaius
Well, unless you need to deal with datetimes or timestamps. And it suffers
from the same repeated-metadata problem that XML does. It’s easy for
JavaScript users, that’s it’s main purpose. For serialised data either at rest
or on the wire there are dozens of better formats e.g. HDF5. Or even ASN.1.

~~~
majewsky
> It’s easy for JavaScript users, that’s it’s main purpose.

I'm writing Go applications that talk to Python applications with (you guessed
it) JSON-over-HTTP. Saying that JSON is only for JS users is like saying the
iPhone is only for people that need to make calls.

> For serialised data either at rest or on the wire there are dozens of better
> formats e.g. HDF5. Or even ASN.1.

A huge advantage of JSON is that it is human-readable and human-writable
without extra tooling (although an indenter sure makes things easier on the
eye). Also, I recall ASN.1 parsers being a rich well of RCE bugs.

------
cpr
Not: as groundbreaking as Scribe (Brian Reid at CMU) was, I don’t think SGML
was inspired by it. They had parallel dev paths...

------
zde
A message format that requires parser to understand both UTF8 and UTF16, cool!

~~~
pitaj
Unsurprisingly, one of the most common data types is text strings, so most
serializations need that anyways.

------
another-cuppa
> only one API exposes data in XML rather than JSON

They are not mutually exclusive. It should be trivial to offer the data as
either JSON or XML at the user's request.

