
Show HN: JSON5 — modern JSON - aseemk
https://github.com/aseemk/json5
======
enneff
JSON is wonderful because it is simple and unambiguous. It just works, and
serialized data will always look the same.

This "JSON5" thing is an abomination. Use whatever quotes you like! More
string escaping rules! Adding these things would make JSON worse, not better.

Here's another idea: why not write a tool that tidies up your hand-written
JSON files? You could even re-use your "JSON5" parser to do it. That way you
can be as lazy as you like when writing files, but not expect the rest of the
world to learn your new dialect.

~~~
aseemk
Thanks for the feedback. It certainly would be great if there existed tools to
let me write and update JSON with nicer syntax like this; any ideas how that
might be possible?

My motivation for this was actually hoping that this gets adopted in some
environments that use JSON. Node relies on package.json files, for example,
which frequently need to be hand-edited. Maintaining a sibling file in a
different format feels equally wrong to me as it seems JSON5 seems to you. =)

~~~
benatkin
That's npm, not node itself, that relies on package.json. When its creator
chose it, he knew exactly what JSON is. He's maintained on HN and elsewhere
that he made a good decision. So even if you stop calling this JSON, he's not
buying it.

The package.json files, by and large, don't have a multiline string problem.
For trailing and missing commas, I think you could make something that
automatically fixes them, if that's the only problem, and run it before npm is
run, perhaps using a shell alias.

~~~
aseemk
You seem to be implying that making an effort to seek change in the format
that npm uses is a bad idea. I'm not sure I understand why that is. Surely the
status quo isn't always perfect?

In the software world in general, and in Node/npm land especially, code talks.
I thought it'd be more productive to build and share a working ES5-style JSON
parser than to simply ask Isaac if npm would support ES5-style JSON. =)

Yes, you're right about trailing commas. One motivation that such an approach
couldn't address is documentation: I've frequently wanted comments in my
package.json files, e.g. explaining a particular dependency version, or a
script command.

~~~
ZenPsycho

      { 
        "name": "My fancy package",
        "comment": "I include both versions here for compatibility with package x",
        "scripts": ["script1.js","script2.js"],
        "comment":"to run the build script",
        "comment":"cd to the bin directory and type:",
        "comment":"node build.js",
        "build": ["bin/build.js"]
      }

or

    
    
      {
        "name": "My fancy package",
        "--": [
          "I include both versions here for compatibility with package x",
          "to run the build script",
          "cd to the bin directory and type:",
          "node build.js"
        ],
        "scripts": [
          "script1.js",
          "script2.js"
        ],
        "build": [
          "bin/build.js"
        ]
      }

------
simanyay
Three things:

* Comments in JSON files are not a great idea because people might (will?) abuse them to add special instructions for their own serializers. As soon as this happens we'll have a format that is not compatible with other serializers.

* JSON today is a universal format. So, what are special characters? Should '1hello' be quoted? If no, then its not a subset of JavaScript anymore. If you're writing a new specification you have to define all those things. Same with multiline strings: space after backslash?

* Why? I know that hand-writing JSON can be annoying but is this really the biggest problem we have?

~~~
aseemk
Comments: yes, I read Douglas Crockford's post on the matter. I guess I'm not
genuinely worried about that, though, and my hope is that shipping a JS parser
that works on both the client and the server will help standardize
implementations from the start.

Special characters: sorry I didn't write a formal spec =), but the intent is
to be a pure subset of ES5 just like regular JSON is a pure subset of ES3.
Unquoted object keys in ES5 can contain only letters/numbers/_/$, and only
begin with letters/_/$. So yes, '1hello' would need to be quoted, just like
regular ES5.

I'm not sure how to respond to your why other than what I've already written.
I never claimed this was the biggest problem we have. =)

~~~
Too
JSON is not exactly tied to javascript. For example the number-type is not
specified in much detail. You could imagine this to be a 64bit int but if you
load it into javascript it will be converted to a double and you will loose
precision if your number is big.

------
YmMot
I won't begrudge someone a fun little personal project, however (almost) no-
one will ever adopt this. It removes too many benefits of JSON/introduces
drawbacks for very little pay off.

Yes, JSON is a pain to edit/write by hand at times, but that's an editing
problem and therefore should be solved in the editor. In any halfway decent
editor it should be trivial to extend it with helpers for editing JSON.
Perhaps have it convert .json files to YAML(etc) on opening, and back to .json
on save etc.

This is a problem that plagues many technical people (present company
included). You had a good instinct, but then took a left turn somewhere. "JSON
is hard to hand edit...let's undo all the speed/compatibility progress we've
made with JSON in the last ~8 years and introduce flaws in the protocol with a
new non-standardized format...all so it's superficially easier to edit by
hand."

~~~
aseemk
I'd love to understand what benefits this removes and what drawbacks this
introduces, aside from the chicken-and-egg problem of having any new format.
Apologies if I've missed that in the discussion so far.

You're definitely right that some of the hand-editing problems could in theory
be solved by tools, but it unfortunately doesn't address
documentation/comments.

I've heard the chorus on YAML though, and I'll definitely be looking into it.
Thanks for the feedback!

~~~
YmMot
JSON is supported natively (i.e. fast and correctly) in all modern JS
environments. Yes, at one point we used Crock's parser but that's the past.
Likewise, it currently is only used as a stop-gap measure to begrudgingly
support older browsers.

If we're going to undo years of progress we should do it for a _very_ good
reason.

JSON has wide support in many languages. Some of them have fast low level
parsers. All that work has to be duplicated. Again, this just doesn't seem
sufficient reason.

Right now if I want to throw together a Python script to crawl a bunch of NPM
packages and read their package.json (let's say to put together a chart or
collect statistics), I can easily.

If NPM switches to JSON5, we lose compatibility without a lot of duplicated
effort. Again, there's just not enough benefits.

I consider comments a flaw, because they add little benefit. JSON is a data
serialization/interchange format and there shouldn't be much need to comment
on it, it should be largely self explanatory. If you really need to tag things
with info, you can in JSON. Adding comments is to tempting for people to
abuse.

However, if you really need lots of comments, there are many existing formats
which might be better suited for your task. JSON is ultimately for machines.
It has a benefit that humans can read/edit it easily...but if you have data
that is intended more for humans and needs lots of meta information like a
complicated config file, you should either create a front end for it or use a
format better suited to that purpose.

When you consider that writing a plugin for your editor, or just a stand-alone
translator would solve all of the problems this does, there's no reason to put
up with all the drawbacks.

What it comes down to IMHO, is this isn't suitable to replace JSON and for
other uses there are already existing formats that are better.

~~~
DavidAbrams
You didn't answer his question, except in regard to comments. Everything else
you said is based simply on the non-standard nature of this dialect.

~~~
YmMot
That's the issue though. It doesn't introduce anything useful in terms of
replacing JSON (it can even make things worse), and for other uses better
solutions already exist.

The fact that in theory if it was magically supported everywhere, it might be
serviceable is neither her nor there. It's not useful information. It's not
gonna happen because there's no reason to switch from JSON to this, and many
reasons not to. Likewise, for other applications there are better solutions.

People started using JSON even though at that point it wasn't standard because
it had real benefits over what they were currently using. There isn't any
situation where using this would provide a real benefit except to satisfy
personal whims, and it would have the drawbacks listed.

------
greghinch
(young) people seem to always want to find ways to skirt the "rules" that
Crockford sets up (see also, jslint). Just stop. The rules he lays out are
there to make your _long term_ programing life easier. Yes you may have to
remember to use semicolons and quote things, but when you have to live with an
application you've been working on for years, hopefully you'll learn to
appreciate the fact that putting in a little extra time now saves you tons of
headaches down the road.

~~~
tomjen3
When was Crockford elected to set the rules for programmers and why wasn't I
given a vote?

Oh, he was never elected? Why the fuck should I care about what one man thinks
then?

~~~
mryan
You should try relaxing a bit more, this seems like an incredibly silly thing
to get worked up about. There are plenty of legitimate things in the world to
get angry about - object serialisers are not one of them.

IMHO, aggressive comments that contain nothing but sarcasm are not welcome
here. Let's try and keep the place a bit friendly (or at least grown-up and
professional), even if we disagree. If you are going to be sarcastic, at least
have the decency to make an interesting point at the same time.

> When was Crockford elected to set the rules for programmers and why wasn't I
> given a vote?

Crockford was obviously not "elected" in any official sense. However, he
produced something that the programming language needed, and it very quickly
became a standard. Have you ever used JSON in a project? Then you implicitly
voted for JSON and, by extension, Crockford.

> Oh, he was never elected? Why the fuck should I care about what one man
> thinks then?

Because the rest of the industry has decided that JSON's benefits outweigh its
downsides, and using it is a net positive. JSON is not perfect, but it solves
an annoying problem rather elegantly.

------
Mizza
Please, don't call it JSON. There is only one JSON, and you didn't make it.
It's cool that you made this, and that you made it open source, just please
call it something else.

------
mitchellh
While we're at it, I felt it necessary to launch HTML7:
<https://github.com/mitchellh/html7>

~~~
aseemk
Haha, I actually found that hilarious. Nice work.

------
zacharyvoase
Can I ask why YAML wasn't good enough?

~~~
erikpukinskis
Partly because JSON has the lovely feature of being able to just eval() it
into an object in browsers.

~~~
alcuadrado
That's not the right way to parse JSON, it's really insecure

~~~
Too
And slow, JSON.parse is magnitudes faster than using user-javascript to parse
it yourself.

------
aseemk
Author here.

This idea stemmed from long-running frustration every time I hand-wrote JSON
(particularly common for Node/npm packages and test cases). After stewing on
it for over a year, I bit the bullet today and made it a reality.

Feedback greatly welcome!

~~~
brentashley
Douglas built the JSON spec

1) to be language agnostic. Following ES5's syntax directions is fine for ES5
focused development, possibly not for others.

2) to be final - i.e. there is no version number. If you want to extend or
build something else, I am fairly certain Douglas would prefer that you call
it something else. See <http://inkdroid.org/journal/2012/04/30/lessons-of-
json/> at 5:30 in the video.

Don't take these points as criticisms, I too would like to be able to use
single quotes and trailing commas and everything else here. I think though
that in order to work with JSON as a static spec, an alternative approach
could be to have your project be a layer like HAML, SASS or CoffeeScript. Each
of those tools compiles or transforms to conforming input for their target
specs rather than replacing them.

~~~
aseemk
JSON5 is also meant to be language-agnostic. It derives its syntax from ES5 in
the same way that JSON derived its syntax from ES3.

I'm aware that JSON is final — it's my hope that a new format that's easier to
write (whether this or something else) picks up steam — but thanks for the
feedback on the name! I consciously used a different file extension (.json5)
to avoid conflicts; hopefully that's a good start.

I already use (and love) CoffeeScript, but my motivation for building this was
to have tools use it natively, so that sibling files in a different format
wouldn't have to be maintained alongside the needed JSON.

------
augustflanagan
I think that it's really a shame to see people slamming aseemk here. He had an
idea, worked on it, and released it. That's fantastic. There are a few
constructive comments here explaining why his idea might not actually be that
great, but there are also quite few comments that seem out of place for HN.

------
jimrandomh
You know that something has gone very deeply wrong somewhere when people
oppose your project because they are ideologically opposed to comments. In my
own experience, finding out that many JSON parsers reject comments was a real
WTF moment, and a deal breaker for my config-file application, so I ended up
using JSON-plus-comments for it instead of JSON. At the same time, lack of
support for trailing commas and unquoted member names have been a minor but
persistent thorn in my side for no good reason.

The justification for not having comments in JSON is that in the great
disaster that was XML, some projects would parse the comments and take them as
semantically significant. However, the real problem there was that parser
libraries would expose the comments, and that some generators would put
important information only in comments. But I think that these mistakes are
unlikely to be repeated, and that the proposed alternative - moving all
comments into the markup, or eliminating them entirely - is just obviously
worse.

One of the things that ruined the XML ecosystem was a persistent belief that
XML was to be read and written by machines, combined with a reality in which
it was mostly used for human-written config files, leading to a tolerance for
awful syntax (like prefixing every single attribute with a namespace, and the
ridiculous CDATA notation). I'm seeing the same thing with JSON: A significant
fraction of its use is for human-written and human-read config files (which
often desperately need comments) and people are pretending it's strictly a
data interchange format that shouldn't be used for that.

It is sometimes said that JSON was discovered, rather than invented - that the
syntax was already out there. So it is with JSON5: There is nothing _new_ in
this, it is simply return to JavaScript Object Notation and bringing in the
rest of what Javascript has.

So, all you pooh-poohing ideologists: please seriously rethink whether
disallowing comments, trailing commas, and quoteless member names is actually
a good idea. Consider this in light of the fact that JSON is widely used,
today, as a config-file language, and that {"--":"This is a comment"} is
ridiculous enough that no one does it in practice, can't be inserted on any
line, and invites consumers of your data to parse the comments.

~~~
aseemk
Thanks for stating so eloquently what I've been struggling to communicate. The
heart of the matter to me really does boil down to the fact that JSON is meant
for humans, too, not just machines. You nailed this sentiment.

(As a friend pointed out, "Do you not comment your code just because it's
meant for machines?")

------
jahewson
This reminds me of I guy I met who was trying to build a clone of Twitter, but
with messages of unlimited length. In his view this made it objectively
"better". He understood the _what_ of the software, but not the _why_.

If you really want ES5 for writing package.json files, then you could save the
time and effort by just using ES5:

    
    
        var _ = 
    
        { 
          some: 'object literal',
          // a comment
          num: 3,
        }
       
        fs.writeFileSync(path.basename(__filename, '.js') + '.json', JSON.stringify(_))

------
drawkbox
JSON is great because it is simple basic cs types (string, numeric, boolean,
object, list, date), a perfect middle ground between messaging and readability
without going all XML/SOAP. And the unchanging simplicity is that way because
it is so broadly accepted from Javascript to web services and beyond. It fits
nicely into the system because of it's limitations but that also gives it
standardized power.

Any changes to JSON should be fought til death. Use YAML or make a new format
if you need more.

~~~
aseemk
Certainly. I'm not advocating for a change to JSON -- this is explicitly a new
format. (The test cases use a .json5 extension to illustrate this.)

FWIW, I'm not adding any new data types or functionality, just making the
syntax more human-friendly. It also continues to be a strict subset of
ECMAScript, but v5 now.

~~~
drawkbox
Yes I was just talking in a general sense that JSON should not go all XML and
mutate. In some cases I have added layers like this for instance in a C++
parser for game engines it allowed comments in the json but were stripped out
at runtime on load and on send/receive from the services. But conforming to
JSON that current browsers, libraries etc can use.

I see where you were going with it, ES5 support as a baseline is nice, also
comments are nice but also bulky to a streamlined messaging format in most
cases (config is usually yaml or some other format for this reason maybe).

I guess the naming is what maybe might cause some confusion and a good
engineer makes things more simple not more complex, the simplicity of JSON is
hard to achieve with such success but it is also its killer feature so that
should be recognized. As long as we don't have to support 50 different 'modern
JSON' formats then we are good. There are some good ones like BSON for binary
JSON. I guess JSON5 works for your ES5 support.

I think here you have good reasons for your work but it goes back to the age
old be 'conservative in what you send, liberal in what you accept'. That
standards agreement makes everything work and to me being an engineer is
taking complexity and simplifying always, bloat is bad. Here it isn't that but
XML started out simple as well and became a verbose, bloated hell so people
get touchy with JSON hehe.

------
pygorex
I have to agree with most of the criticism in this thread. The lack of
comments, strict quoting, no trailing slashes - these aren't problems to fix
they are features.

As others have noted this is not JSON and is not necessarily an improvement to
JSON - so calling it JSON5 is presumptuous.

Embrace the criticism. Instead of JSON5 call it __NotJSON __\- a standard that
breaks the JSON rules for a few narrow use cases - namely when you want a
hand-written, commented JSON-like syntax.

------
kwamenum86
I don't think any of these suggestions make JSON easier to read, generate, or
consume. What problem are you setting out to solve?

Opening paragraph: "JSON is strict. Keys need to be quoted; strings can only
be double-quoted; objects and arrays can't have trailing commas; and comments
aren't allowed."

If this represents a comprehensive list of the problems JSON5 solves I think
you'll have a hard time rallying people around your cause.

~~~
aseemk
Yes, I realize I didn't explain it as well in the README as I had in
conversations and emails.

I find JSON tedious and error-prone to generate by hand. I also frequently
wish I could document the data with comments.

Others have written about these same ideas, e.g. [1]; after feeling the
frustration for over a year, I decided it might be productive to actually
build a JS parser to get the conversation started.

My main use cases are Node/npm package.json files, and test case data.

[1] <http://bolinfest.com/essays/json.html>

~~~
ninjin
> I find JSON tedious and error-prone to generate by hand. I also frequently
> wish I could document the data with comments.

I whole-heartedly agree with you on the tediousness, but JSON has this dubious
double-nature that it is sometimes written by hand but mostly used as a
protocol. As a protocol we want (and I would even argue need) this strictness
since it makes day-to-day operations easier and parsers generally more
reliable (how many times have we seen someone implement their own JSON parser?
Now, what about XML? The first is a nice evening the latter is, well, not...).

As for package.json, I would argue the problem is their choice of
serialisation rather than with JSON itself. Right tool for the wrong job. YAML
would probably be a better pick.

------
genericdave
Seems like you're getting a lot of criticism for this, a lot of which I have
to agree with. Writing valid JSON is extremely simple and unfortunately it's
hard to see how this project would do anything besides promote bad practices
for a near-universal format.

Suggestion: turn these criticisms to your favor. Make this into something that
_promotes_ good form. Perhaps by acting as a compiler that transforms your
forgiving JSON into valid JSON. Make a command line utility. Emphasize how the
output passes JSONlint or something. In other words, I'm suggesting that you
remake this into a sort of coffeescript for JSON.

In any case, the path you are on isn't gonna garner much support. Change up
your stride a bit and you could be on to something rather nice, though.

------
cheeaun
JSON 3 here: <https://github.com/bestiejs/json3>

~~~
nfriedly
I believe Facebook recently switched to JSON 3 for their JS SDK.

------
MarkR42
Congratulations, you've made something which looks superficially like JSON,
but most documents won't parse in 99.9% of JSON parsers. Interoperability is
lost, nothing is gained!

------
iamcarbon
If only it were this easy to extended JavaScript with a new serialization
format! If the serialized text can't be assigned to a JavaScript variable,
it's not JavaScript notion.

I also tried something similar a few years back when I was sick of dealing
with XML and coming to terms with the required quotes.

<http://xjson.org/>

While it's certainly prettier to have native literals and optional quotes,
it's simply not worth giving up the interoperability with JavaScript.

------
tferris
Please rename your project or take it down.

People here gave you enough reasons.

~~~
veemjeem
don't worry, next week I'll release JSON6 and nobody will remember JSON5
anymore.

------
jwr
I don't think making JSON more human-writable and readable is a goal worth
pursuing.

JSON has its flaws, but is a surprisingly robust and elegant solution to many
data interchange problems. But it is not (and I don't think it was ever
intended to be) a language for human-edited configuration and data.

------
timrobinson
> [IDEA: Allow octal and hexadecimal numbers.]

Who actually uses octal, except by accident?

~~~
sneeu
The only time I ever use it is with `chmod`.

------
ZenPsycho
This entirely misses the point of JSON. if you want these features it's
perfectly acceptable to use YAML instead of JSON.

------
jQueryIsAwesome
I can't believe people are taking this seriously...

Lets write a python parser who accepts semicolons instead of indentation, just
to fix when I forget that i am not writing Javascript; oh, and and make it
accept the "function" keyword instead of def.

------
DavidAbrams
Seems like quoted keys are a pain in the ass. That's the only useful change I
see here.

