
Why Dart should learn JSON while it’s still young - Max_Horstmann
http://maxhorstmann.net/2014/01/31/why-dart-should-learn-json-while-its-still-young/
======
skybrian
This is more complicated than it seems.

First, how does it work with minimization? In Dart, string constants are not
minimizable (will be included as-is in the source code). Symbol constants will
be minimized. This includes all method names, field names, and things like
named parameters. You can see this in the arguments to noSuchMethod; the named
parameters map has Symbol keys, not strings.

Probably something could be worked out, though. Maybe put an annotation on the
customer class to tell the compiler to save the original field names?

But a second issue is how it works with inheritance, including mixins. If a
class is serializable, does that make all the superclasses and subclasses
serializable as well? JSON doesn't have a standard way to represent a type
tag, so does that mean we should invent one? If we do that, what happens to
JSON interop with other languages?

Also keep in mind that Dart supports types that don't interop with JavaScript
like unlimited precision integers. (Well, okay, that only works with the Dart
VM, not dart2js...)

It gets worse. What if your class has a field type like List<dynamic>, which
could contain any object? Does that mean the Dart compiler now has to preserve
all symbols because any class could be serialized? And keep in mind that with
optional types, all lists should behave the same when not in checked mode.

Having seen the mess that resulted in GWT-RPC, I would recommend just saying
no to default serialization. Serialization and object inheritance don't really
mix well which is why specs like JSON and protobufs don't support it.

(And now you know why language designers don't throw things into a language
just because it seems like a good idea at the time.)

~~~
shadowmint
does _anything_ actually work with dart2js?

All I seem to hear is people complaining about it.

Given so little effort is given to it, I halfway think they should just give
up on javascript and javascript interop completely and focus on making dart
awesome.

Anyhow, what you say is totally true, but surely iterating recursively over an
object and dumping values into a json block would be trivial.

The hard part is _deserializing_ that back into objects, but maybe you dont
need to? The python json package does a pretty good job. You could just have
it return a dynamic or something.

~~~
floitsch
Disclaimer: I'm the TLM of dart2js.

The dart2js team is one of the teams with the most developers on it. There is
a huge effort going into it.

Dart2js generally does a good job, but a few features (like mirrors) that are
still in beta-quality, are currently a constant source of grieve for the
developers. We would have liked to delay their release until they were more
stable, but several customers really, really needed them.

Things are getting better, though, and I hope that dart2js will fade into the
background (as a simple working tool) soon.

~~~
shadowmint
I understand you guys are working on it, and it's a tough problem, but for a
flagship javascript drop in replacement isn't releasing something that griefs
developers _really bad_?

Surely releasing a top quality library that works well is _far more important_
than getting in early and releasing something that alienates developers and
makes them try dart and them give up on it?

It's obviously your prerogative to do what you feel is best, but it's
certainly not the choice I would have endorsed.

(I supposed the horse has bolted to a certain degree now, it's good to know
that a lot of effort is being directed at dart2js I suppose, although from a
developers persepective I don't really feel that it's particularly visible...)

~~~
magicalist
> _releasing something that griefs developers_

Maybe you only see (and/or notice) people talking about corner cases that
still need addressing (like azakai's sibling comment about the mapping of
arbitrary precision integers to JS's number type)? For the vast majority of
code, AFAIK, dart2js has no issues and emits good JS code.

I don't develop with dart, but I know a handful of developers that use dart2js
to release their dart code on the web and it works just fine for them. mraleph
just released IRHydra2[1], which is all Dart from top to bottom via dart2js
and works really well (despite using the apparently still-nascent Dart port of
Polymer).

If you have specifics, by all means, talk about them, but "All I seem to hear
is people complaining about it" seems like really shaky ground from which to
make comments about griefing and alienating developers.

[1] [http://mrale.ph/blog/2014/01/28/prerelease-
irhydra2.html](http://mrale.ph/blog/2014/01/28/prerelease-irhydra2.html)

~~~
shadowmint

        - huge js file size (2MB+ easily)
        - obscure hard to use js interop facilities
        - very slow js interop
        - broken js minification (ignored tickets)
        - no older browser support
        - ridiculously long compile times
    

I honestly welcome anyone who uses dart in a production environment to link to
a blog post or explain how they worked around these, because we found they
made using dart2js such a nightmare we gave up on it.

I speak only for my own experience, perhaps others have different experience,
but no one seems to have _good_ things to say about the experience working
with it that I can see...

(amusingly, from your link: 'Does not work in Firefox (neither Stable nor
Nightly), but works in Chrome Dev and Safari.')

~~~
mraleph
> (amusingly, from your link: 'Does not work in Firefox (neither Stable nor
> Nightly), but works in Chrome Dev and Safari.')

It's most likely problem with polyfills, not with dart2js compilation problem,
I have not tried to debug it yet.

~~~
mraleph
Fixed. It was purely my bug: I was relying on a certain ordering of
propChanged() callbacks, but it does not seem like Polymer.dart guarantees
any.

Now it works in Chrome/FF/Safari all the same.

------
floitsch
I was the TL of the libraries for almost a year, and there are good reasons we
didn't go this route:

\- it adds overhead. Being able to write JSON.encode(customer) would mean that
every class needs to retain its field names. Minification could still work if
this information is stored on the side, but it would add overhead.

\- it doesn't make sense to automatically encode elements that cannot be
decoded. You should have: x =~= JSON.decode(JSON.encode(x)). (where x =~= y
means that the two objects are Json-equivalent).

\- The Dart-Editor already provides a quick-fix to write a toJson for you. To
be honest I'm not really a fan of it, since I believe that serialization
information should be outside the object, and not in the object (that's what
"toEncodable" is for), but that's another story.

\- It's _extremely_ easy to get the behavior you want. Just use (or write, if
none exists yet) a package that has your intended behavior. This is really a
one-line change... This way you pay the price when you want to, and don't
force it on all users.

~~~
syntern
Where is toJson in the Dart Editor? (I'm looking for it in 1.1.1, but couldn't
figure it out.) And does it have a fromJson generator too?

~~~
floitsch
It was recently added, but apparently removed again. (I don't know the exact
reasons).

Edit. you can see the announcement here:
[https://plus.google.com/109866369054280216564/posts/XbjvN5QV...](https://plus.google.com/109866369054280216564/posts/XbjvN5QVpn9)

Edit2. Removal of the feature:
[https://code.google.com/p/dart/issues/detail?id=16425](https://code.google.com/p/dart/issues/detail?id=16425)

------
grimlck
What's wrong with just using a library?

Scala learnt XML while it was still young, when XML was just as popular as
JSON is now, yet incorporating XML into the language is now considered a
mistake.

------
gwbas1c
The problem with magic serializers is that they tend to be brittle in
practice. This is because the assumptions they make don't work across
languages.

For example, if you use a magic serializer, your properties might have to
follow a different capitalization format than what's normal for your language.

Recently, I ported some Java code to C#. It was written with a magic JSON
serializer. When I got to test my port, nothing worked! It turns out that
Java's serializer used lowercase names in JSON, but my serializer used
uppercase names! The magic, unfortunately, didn't work, because assumptions
that were valid in Java did not carry over to C#.

I think it's best to think of magic serializers as rapid prototyping tools
that work well when both ends are the same languages. Thus, while might be
useful for Dart to include one very simple quick-and-dirty serializer to JSON,
any production code of merit will most likely outgrow it.

------
tyleregeto
Golang has great JSON support that I think could translate to Dart. Any
overhead could be minimized through compiler support so we don't have to rely
on the current mirror functionality. I think that would be a nice balance
between what the author of this article is asking for and dart2js performance.

~~~
skybrian
Go doesn't have inheritance and that makes things quite a bit easier; there's
an obvious way to deserialize a map to a struct type.

You can see the seams in the limited support for interfaces; basically there's
no way to deserialize to non-empty interface type. (How would that work?)

In Dart, since any class can be extended or used as an interface type, the
issues are similar to interface types in Go. There's no bare struct type.

There might be an interesting compromise, but it's not a trivial design
problem.

------
oscargrouch
Is there still people willing to use Dart instead of Javascript for web now?
(no irony)

~~~
michaelwww
Yes. For some use cases the speed of development and rapid prototyping is
worth it.

~~~
shadowmint
...but there are unavoidable severe limitations like slow js interop, large
file size and poor browser compatibility.

Its really only plausible for very specific project where these arent issues.

~~~
michaelwww
Dead code elimination and minification do a good job of reducing Dart2js file
size. Dart converted to JavaScript works on IE9 and above and all other modern
browsers. JsInterop is done through port send and recieve, so if you need to
make a lot of those calls it is potentially slow, but if not, for example I'm
making one call to PDF.JS, it works beautifully. Dart's real problem in my
opinion is that many 10's (100's?) of nifty JavaScript libs are written
everyday that won't work well with Dart. The JavaScript ecosystem just keeps
getting better, while the Dart ecosystem plays catchup. Because of this I use
TypeScript for some projects.

------
deathanatos
It's the type data that makes this hard: JSON has no support for encoding new
types into the serialization. This is a blessing and a curse: it makes JSON
simple, but also inflexible.

Take this parting remark:

> So, here’s another thing I wanna add to my suggestion: consider adapting ISO
> 8601 for a default JSON serialization of DateTime, so the following just
> works:

> var dt = DateTime.parse("2014-01-26T11:38:17");

> print(JSON.encode(dt));

Ignoring the fact that it's a simple value you're attempting to encode to
JSON, let's say this works. Do you get:

    
    
      "2014-01-26T11:38:17"
    

Now do to the reverse: what does that decode to? Either it decodes to a
string, and you've lost the type information, or it decodes to a date, in
which case, what does `JSON.encode("2014-01-26T11:38:17")` decode to? i.e., do
strings sometimes decode to dates? What if someone uses what should be a
string to cause the decoder to decode it as a date? Now anywhere you have a
string, it might decode to a date!

YAML has support for this sort of thing, with a syntax like `!!date
"2014-01-26T11:38:17"`. (If you want a literal `!!`, it's always `"!!"`.)
Notably, the Python library for YAML _can_ do what the author asks: encode and
decode arbitrary objects. (but it turns out to be a security risk to do so on
untrusted data.)

> The Dart editor should be able to display a JSON serialization of anything
> the mouse pointer touches.

If my class contains a file handle, what's the JSON representation of a file
handle? Further, if I send that serialization across the wire, how do you
deserialize it? I mention this because you can't, at least, not really.
Python's `repr` is a good example here. When it can, it gives you close-to
Python syntax for the object, such as strings (`"hello world"`), ints (`1`),
decimals (`Decimal(3)`), etc. Perfect for debugging. For things it can't, you
get something like `<file object "~/foo.txt" at 0xdeadbeef>` — which doesn't
parse (on purpose) as Python.

I'd venture to say that what you really want is the ability to introspect.
Take your customer example: if you could introspect it, and see what member
variables were present, you could build a function that might output:

    
    
      {"type": "Customer", {"Id": 1, "Name": "Joe Bob"}}
    

Again, note the explicit serialization of the type here, in JSON. This is
something _I_ made up, of course, and not JSON. Of course, some language
feature can make introspection interesting. (Can you even get a list of
members? Some languages allow objects to make up members on-the-fly.)

Hopefully, I've convinced you language design is hard. :-)

~~~
pcwalton
With enough static typing, or some sort of schema, none of this is a problem:
all you need to do is to specify the type you're deserializing to. That said,
it sounds like Dart is in trouble here due to other problems with its design
(inheritance).

~~~
cwp
Well, there is a language that's perfectly suited to JSON serialization, and
it has inheritance but not static-typing.

The point here is that designing a language around the current fashion in
serialization is a bad idea. The problem as laid out in the article is
unsolvable. A serialization format can automatically capture the semantics of
a given language's native data structures, or it can easily interoperate with
other languages, but not both.

JSON hits a sweet spot on the web because it captures a large subset of
Javascript's data structures, while being simple and inflexible enough that
other languages can work with it via libraries.

------
daurnimator
You don't just want ISO 8601, but more exactly: RFC 3339 (a subset)

~~~
Max_Horstmann
Ah, you're right. Thanks!

------
TillE
The lack of easy serialization is pretty strange. Even on stodgy old non-
dynamic versions of C#, YamlDotNet does a really good job of translating
arbitrary objects to and from YAML/JSON just using reflection.

~~~
skybrian
Server-side languages have it easy. Code size isn't much of an issue so
reflection works fine.

In a browser where code size matters, there are two ways to go:

1) In JavaScript, every library author needs to figure out how to crunch their
library as much as possible and library authors brag about their small code
size and minimal dependencies. We have debates about whether JQuery is worth
it.

2) You implement tree-shaking so you only pay for the functions you use (GWT,
Dart, and the Closure compiler's advanced mode). But tree-shaking doesn't play
well with reflection, which is why dart2js reflection is a work in progress.

You can also go with code generation (essentially compile-time reflection) but
that also increases compile times and adds code bloat due to the generated
code.

------
cwp
Yes, it would be great if everything always just worked, and libraries always
had your desired behavior as the default. Google should implement that right
away.

------
the_gipsy
I was recently writing a JSON un/serializing component in TypeScript, which is
much nearer to JavaScript than Dart I believe, and even though I just had to
make it work with a few certain types I knew in advance, it was still a PITA.
I don't think there's a good way to JSON-to-anything-serializing, you always
end up with some type related "tags" which shouldn't really be in the JSON
data.

------
coldtea
Thanks, was thinking of delving into Dart, but between the mediocre JS interop
and this, I decided not to. Not to mention that ES6 brings tons of goodies to
JS too, and the speed difference is not that great anyways.

Great JSON integreation is one of the things I enjoy most from Javascript
(well, it makes sense given the J in JSON).

------
pjmlp
Personally I only care about JavaScript and am looking forward to ES6.

I don't care about transpilers and still looking for a use case worthwhile for
Dart.

Unless Google pushes it, for example as Android language, ChromeOS or whatever
might be the language killer application.

~~~
deathanatos
I know ES6 brings a lot of new shiny to the table, but JS is a poor language.
It's riddled with traps and interesting stuff, and you have to put up with
excessive differences between browsers. It's just stuff like

    
    
      Object.prototype.hasProperty.call(my_object, "property");
    

or not doing

    
    
      {
        foo: 1,
        bar: 2,
        baz: 3,
      }
    

or the truth table, things like `{} + []` and `[] + {}`, the insane implicit
line joining… the list goes on and on.

I'm not sure Dart is the answer, but I welcome some competition. I think I'd
be really interested to see a browser VM that languages could compile to,
which would open the door to many languages. (I suppose asm.js might argue
that they're this, and there is LLVM.)

Finally, I leave you with [1].

[1]:
[http://wtfjs.com/2012/08/11/Slashes](http://wtfjs.com/2012/08/11/Slashes)!

~~~
spion
Reports of JS warts are greatly exaggerated. The two big remaining ones (post
ES6) are type conversion by operators and silent failure when accessing object
properties. Both are fixable with a nice type system with HM inference and
structural types. I wish someone did JUST that, instead of trying to bring all
the problems and warts of Java to JS while trying to fix it.

Roy looks very promising[1], and I'm hoping it will mature to the point where
it can be considered a viable option for production projects

Typescript would've been ideal, but its kind of fake-open-source (no actual
development can be seen on the repository, some of the code looks generated,
no discussion about the internals or providing an API) and its moving towards
having LESS type inference in the name of improving compiler speed, less type
safety in the name of... "its too hard to do" I guess. And they still are not
releasing anything which might help other vendors improve their tooling (e.g.
Intellij IDEA, the second-best contender sucks at TypeScript). And the module
system is a total disaster, especially if you're trying to do anything more
complicated and CommonJS.

[1]: [http://roy.brianmckenna.org/](http://roy.brianmckenna.org/)

------
Max_Horstmann
Thanks for all the great feedback! I've updated the blog post with some of the
key points from this discussion.

------
thatthatis
Learn JSON, but please for the love of Turing's ghost, teach it to use
trailing commas.

So many needless syntax errors.

------
outside1234
Dart should be JavaScript. Seriously, what's the point of Dart again?

~~~
oscargrouch
I think they just mess it up with the marketing of it.. but the technology
per-se is very good.. and if you try to see it not as a contender to any other
technology like javascript.. but just as another player out there to create
programs.. there's nothing wrong with it..

People tended to be very sentimental about it.. because they see as a threat
to javascript, as something that came to replace it.. and thats what was bad
in the image that came out, when they were launching dart..

but if you leave this wrong image behind and look only to the technology
achievement with it, what's not to like?

~~~
pjmlp
Dart is basically a modern Smalltalk looking for a market.

~~~
densh
And under modern Smalltalk do you imply modern Java?

~~~
pjmlp
No, because Java isn't able to provide the type of dynamic enviroment coupled
with live coding as Smalltalk does.

------
munro
Dart is actually trying to teach the web about static typing.

~~~
pcwalton
Static typing is not incompatible with JSON serialization, if done properly.
It sounds like Dart's semantics are the problem here (and since they've
reached 1.0 they can't fix it).

Dart effectively has very little static typing anyhow.

------
r0man
We should all support EDN

