

How I learned to stop worrying and love the schema - granders
http://blog.confluent.io/2015/05/19/how-i-learned-to-stop-worrying-and-love-the-schema-part-1/

======
twic
The guys at Snowplow Analytics are really into applying schemas to event
streams, and will write or bang on about it at any opportunity:

[http://snowplowanalytics.com/blog/2014/07/11/on-the-
benefits...](http://snowplowanalytics.com/blog/2014/07/11/on-the-benefits-of-
configurable-schemas/)

[http://snowplowanalytics.com/blog/2014/05/15/introducing-
sel...](http://snowplowanalytics.com/blog/2014/05/15/introducing-self-
describing-jsons/)

[https://github.com/snowplow/iglu](https://github.com/snowplow/iglu)

I think there's a lot to be said for knowing for sure what shape your data is.

------
ingenter
My experience might be insignificant.

I've been working with several APIs, both XML and JSON, and I'm yet to see an
example where a schema have helped me writing an automated parser. Validating
the response/request? Sure.

Maybe I did not have the right tools, maybe I lacked the knowledge, but the
fact is, I've been writing such parsers by hand. I'm vaguely aware that there
are various C++ parser generators from XML schemas, but I could not use those
because of license/compatibility issues.

I suspect it would be possible to write a schema->parser converter using meta-
programming, but how many languages are capable of that without huge amounts
of pain? I could probably do that in Python or LISP, but not in some other
language.

And I have not seen a JSON schema being used... anywhere?

~~~
twic
Someone sent me a WADL file the other day:

[http://www.w3.org/Submission/wadl/](http://www.w3.org/Submission/wadl/)

It didn't take long to find a tool which would generate a client library from
it.

Three caveats:

Firstly, i was working in Java; WADL originated at Sun, so there are Java
tools for it, but i doubt any other language has support.

Secondly, i don't think it really saved me much time. Hand-coding a JSON API
client isn't exactly difficult. The main advantage i can see would be in
dealing with an evolving API, where the implementer of the API can continually
update a WADL file which clients can use in their build process to stay up to
date.

Thidly, the generated code had a bug, and blew up on one of the data types
returned from the API!

------
fintler
A good way to enforce schema backwards compatibility is to have something like
a pre-commit hook or an auto-revert that triggers when the compatibility check
fails during the build of your schema class library (assuming Java).

In addition, consistent use of schemas is also essential to operating
something like Kafka audit:

[http://www.slideshare.net/JonBringhurst/kafka-audit-kafka-
me...](http://www.slideshare.net/JonBringhurst/kafka-audit-kafka-meetup-
january-27th-2015)

------
webnrrd2k
I'm, not sure I agree that the solution is always to use a schema... In
general, yes, it's a good idea. However, it seems to me that, in this
particular example, the correct format could be determined by the device_id in
the JSON formatted text. That is, the date format when device_id=123 is X, the
date format when device_id=watch_345is Y, etc... If there are a small number
of device_ids then it's a workable solution, maybe even preferable if the
amount of data to be transferred needs to be minimized.

However, once JSON was used, then I would almost certainly have put in a
schema/format/version field in a header.

------
gizzlon
Two things:

1) The people on the "temp-o-meter watch team" are all retarded

2) He fails to show a schema that will solve this, and I seriously do not know
what he's talking about. What would fix this:

 _" Had this company been using schemas, the watch and phone teams could have
simply reused the same schema, avoiding the need for the watch team to
reinvent the wheel, "_

Given that the different languages are used [1], what kind of magic fixes
this? Without requiring all the parts to support something common technology I
might add..

[1] if not, there wouldn't be a problem

------
kolev
There's nothing prettier than a pretty ERD!

