
Jolt – JSON to JSON transformation - bencevans
http://bazaarvoice.github.io/jolt/
======
fiatjaf
I see this is a Java library, but if you're in the command line (or even if
you are able to call an external process for the job) jq[1] is great.

[1]:
[https://stedolan.github.io/jq/manual/](https://stedolan.github.io/jq/manual/)

~~~
donpdonp
I was hoping jolt was a command line tool in the style of jq. I dont believe
jq does transformations of json, only queries.

~~~
mdaniel
> I was hoping jolt was a command line tool in the style of jq. I dont believe
> jq does transformations of json, only queries.

I guess that depends on one's definition of transformation but:

    
    
        echo '{"hello": "world"}' | jq '{brave:{"new": .hello}}'
    

it also doesn't even require JSON input, meaning one could use it strictly for
composing valid output JSON:

    
    
        # where -n = null input
        jq -n --arg foo bar '{"meet me": ("at the "+$foo)}'
    

I use it for transforming AWS responses all the time. I put jq up there with
vim as one of the "must have" tools. In fact, the keys `%!jq .` are burned
into my fingers because I type them a lot

_ed: fixed a mobile character replacement, added in the --arg flag I couldn't
remember at the time_

------
JayOtter
The idea of supplying a spec to transform arbitrary data is interesting to me.
I did one in JavaScript called Reshaper[1], and then hooked it up to a library
wrapper called Smolder[2].

The result was a sort of system whereby data going into a function would be
automatically 'reshaped'. It worked well as a proof-of-concept, but obviously
was too fragile for most uses (though it's used in the automatic graphs in
Kajero[3]). The difference here seems to be that the spec defines the actual
transformations, rather than just the desired final structure.

[1]
[https://github.com/joelotter/reshaper](https://github.com/joelotter/reshaper)
[2]
[https://github.com/joelotter/smolder](https://github.com/joelotter/smolder)
[3] [https://github.com/joelotter/kajero](https://github.com/joelotter/kajero)

~~~
marktangotango
Are you familiar with Xml and xsl/xslt? They took this concept a long, long
way.

~~~
ygra
And DSSSL before that. But I guess XML is just not hip enough anymore.

Actually, having done some work on XSLT these days again I was joking to a
colleague at work that one day someone will surely think what a great idea it
would be to create a programming language in JSON to transform JSON to other
JSON. And lo and behold, a few days later I stumble over this.

I have to admit, XSLT looks far nicer to me, but that might be familiarity and
a proper specification to read to understand things instead of just a bunch of
examples.

------
pluma
This seems like something that would lend itself to not tying itself to any
specific implementation, yet it seems to be entirely based on a particular
Java implementation.

It would probably be more useful if it more explicitly tied itself to the Java
implementation (i.e. stop pretending to be its own thing) or were more
abstract to be worth implementing in other languages.

In the latter case it'd be helpful if the operations were part of the actual
unified DSL instead of having a DSL for each transformation (with the
implication that each transformation is applied individually?).

EDIT: But if you abstract this away from Java, why not just use JSON Patch:
[https://tools.ietf.org/html/rfc6902](https://tools.ietf.org/html/rfc6902)

~~~
jonaf
The spec you linked to is dated Apr 2013. Jolt's first release was in Feb
2013.

------
zeveb
_sigh_ , can we _please_ just stop re-implementing S-expressions and Lisp, and
instead just use S-expressions and Lisp?

It's like I'm the only one using electric lighting while all the hipsters are
upgrading their wax-dipped hemp brands to artisan whale oil …

------
mozey
I've used template libs to do this in the past, e.g. mustache.js,
handlebars.js, Jinja2, etc. Then generating JSON as output instead of HTML.
Usually quick to learn the template libraries DSL, and I find templates easier
to read than transforms.

------
lolive
Some food for thought:

JSON format+JSON.parse() make you loose the graph structure of the data you
have on your server and that you send to the client. Because it is basically a
tree structure.

The Semantic Web defines a graph description langage called N3. If your server
can serialize and send the data in such a format, and if you use the function
N3.parse() on your client, you eventually retrieve, on the client, a graph of
in-memory objects that corresponds to the data graph on your server. You can
then traverse that graph in any direction you want.

So basically, with N3, you never lose the graph structure of your data.

And you do not need to restructure your JSON.

~~~
jsight
I have often wished that RDF and semantic web technologies had caught on. I
think the overly complicated specifications and poor tools doomed, them,
though.

I completely agree that trying to represent a graph structure of data properly
into JSON can be inordinately difficult.

~~~
lolive
I paste a previous comment of mine, from an old HN thread:
[https://news.ycombinator.com/item?id=10947013](https://news.ycombinator.com/item?id=10947013)
"RDF has no adoption. JSON is the winner. Let's try to understand why: Because
JSON.parse() is instantly available on the client. On the contrary,
RDF.parse() (or more precisely N3.parse()) is available on the client only if
you use that obscure library I mentionned above. Noone knows the library, so
noone uses the N3.parse(). Ok. End of discussion. Now my own feeling: RDF (and
especially its N3 dialect) is the only description language I know of that
serializes and deserializes graphs with no specific code, and has a good JS
lib for client-side consumption. JSON (or XML, or any other tree descrption
language) requires extra effort whenever you use it to
serialize/deserialize/traverse a graph structure. That is why i think they
suck. And that's why I prefer RDF/N3."

~~~
jsight
I think that there is more to it than just lacking a RDF.parse() function. The
entire web of technologies surrounding RDF is just vastly more complicated and
the formats themselves feel unnatural to people (in a way that JSON does not).

------
intrasight
In .Net-land, I've switched to doing this type of thing in Linq. But if you
like XSLT (I still do), you can do it in three lines of code. 1\. Convert JSON
to XML 2\. Run XSLT 3\. Convert XML to JSON

~~~
ygra
That approach is listed on the site under alternatives.

------
dr3s
Interesting but with no mention of json patch in the alternatives. I'm not
sure why this project would be better.

~~~
themihai
json patch is verbose and limited. It's designed to patch documents, not to
transform them.

------
lobster_johnson
Related/shameless plug: We've been working on a reimagining of JSONPath [1]
which we hope will be useful to other people.

Unfortunately, it's not public yet — though we have working libraries in
JavaScript and Go (a heavily modified fork of Kubernetes' JSONPath code) we
intend to release — mostly because we want to publish a proper spec. In fact,
we're looking for a good name for it, as we feel that releasing "JSONPath 2.0"
would be a little presumptious. I was thinking something like JSONMatch. If
there's sufficient interest I could prioritize it (email me!).

Our own version of JSONPath is intended for both searching and patching
documents in general. A simple search would be something like:

    
    
        friends[name == 'bob'].id
    

or

    
    
        shoppingItems[1:3][description, id]
    

We use this to declaratively extract data from documents, but also change it.
The patching support lets you do things like:

    
    
        match(document, "..[role == 'owner'].role")
          .set('admin');
    

This will set "role". It gets rather magical and beautiful when you have
multiple paths that involve unions, subindexed arrays and constraints.

We also have a separate patch notation, expressed in JSON, to declaratively
transform documents. It uses these JSONPaths to select and set values. We
might write a spec for that, too, although I'm not sure the utility outside
our app is that great.

[1]
[http://goessner.net/articles/JsonPath/](http://goessner.net/articles/JsonPath/)

------
jonaf
The readme is more up-to-date than GitHub pages. It answers some of the
questions in the comments.
[https://github.com/bazaarvoice/jolt](https://github.com/bazaarvoice/jolt)

------
anilgulecha
This is an interesting take on schemas in json. Is the big advantage that this
provides additional data validation (on top of type validation?) Can there be
custom transforms written in javascript?

------
otabdeveloper1
Are we _really_ going to reinvent all the spectacularly bad ideas of XML,
except this time in JSON?

~~~
arethuza
XSLT did have _some_ nice bits - I still like XPath.

Edit: Just to be clear some of the most terrifying code I've ever seen (and
possibly wrote) was in XSLT.

~~~
manarth
Increase the terror further by adding Jsonx to the mix:
[http://www.ibm.com/support/knowledgecenter/SS9H2Y_7.5.0/com....](http://www.ibm.com/support/knowledgecenter/SS9H2Y_7.5.0/com.ibm.dp.doc/json_jsonx.html)

~~~
arethuza
Isn't one of their examples invalid XML?

[http://www.ibm.com/support/knowledgecenter/SS9H2Y_7.5.0/com....](http://www.ibm.com/support/knowledgecenter/SS9H2Y_7.5.0/com.ibm.dp.doc/json_jsonxconversionexample.html)

Shouldn't

    
    
        <json:string name="ficoScore"> > 640</json:string>
    

be

    
    
        <json:string name="ficoScore"> &gt; 640</json:string>

~~~
manarth
It's one of the many peculiarities of XML: the "less-than" bracket is not
allowed in text (because it could indicate the opening of a new XML tag), but
the "greater-than" bracket does not need encoding when it's in text, because
it's not ambiguous.

~~~
arethuza
I'm pretty sure most XML handling libraries would use &gt; - never seen a raw
> used in XML (though possible I might not have noticed).

Edit: Forgot to say thanks for pointing that out!

~~~
benibela
Just check what the W3C says about writing a raw >:

> For characters such as > where XML defines a built-in entity but does not
> require its use in all circumstances, it is implementation-dependent whether
> the character is escaped.

([https://www.w3.org/TR/xslt-xquery-
serialization-31/#serphase...](https://www.w3.org/TR/xslt-xquery-
serialization-31/#serphases))

------
fiatjaf
Seems enormously complicated. Is this "cool" in Java-land?

~~~
zamalek
Couldn't agree more. If you really think about it, Javascript _is_ the
transform language for JSON.

~~~
mvindahl
Yup. There is an insane difference between how straightforward it is to
transform data structures in Javascript and how cumbersome it is in Java.

------
oweiler
This is much easier to do with Groovy.

~~~
CorpOverreach
Let's be real though.

This type of thing is targeted towards enterprise and the usual large and
well-established environments.

Environments where replacing a 20+ year old app written in COBOL isn't a
question.

No one is ever going to say "Let's write this new app in Groovy!". On top of
management saying "WTF is Groovy?", it'll never get approved because it's a
huge amount of technical risk to take on.

In big enterprise, you want to have your stuff written in the most supportable
language you can find resources for. This is why Java is so popular - it's a
lot easier to find a Java developer than it is someone who knows Groovy, Rust,
Scala, etc.

~~~
vorg
> it's a lot easier to find a Java developer than it is someone who knows
> Groovy, Rust, Scala, etc.

By mentioning Apache Groovy in the same breath as Cobol, Java, Rust, and
Scala, you're comparing apples and oranges. Groovy is to the JVM what bash is
to Linux. Groovy is a dynamically-typed language, best used for writing
scripts used for glue code, testing, build control, and what not, whereas
those others are statically compiled, used for creating actual systems.
Although static compilation was added later, virtually no-one uses it.

------
legulere
The website stutters here with safari when scrolling, making reading the site
a bad experience.

------
Vinkekatten
Well wy not go all the way there? It's just a matter of time. Here's the
manual.

[http://www.wrox.com/WileyCDA/WroxTitle/XSLT-2-0-and-
XPath-2-...](http://www.wrox.com/WileyCDA/WroxTitle/XSLT-2-0-and-
XPath-2-0-Programmer-s-Reference-4th-Edition.productCd-0470192747.html)

