
Jq: A lightweight and flexible command-line JSON processor - wtetzner
http://stedolan.github.io/jq/#
======
glesica
_jq is written in portable C, and it has zero runtime dependencies._

Thank you! When I clicked on the link I was fully expecting it to be written
in Node and I was going to cry.

~~~
Xavi
Why is having node as a dependency such a bad thing? No snark. Genuinely
curious.

~~~
cryptonector
Node is great, and a port of jq to node would be awesome (someone once hinted
at such a port, but I've lost track of it).

But jq also has libjq, a C library. And jq uses a copy-on-write, reference
counted representation of JSON values, which is, for example, inherently
thread-safe (though jq isn't using atomic operations for refcount management
yet). The library is easy to use and powerful.

Ultimately the main thing I love about jq (and why I contribute to it) is the
jq language itself. It reminds me of my one-time favorite, Icon. But in a
world where C is still the champion of systems programming (until Rust takes
over?), it's real handy to have a C JSON library _and_ a functional DSL that's
easy to invoke from C.

~~~
slapresta
One of jq's maintainers (nicowilliams) often mentions Icon as a source of
inspiration.

~~~
cryptonector
@nicowilliams (github) and cryptonector (HN) are one and the same ;)

It's no secret either, just look at my commit messages' email address :)

------
Zolomon
They use this tool extensively at Spotify. I saw a presentation by
[https://twitter.com/jooon](https://twitter.com/jooon), when I was over there
in September last year.

It is a really cool tool. Especially since they use JSON internally for almost
all of their data representations.

Jq + Percol[1] is kind of cool.

[1] [https://github.com/mooz/percol](https://github.com/mooz/percol) \- was on
HN a few days ago.

~~~
X4
If percol is based on aut-fu, then it's probably not that good. The screenshot
shows auto-fu, I know that zsh-plugin, but it's really messy and does very
aggressive auto-completion (erasing your previous input). I'd rather use fish-
shell instead.

~~~
ithkuil
no percol is not shell plugin, but a simple cmdline util used in a pipeline, I
suggest you take a look. I understand that animation on the website is a
little bit confusing though.

~~~
X4
oh, cool! I'll take a look then. =)

------
simonw
This is a really neat tool.

If you just need pretty printing and don't have jq installed, you can use the
python command line with the built in JSON module:

wget [http://reddit.com/.json](http://reddit.com/.json) | python -mjson.tool

~~~
jamiesonbecker
# (fixed) output to stdout:

wget -O- [http://reddit.com/.json](http://reddit.com/.json) | python
-mjson.tool

------
davidbanham
This script drops you into a coffescript REPL with the json loaded into memory
under the variable j.

I find it nice to be able to loop the data, define new subsets, filter it with
functions, etc.

It was a 10 minute slap-job to get it together, so it does barf on
particularly large datasets. The obvious reason is that it serialises the JSON
into an argument for a child process. Shouldn't be too hard to resolve if
anyone feels like it.

[https://github.com/davidbanham/dotfiles/commit/0ea950373604e...](https://github.com/davidbanham/dotfiles/commit/0ea950373604e74f71c44fe99c8758572dc656db)

------
skywhopper
I've been using this tool to build a bash library for a JSON/REST API and I
can attest to its usefulness. Filter huge data sets down to just the fields
you want, mutate, select, search, filter. Most everything I've wanted to do
this can handle. Plus it's fast and dead simple to install. Kudos to the devs
for this fantastic tool that I hope will soon be part of the default CL
toolset across the board.

------
mathieuruellan
I'm using this tool with curl in scripts calling AWS api. Looking for an id of
an object (security group, autoscaling group etc.) with its name and
referencing it afterward in another command.

~~~
pavel_lishin
Hey, me too! It's pretty awesome, although the documentation/examples aren't
as great as they could be - it took me some time to figure out how to properly
parse arrays, and I'm still not sure how to actually do what I want to do (I
want to convert an array of two instance objects into an array with just two
instance IDs.

~~~
mh-

        % aws ec2 describe-instances --instance-ids '["i-3026a249","i-28739551"]' \
        | jq '[.Reservations[].Instances[].InstanceId]'
        [
          "i-28739551",
          "i-3026a249"
        ]

~~~
pavel_lishin
Thanks!

------
edgarbob
Does anyone know of a similar tool for XML? In particular, it would be cool to
have a useful way to run xpath, and maybe xquery, from the command line. The
tools I've found have unfortunate interfaces, e.g. they won't read XML from
stdin.

~~~
gav
Have you tried XMLStarlet[1]? It's my go-to for command line XML processing.

[1] [http://xmlstar.sourceforge.net/](http://xmlstar.sourceforge.net/)

~~~
unhammer
Even with libxml's xmllint you can do some xpath stuff:
[http://wiki.apertium.org/wiki/Xml_grep](http://wiki.apertium.org/wiki/Xml_grep)

------
kolev
I've been using it for over a year - especially with AWS CLI, but, I'm a
little displeased that it only has the query feature (duh) and does not have a
little more like merging JSONs (I think it only has appending). Another issue
is that I wasn't able to find release notes for v1.4 - what's new since v1.3?
There other similar tools, but they require node, Python packages, etc. jq
really is installed with one command and no dependencies.

~~~
cryptonector
Yeah, I'm a schmuck, I forgot to add release notes :(

I've been thinking I need to do a 1.4.1 release just for that!

(Plus we're getting some cool new features suddenly. @wtlangford (on github)
is working on regexps, for example. Maybe a 1.4.1 so soon would be justified.)

~~~
kolev
No, you're anything, but a schmuck! Thanks for the great project!

------
lugg
Been using jq for a while now. Probably since the last hn post. My biggest
gripe with jq is the name and asking for help on stack overflow - the jq tag
is an alias to the jquery tag. Someone with tag edit level rep should really
fix that.

~~~
yincrash
You can probably make a post on meta stack about it

~~~
cryptonector
[http://meta.stackoverflow.com/search?q=jq](http://meta.stackoverflow.com/search?q=jq)

~~~
yincrash
the system works!

------
Tyr42
Here's a cute little jq script I used to munge some json around.

[https://gist.github.com/tbelaire/d00cb77fb0db7b38bed2](https://gist.github.com/tbelaire/d00cb77fb0db7b38bed2)

------
otikik
I just use it just as a pretty-printer for the console (jq '.') , and it does
the job very well.

------
slapresta
I can't stress how useful this is. I use it to process AWS SQS messages for
debugging. Incredibly powerful tool.

I have also used it to validate sudokus, because I can.

[https://github.com/slapresta/jq-sudoku](https://github.com/slapresta/jq-
sudoku)

------
minaguib
If you get and build latest from master, you get the --sort-keys commandline
option

And just like that, you have an amazing base for diffing json documents of
arbitrary complexity:

    
    
        vimdiff <( cat doc1 | jq --sort-keys . ) <( cat doc2 | jq --sort-keys . )

~~~
cryptonector
It's in 1.4!

------
rev087
I've been using this for a while, and it's wonderful to parse the output from
API calls requested via curl, as well as quickly checking a specific
package.json field from the npm output. The --json flag is handy for the
latter.

------
reinier_s4g
Awesome stuff, thanks! Will give it a try really soon.

------
facorreia
This can help Docker automation by processing the output of `docker inspect`.

------
chrismonsanto
This is a great tool. I've longed for an equivalent for YAML.

------
JoshTheGeek
underscore-cli: a similar tool, in JS (node)

[https://news.ycombinator.com/item?id=7896054](https://news.ycombinator.com/item?id=7896054)

------
samuell
Cool, just grabbed it with sudo apt-get install jq

