
Jtc – CLI tool to extract, manipulate and transform source JSON - vyuh
https://github.com/ldn-softdev/jtc
======
meddlepal
> jtc is written in idiomatic C++ (the most powerful programming language to
> date)

Citation needed. The `jq` vs `jtc` section is interesting, but author seems a
little full of himself with some of the explanations.

~~~
kbenson
Oh yeah, immediately above that line:

> jq is written in C, which drags all intrinsic problems the language has
> dated its creation

And then to follow on the C++ claims:

> Main JSON engine/library does not have a single new operator, nor it has a
> single naked pointer acting as a resource holder/owner, thus jtc is
> guaranteed to be free of memory leaks (at least one class of the problems is
> off the table) - STL guaranty.

That's a lot more faith than I would be willing to put into C++ or C. Sure,
that claim might be correct, but there's enough edge cases and undefined
behavior in the language that I take it as a fairly bold claim unless it's
also been _thoroughly_ reviewed for any places some undefined behavior may be
induced.

I mean, it's probably fine, but just the willingness to make such a bold claim
in that way communicates the opposite of what the author likely intended for
me.

~~~
otabdeveloper4
> "undefined behaviour"

Yeah, you don't know what you're talking about.

"Undefined behavior" simply means "stuff not covered by the ISO standard".

So Python and Rust are 100% UB, and people don't really care.

~~~
kbenson
Undefined behavior can cause perfectly functioning code in one compiler
version to be completely elided in a different version, as we've seen numerous
times in the past. Review the article and comments here[1] for a good example
of exactly this and discussion about the how and why, but the TL;DR is that a
specific optimization in a specific compiler release determined that the
adding of two variables, one of which a pointer, would result in an overflow
which is undefined for pointers, and thus elided the check to see if the
result was less than should be possible (overflowed), and the entire error
handling case it resulted in. This caused the value to be used while invalid
(as the error check and handling was elided), causing a segfault.

Perhaps the features used in this project protect against that, but my point
is that undefined behavior and the way compilers deal with it is variable and
problematic.

1:
[https://news.ycombinator.com/item?id=14163111](https://news.ycombinator.com/item?id=14163111)

~~~
otabdeveloper4
It's definitely problematic, but so is having your interpreter completely
redefine language constructs on every point release. Or having different
interpreter implementations act differently because there's no standard.

Somehow we make do in these cases and don't act like the sky is falling.

~~~
kbenson
> Somehow we make do in these cases and don't act like the sky is falling.

I think "there's enough edge cases and undefined behavior in the language that
I take it as a fairly bold claim unless it's also been thoroughly reviewed for
any places some undefined behavior may be induced" was me acting like the sky
is falling. I think that's a very sane push back against a claim that I think
either wasn't well founded or wasn't well explained.

Ultimately it's up to you to decide whether to accept I'm expressing what my
words actually say or whether I'm expressing what you presented me as in your
straw man argument you originally replied with. At this point, there's not
much else I can or am willing to waste any more time saying that I haven't
already.

------
vyuh
The author has also written related tools. One to convert XML to JSON and back
([https://github.com/ldn-softdev/jtm](https://github.com/ldn-softdev/jtm)) and
another to convert JSON to SQLite tables ([https://github.com/ldn-
softdev/jsl](https://github.com/ldn-softdev/jsl)). Combining these with the
hxnormalize tool ( [https://www.w3.org/Tools/HTML-XML-
utils/man1/hxnormalize.htm...](https://www.w3.org/Tools/HTML-XML-
utils/man1/hxnormalize.html)), one can do very sophisticated manipulation on
HTML web pages.

HTML -> XML (via hxnormalize) -> JSON (via jtm) -> process using jtc (or even
jq)

~~~
namibj
This suggest a very scalable, easy approach to extract data from somewhat
regular HTML...

~~~
Yetanfou
I generally use _xidel_ [1] for that type of task. Feed it _xpath_ , _css
selectors_ or its own pattern matching thing.

[1] [https://github.com/benibela/xidel](https://github.com/benibela/xidel)

------
pmoriarty
At first I was a little skeptical of this tool's claims of being simpler than
jq, as I struggled to understand why (for instance) they were using <> in some
parts of their example and [] in others, but after going through their step-
by-step explanation it all made sense, and it does seem very simple. A tool
that simplifies things is always welcome.

~~~
bradknowles
The "User Guide.md" file is over 3000 lines long, comprising some 133KB. I
read through about a quarter of it, before my eyes started to glaze over.

I'm not convinced that I would call this "simpler" than jq.

~~~
ak217
jq is one of the most intentionally designed pieces of software I've seen. In
due time, I expect it to become as ubiquitous and indispensable as ls, grep,
sed and awk. The jq language could use a formal spec, but other than that jq
has a sublime elegance.

By contrast, this seems like one of the least intentionally designed pieces of
CLI software I've seen.

------
justinsaccount
Looks neat, but seems slow?

    
    
      $ wc -l conn.log
        8505 conn.log
      $ cat conn.log |time ~/src/jtc-standard.json/jtc -a -w '<uid>l' -qq  |shasum
            0.44 real         0.43 user         0.00 sys
      167f6d638a4ccf9e0be1ff4ed74caa01a461bd9a  -
      $ cat conn.log | time jq -r .uid|shasum
            0.10 real         0.10 user         0.00 sys
      167f6d638a4ccf9e0be1ff4ed74caa01a461bd9a  -
      $ cat conn.log | time json-cut uid |shasum
            0.04 real         0.01 user         0.01 sys
      167f6d638a4ccf9e0be1ff4ed74caa01a461bd9a  -
    

json-cut is a crappy tool I wrote that use github.com/buger/jsonparser

------
techie128
It probably should not invent its own language or if it does, it should take
inspiration from XPath/JSONPath which is arguably simpler.

~~~
Rotareti
I agree. Modern SQL supports JSONPath. Postgresql 12 has support for it. A lot
of people are/will be familiar with it.

~~~
chrismorgan
Be careful with terminology: the thing defined in SQL:2016 is called the _SQL
/JSON path language_, and although it’s similar to the older JSONPath, it’s
not the same.

PostgreSQL also calls the SQL/JSON path language “JSON Path”, and the data
type is called `JSONPATH`.

~~~
Rotareti
Thanks for the clarification! I guess it would make a lot of sense for a tool
such as jtc to use _SQL /JSON path language_ then. I'm sure it'll be the most
common JSON query language soon.

------
jtms
Sorry, but the api seems quite obtuse on first glance. I’ll just use Ruby or
JavaScript or literally any modern language with good Json support

------
fergie
JSON functionality is a subset of Javascript. Surely therefore the most
sensible approach to pipe to a JavaScript runtime such as node.js? I can see
how a C/C++ lib might me useful, but why build a standalone CLI just for JSON
manipulation? Surely this functionality exists already? Or am I just missing
something?

~~~
aargh_aargh
[https://developer.mozilla.org/en-
US/docs/Web/JavaScript/Refe...](https://developer.mozilla.org/en-
US/docs/Web/JavaScript/Reference/Global_Objects/eval#Do_not_ever_use_eval)!

------
mastrsushi
What makes this different then all the 300 json libraries provided by and for
the programming languages we already use?

If I really wanted CLI, I would use one of those languages to write a program
that reads arguments while making use of one of those JSON libraries.

------
bachmeier
That's a complicated query tool. I'll be sticking with Taskwarrior for my
simple use cases.

~~~
triangleman
Taskwarrior?

------
munk-a
> jtc stand for: JSON test console, but it's a legacy name, don't get misled.

It's time to stop admitting that (if the name is so terrible) and re-brand or,
more likely, come up with a better backronym meaning for your name.

~~~
gojomo
JSON Transform Command

~~~
munk-a
That is pretty perfect - the term usage is a bit unnatural but nobody will
notice since developers use weird names for things all the time (thanks,
General Regular Expression Parser!)

------
alanbernstein
Can anyone compare to jq?
[https://stedolan.github.io/jq/](https://stedolan.github.io/jq/)

~~~
vyuh
[https://github.com/ldn-softdev/jtc#jtc-vs-jq](https://github.com/ldn-
softdev/jtc#jtc-vs-jq)

