On: Gron: Make JSON greppable

[zimpenfish]

It does - I patched that into my local fork of `gron` two years ago.

[zimpenfish → zimpenfish]

Just done a test with my 800MB stress test file.

`jq`: 1m26s 21G resident

`mygron -e --no-sort`: 18m14s 19M resident

`gron --no-sort`: 1m51s OOM killed at 54G resident

[xiphias2 → zimpenfish]

Can you try https://github.com/adamritter/fastgron as a comparision?

[zimpenfish → xiphias2]

`fastgron`: 8.5s 2.2G resident

edit: Interestingly whilst doing this test, I piped the output into `fastgron -u` (39.5G resident) and `jq` rejected that. Will have to investigate further but it's a bit of a flaw if it can't rehydrate its own output into valid JSON.

[xiphias2 → zimpenfish]

I released fastgron v0.7.5 which contains fixed in string escaping. Could you please take another look?

[zimpenfish → xiphias2]

Already commented on the github issue but will have another look, yep.

[xiphias2 → zimpenfish]

Sure, I saw it, thanks!

I fixed the semicolon bug, but of course correctness is more important.

[zimpenfish → xiphias2]

Update for anyone following - 0.7.6 recreates my 800MB input JSON correctly after trip through `fastgron | fastgron -u` which is good work.

[xiphias2 → zimpenfish]

Thanks, it's a clear bug. I created a new issue for it: https://github.com/adamritter/fastgron/issues/19

[Alifatisk → zimpenfish]

> `gron --no-sort`: 1m51s OOM killed at 54G resident

Oh dear

[zimpenfish → Alifatisk]

If I remember correctly, it took a 128GB AWS EC2 to parse that file without OOMing. Go is not that efficient at deep multi-level size- and type-unknown data structures.

[icholy → zimpenfish]

Thanks for the follow up. Is your fork public?

[zimpenfish → icholy]

There's a branch here - https://github.com/rjp/gron/tree/f/reduce-memory