
CSVJSON – Self Rise of an Online Tool - martin_drapeau
https://medium.com/@martindrapeau/csvjson-self-rise-of-an-online-tool-3a91fef3a201
======
userbinator
Having actually written something like this before --- and it remains on my
machine as an executable of only a few kilobytes --- I'm curious about the
thought process that could lead to someone uploading a considerable volume of
potentially sensitive CSV data to this site, and then downloading a
proportionally larger amount of JSON from it.

The site doesn't even appear to support HTTPS.

~~~
me_bx
Technically, all the file processing can happen client-side now, thanks to the
HTMl5 file API. Users' data doesn't have to leave the browser.

[shameless plug] This is the approach I have taken creating a similar
converter (which also works offline):

[http://mango-is.com/tools/csv-to-json/](http://mango-is.com/tools/csv-to-
json/)

~~~
martin_drapeau
Yes, that is a good idea. However I chose to upload it in order to allow
sharing the conversion via a unique generated URL.

~~~
plugnburn
Why not use location.hash and URL shortener? That way no upload is needed.

~~~
userbinator
What's the maximum length of a URL?

Hint: a lot smaller than almost all the CSV files I've had to deal with.

To the grandparent: you might want to make it more explicit that you're
actually retaining the uploaded data and not just using it to generate the
result page.

------
kawera
Thanks, I've used your tool a few times and it works beautifully. Quite
interesting article too from the SEO pov.

BTW, does anyone know of a tool to do the reverse? I have some json files with
deeply nested arrays and would like to output a csv file with one line for
each "leaf". I can write it myself but would be happy to use an existing tool.

~~~
yareally
> BTW, does anyone know of a tool to do the reverse?

I created a java/scala library[1] (source is scala, but there's a jar with
dependencies for java) a few years ago for an Android app I was hired to work
on that converts nested JSON to CSV. I was allowed to open source the library
later on and released it under the MIT License, so anyone can use it now.

Never thought anyone else would ever need to convert json to csv, but I guess
I was wrong :)

I don't have a use case for it anymore, so I haven't maintained it, but if
someone wants to update it, feel free to send a pull request.

[1] [https://github.com/yareally/json-to-
csv](https://github.com/yareally/json-to-csv)

~~~
EvanPlaice
Yes. Use the built-in JSON.parse() to get the data into memory, then use my
jquery-csv library's $.csv.fromArrays() to format the data as CSV.

Guess I found a good candidate for a blog post. It's amazing How much I see
others struggle with such a aimple concept.

~~~
yareally
Not sure if you parsed my comment incorrectly or you meant to reply to someone
else :)

~~~
EvanPlaice
Somehow missed the 'java/scala part'.

Either way, why do 2 data format transforms as one lib? It makes more sense to
parse JSON into memory in one step then format it as CSV in a separate step.

Ie, what about the 'Single Responsibility Principle'?

~~~
yareally
1) The parsing and converting are done with separate methods. Thus single
responsibility.

2) It's a very small library. Less than 200 lines (including comments).
There's no reason to put this in 2 jars, classes or packages. Have you looked
at the code at all?

3) The method call to parse JSON is in the method that converts the CSV, but
it's trivial to move it out and run it explicitly. Simply remove the call to
parse and explicitly run it separate.

4) Client wanted the ability to filter out JSON data they didn't want in the
CSV. Thus, it's a little more complicated than a call to something like
JSON.parse() in Scala/Java.

5) Code does what it needed to do and made the client happy.

If you think you can do it better while keeping the code in Java/Scala, send a
pull request to my repo and we'll discuss :)

------
HoyaSaxa
Coincidentally, I used this today for the first time. It was the 2nd result on
Google for me, but was far superior to the first in ease of use. Thanks for
your contribution and good luck going forward. My use case today was to create
a fixture of initial data for a django app from csv. Oddly enough I wasn't
able to find a specialized online tool to create the django fixture and was
forced to use your tool along with some manual hacking.

~~~
martin_drapeau
Glad it was helpful. And thanks for the nice comment.

------
EvanPlaice
The funny thing is, very capable libraries already exist to handle CSV
parsing.

I know because I authored jquery-csv, te first RFC complete CSV parser
implementation in vanilla JS.

The transform is ridiculously simple and data can be fully loaded and
processed client-side using the HTML5 File API.

As soon as JS gets the :: function bind operator I plan to reimplement it as a
pure async function.

~~~
martin_drapeau
Wow, I wish I had found your library when I wrote this. Looks like you
published it around the same time I wrote it. I especially like the idea of
client-side loading using the HTML5 file API. Things to consider for the
future.

~~~
EvanPlaice
Unfortunately. I made the mistake of hosting it on Google Code rather than
GitHub. Everything has since been moved and it's gradually gaining traction
again.

The HTML5 part was very appealing to me because I wanted to build a client-
side tool to load/save/edit database imports/exports.

It unlocks a lot of potential for fully-featured client-side applications that
work more like desktop apps.

The bad part about it is, browser vendors, specifically MS, have been dragging
their feet when it comes to fully supporting the File APIs.

------
EvanPlaice
Honestly, I see JSON-to-CSV and the opposite tools as an anti-pattern.

You're essentially applying 2 pure function transforms to data so it should be
done in 2 steps.

The built-in JSON.parse() provides a native implementation to handle the first
transform. My jquery-csv lib handles the second. If you require stream
processing and/or need to process large data sets client-side use the more
fully featured PapaParse lib instead.

The number of half-assed CSV parser implementations I've seen in the wild is
what drove me to provide a fully-featured RFC implementation. To do it right
requires a DFA, anything less will fail on many hard-to-identify edge cases.

My project has been downloaded over half a million times, used on every
platform/environment imaginable, and every technical limitation painfully
identified. Seriously, I don't understand why devs feel the need to roll their
own and miss out on benefiting from existing battle-tested solutions.

Not to mention all the devs who embed a wholesale copy of an old/outdated
version of my lib in their projects.

Nowadays, half-assed CSV implementations have changed to half-assed JSON-to-
CSV implementations. The same holds true for JSON and/or CSV to HTML
transforms.

Formatting a HTML table by iterating over data is so painfully sinple I don't
know why devs feel te need to provide it as a standalone library.

------
homero
You don't rank on beutifier, I've searched lots of times and have never ever
seen you, I wish I did

~~~
supermatt
Um... beutifier or beautifier?

~~~
homero
Maybe user error ;)

------
duncans
FYI: The term I search for when needing something along these lines is "JSON
prettify"

~~~
martin_drapeau
Interesting. I had never considered that search term. The volumne is low
(Google AdWords Keyword Planner shows a 2,400 search volume per month, 10% of
JSON beautifer). Yet it is a keyword I will add. Thanks!

------
l_beka_l
[http://2json.net/](http://2json.net/)

------
hayksaakian
from 15000 quality page views you should expect at least $15 of revenue ($1
CPM) even higher numbers from developers (valuable users).

Check out something like [https://carbonads.net/](https://carbonads.net/) and
it might at least pay for your hosting. It's invite only though, so you might
have to ask for an invitation.

~~~
martin_drapeau
Personally I hate it when ads get in my way. Like anyone else, I do like money
though. So I'm struggling with trying to monetize this. I don't want to annoy
developers.

This being said, I will consider trying carbonads. Thanks for the tip.

~~~
kawera
Have you thought about a long term sponsor? It may interest a toolmaker in the
ETL space by offering to host a respectful ad yourself with a link and no
tracking. You won't get rich but it's probably much better than using an ad
network.

~~~
martin_drapeau
I am open to this. Just haven't actively looked for one yet.

