Jd is also decent. More expressive languages for data.
I find VisiData is great for quickly exploring and querying data from the CLI. It can handle many types of files (SQLite, CSV, TSV, Excel, JSON, YAML, etc). Visidata loads all the data into memory, and so is very responsive when exploring the data. It allows you to quickly do all sorts of of adhoc queries interactively, without having to write a valid SQL query.
I haven't used Q. When I first heard of it, I liked the idea that Q allowed you to run random queries on CSV and TSV files. However, it seemed like it would be slow if you wanted to do follow up queries, since it had to repopulate the in memory SQLite file for each query. Though it looks like the latest version has a way to cache the generated sqlite file. So that seems like it could help.
Also, if I have some CSV, TSV, JSONL data sqlite-utils is useful for converting them to SQLite, and then exploring with Visidata or SQL queries.
PS Unaffiliated, just a fan since 2016(?)
PPS Despite the name and original / primary use case, lnav is useful for things beyond "just" logfiles(!)
And of course the SQLite part is wasm.
When I am reading through the source code of somebodies personal project, it's absolutely amazing to:
- follow along significant architectural decisions
- watch a web API in action, not a specialized abstraction that will necessarily hide some capabilities
Column UI would be awesome
This could then be pitched to non-techie folk who want to simply ask questions from their data on flat file.
I don’t knock the research at all, it’s very cool. But it has to be basically perfect for someone even slightly experienced to consider using, otherwise it will cause more frustration then time saving.
It might take you, as a programmer or technically literate person, tens of minutes to learn the SQL necessary to parse a csv file.
But I have met a whole raft of people that would firstly take much longer to learn enough SQL to get by, then would still need help when basic syntax error messages came up, and then would get frustrated before getting someone else to do basic analysis for them.
Analysis has a lot of meanings. Getting basic facts (there are N rows that X) and selecting subsets of data are realistic analysis non technical people might do on csv files.
And I want to be clear that parsing a csv file with sqlite or MySQL is much more complex than learning “SELECT .. FROM .. WHERE” (with no joins!) when there is a tool like OPs that magically does the “parsing”.
Sqlite CSV import:
.import file.csv tablename
It would be great if I could use it as a kernel in JupyterLite!