
Datasette Facets: Faceted Browse and a JSON API for any CSV File or SQLite DB - simonw
https://simonwillison.net/2018/May/20/datasette-facets/
======
simonw
Here's a demo of it running against 190,000 trees in San Francisco, using my
datasette-cluster-map plugin to include a map visualization: [https://san-
francisco.datasettes.com/sf-trees-ebc2ad9/Street...](https://san-
francisco.datasettes.com/sf-trees-
ebc2ad9/Street_Tree_List?_facet=qCareAssistant&_facet=qLegalStatus&_facet=qCaretaker&_facet=qSpecies&qSpecies=12)

More about that plugin: [https://simonwillison.net/2018/Apr/20/datasette-
plugins/](https://simonwillison.net/2018/Apr/20/datasette-plugins/)

------
chrisweekly
My favorite tool in this space is [lnav]([https://lnav.org](https://lnav.org))
which has an embedded sqlite engine and works with all kinds of structured
data. It might obviate the need for datasette, or maybe complement it in a
scripted workflow....

------
donut
Nice. Looking forward to trying this out.

Reminds me of [http://openrefine.org/](http://openrefine.org/).

~~~
aorth
Indeed! I was just thinking that this could be a substitute for viewing where
I don't want to have to fire up OpenRefine, but for editing, GREL,
reconciliation, etc OpenRefine is still king! I've just started playing with
reconciliation in OpenRefine via conciliator + Solr. Pretty cool.

[https://github.com/codeforkjeff/conciliator](https://github.com/codeforkjeff/conciliator)

------
obelix_
I see something about uploading the CSV file. Not clear where the sqlite dB is
being created. Is it local? Does the browser create it? Ideally I would like
to run this locally when I am playing with data.

What's been the largest file tested? Basically what's the max rows/cols it can
handle fast?

~~~
simonw
You can run it locally - that's the default way to use it. "pip3 install
datasette", create your SQLite database file and run "datasette mydb.db" to
start exploring.

I've run it successfully on SQLite files up to a GB in size and theoretically
it should work with much bigger files than that.

The [https://publish.datasettes.com/](https://publish.datasettes.com/) tool
works by taking your uploaded CSVs and running my "csvs-to-sqlite" script on
them, then deploying the datasette application alongside that new SQLite
database file.

~~~
obelix_
Thanks! Looks pretty useful. Will be checking it out soon.

------
hprotagonist
i’ve been keeping tabs (ha!) on this for some time.

keep up the good work! it is incredibly useful for me to be able to rapidly
poke around a new data set and get a lay of the land.

Perhaps equally importantly, being able to do this in a meeting with the semi-
higher ups, live, makes you look fiendishly smart.

------
brianzelip
The final section of this recent Changelog podcast provides an interview with
the Datasette author,
[https://changelog.com/podcast/296](https://changelog.com/podcast/296).

------
gigatexal
This is awesome. Is there something like this for Postgres?

~~~
simonw
Not that I've seen so far, but I did write a related tutorial on building
faceted search on top of Django and PostgreSQL a few months ago:
[https://simonwillison.net/2017/Oct/5/django-postgresql-
facet...](https://simonwillison.net/2017/Oct/5/django-postgresql-faceted-
search/)

------
chrisweekly
Nice publish workflow, leveraging Zeit `now`.

