Ask HN: SQLite as Dump Export?

[ceritium]

I mean generating a SQLite db for exporting all the data in an API.

I think it would solve several things: - Your customers will not hit rate limits - They don't depend on your uptime - Custom queries - Even GPDR exports for end users

Depending on the amount of data and the nature of the API, the service could generate the dump from scratch or an incremental based on Litestream, for example.

What do you think? Would you like it as a customer for your services? Would you like to use (and someone pay) a service that consumes your API/webhooks and generates an SQLite for your customer?

[fiedzia]

For one-time dump of everything, that would work fine. For keeping in sync regularly, look at Dolt (google "git for data" for more solutions like this one, there are a few already).

[ceritium → fiedzia]

Hi, I didn't know about Dolt, it looks interesting, but probably it's more complex and has more features than I was thinking. In the end, for the idea of generating a dump of the customer data, I don't need all the "git" features.

I suggest using SQLite instead of the typical XML or JSON because SQLite is ready to be queried and easy to generate.

On SQLite, you can even store attachments b64 encoded. It will make it bigger and slower, but not so slow as trying to read a JSON or XML with the attachments inlined.

[jdougan → fiedzia]

I keep reading that name as "Do It"