Electric is designed to support partial sync, and so you don’t have to sync your whole dataset. (Note that this is feature is under development and not yet public)
There are limitations on how much data a browser will store for an individual site, so the number of rows you can sync will depend on the shape of your dataset. Finally there are also some performance considerations with WASM SQLite, this is something the SQLite team are working on in collaboration with the browser developers, particularly with the development of the new OPFS apis which we plan to support as they mature.
So, thousands of rows are definitely viable, and we have that working with our own internal development tests. Hundreds of thousands or millions may cause issues right now, but are something we do want to support.
I wish they didn’t remove sqlite from browsers “because it’s not an open standard”. It was one very important thing in common between mobile and web, and now it’s clunky and slow on web when running over WASM. I don’t see how that was in any way good for users.
Yeah but SQLite doesn't protect against unlimited CPU use and has historically had issues where "malicious queries" cause crashes -- not OK for a browser to surface to scripts.
> Almost all CVEs written against SQLite require the ability to inject and run arbitrary SQL.
> The advertised consequence of most CVEs is "denial of service", typically by causing a crash through a NULL pointer dereference or a division by zero, or similar.
> But if an attacker can already run arbitrary SQL, they do not need a bug to cause a denial of service. There are plenty of perfectly legal and valid SQL statements that will consume unlimited CPU, memory, and disk I/O in order to create a denial-of-service without requiring help from bugs.
> Hence, the mere fact that an attacker has a way to inject and run arbitrary SQL is in and of itself a denial-of-service attack. That the arbitrary SQL might also tickle a bug in SQLite and cause a crash is not a new vulnerability.
JavaScript can do that too. Web browsers defend against DoS attacks in the browser by popping up an alert asking if you want to kill the tab process or not. I don’t see why sqlite can’t be run under that model as well.
You’re saying you have partial sync already working for browser sized datasets say 5k rows which are an arbitrary subset of server records which are much larger? What limitations are there on the query criteria for client-synced records, can the subset be expressed as an arbitrary join? How do you implement local first on arbitrary join?
Electric is designed to support partial sync, and so you don’t have to sync your whole dataset. (Note that this is feature is under development and not yet public)
There are limitations on how much data a browser will store for an individual site, so the number of rows you can sync will depend on the shape of your dataset. Finally there are also some performance considerations with WASM SQLite, this is something the SQLite team are working on in collaboration with the browser developers, particularly with the development of the new OPFS apis which we plan to support as they mature.
So, thousands of rows are definitely viable, and we have that working with our own internal development tests. Hundreds of thousands or millions may cause issues right now, but are something we do want to support.