Hacker News new | comments | ask | show | jobs | submit login
AlaSQL.js – JavaScript SQL Database for Browser and Node.js (github.com)
120 points by truth_seeker 36 days ago | hide | past | web | favorite | 43 comments

This is very cool. But Github tells me that the minified version is still over 400KB.

I really wish browsers hasn't decided against WebSQL and replaced it with IndexedDB.

As much as I also wish we could have a full SQL engine in browsers, it's just too much work.

The web as a platform thrives on multiple implementations, even going so far as to require it for new standards in many areas.

All implementations of WebSQL were basically a lightly wrapped SQLite, and that meant that a bug in SQLite was a bug in all browsers, and design decisions in SQLite would impact the entire web ecosystem, and that was consitered by many involved to be a bad tradeoff.

IndexedDB is much simpler to spec out, much simpler to build multiple implementations of, and much simpler to maintain and improve over time.

In a way it's punting the problem down to userland, but userland doesn't need to abide by the same rules and restrictions as a blessed standard does, so it makes sense to push more complexity down to that level.

400kb is a lot to pay, but it just goes to show how complex an SQL engine really is. Paying for that in the form of a JS bundle download is one thing, but having that complexity in every browser means a larger attack surface, larger maintenance surface, and at the end of the day still doesn't completely remove the need for things like AlaSQL.js (which may target a different implementation than WebSQL)

SQLite is already likely the most widely distributed software library in the world. It's in most browsers, it's in both major mobile platforms. It's in cars, planes, boats, trains, and probably in more than one software on your desktop, if not your desktop OS/platform itself. It's also one of the most thoroughly tested libraries on the planet.

In any case, I would rather have the W3C, etc. just setting on a minimal SQLite version implementation as a baseline. Or, similar to JS spec now follow with the minimal active version in major browsers. MS IE/Edge was the main holdout, and they're going with Blink (Chrome) based builds now.

As to AlaSQL, I'm curious how it compares to an emscripten build of SQLite.

> it's just too much work.

Creating two+ independent implementations before allowing the spec to be adopted may be too much work, but the same would be true of HTML5 or modern DOM specs to implement from scratch today. Fighting to ensure that there are multiple implementations of standards is a noble goal (incidentally, a fight we're in dire need of for HTML/DOM), but expecting these implementations to materialise fully formed for new standards is a severely limiting requirement.

Without that requirement, supporting WebSQL would be less work for browser makers and web developers alike. Who knows, competing implementations might even have followed down the line had the spec. been adopted.

But that requirement is the only thing that keeps one browser engine from being able to run FAR ahead and take over the standards process.

Without that requirement, PNaCL would be the standard not WebAssembly, without that requirement ShadowDOM would be Chrome's version and not the version that most browsers are implementing now and which most people agree is better.

It's not just a philosophical goal, but one that has time and time again found issues with proposed standards and prevented them from being included.

It may keep some things out of the standards which should be in it, but it also keeps a LOT of mistakes out as well.

I think it is too bad. Almost all browsers except for Firefox use Chromium (blink) as their browser engine. That means that a bug in Chromium is a bug in almost all browsers, and design decisions in Chromium impact the entire web ecosystem.

I agree with your above comment, and we can argue that working on a single web-engine might lead to less duplicated work for Front-End devs, and less bugs in the web overall.

Given that sqlite is the most widely deployed DB already, making it part of web browsers doesn't seem like a huge deal to me. https://www.sqlite.org/mostdeployed.html

But the bug aspect is only one part of it, the other is that it ties the future goals of the projects together.

What happens when the web wants to implement some special optimization or security addition but SQLite doesn't want or support being used that way? What happens when SQLite makes a breaking change in a future version that might make sense for them, but would break a large number of web apps?

Once something is in the web standards, it's VERY hard to drop support for it. SQLite might make sense now, but what about in a decade?

No matter how you slice it, WebSQL has the ability to become a massive maintence liability, one that all major browsers agreed was too significant to codify into a standard.

The user seems to be complaining more against Chromium's dominance and not about SQLite being left out of the web ecosystem.

You can replace every mention of "SQLite" in your comment with Chrome (or Google) and suddenly you're agreeing with the comment instead of expressing your opinion against it. Chrome currently dominates web standards, and other browsers must keep up or risk losing even more market share to Chrome.

But that's not exactly something that Chromium or the web standards bodies can change... (well, Chrome might be able to, but i'm not so sure it would be beneficial)

Many web standardization systems already require multiple implementations of a feature before it can become "standard", and by that nature some will have to implement it first. Chrome has Google behind it, and they have a lot of money and a good reason to spend it here (the better the web is as a platform, the better their company which is based on the web works).

But there are plenty of examples of where Chrome's "dominance" didn't allow them to just unilaterally push through APIs or platform changes. PNaCl failed and we got WebAssembly which is much better. The original version of ShadowDOM was shot down and it left Chrome behind other browsers when it came to the true implementation. HTML Imports were again something chrome implemented first and is currently being removed due to it ultimately not working out well and the platform as a whole moving in a different direction. Hell most of the WebComponents spec was changed or removed before it started getting in other browsers in a major way.

Sure, Chrome could just stop improvements altogether, but that seems like a poor solution. One thing I'd love to see from Google is for them to pull their (in)famous move of eating their own lunch. I'd love to see them pull a Servo and try to spend the time and money creating a new rendering engine from scratch to compete with their own Blink engine.

Don't forget that all iOS devices use WebKit. Even Chrome and Firefox on iOS are using WebKit under the hood.

> Almost all browsers except for Firefox use Chromium (blink) as their browser engine.

Edge and Opera have only a fraction of the Firefox, IE, and Safari usage, so I'm not entirely sure why you're phrasing things this way.

> IndexedDB is much simpler to spec out, much simpler to build multiple implementations of, and much simpler to maintain and improve over time.

Agreed. It's not easier to use though - almost everyone ends up bundling 10s of KBs of code just to wrap around it and make it useful. That doesn't seem great to me.

I completely agree, IndexedDB sucks to use on it's own. But that is the direction most web-platform APIs are going, and if I'm being honest, it's working out pretty well for most people involved.

Using bare IndexedDB sucks, but it's flexible enough to support a pretty wide range of needs. Using bare Fetch sucks in some cases (it's a hell of a lot easier than xmlhttprequest!), but most people create small wrappers in their applications to do things like add headers, auth information, and simplify JSON parsing and error handling. Using bare canvas APIs sucks, but with a small wrapper allows pretty powerful abstractions with very little code.

It has the unfortunate side effect of a lot of code written and shipped to users to simplify APIs and make common use cases easier, but if that downside allows the APIs to be as flexible and powerful as they are, then I'm personally all for it.

Agree. And while IndexedDB deserves credit, I cringe a bit every time I see a SQL implementation in JS. There's clearly a SQL need for developers, so I still feel we missed an opportunity w/WebSQL.

To supplement, see my 2015 comment in response to Lovefield: https://news.ycombinator.com/item?id=10197672

> And as a result, we're having to implement our own relational databases. In JavaScript. 4-5 years later.

Now nearly 10 years later. The notion that a WebSQL-like library would be created on top of IndexedDB to replace the deprecated API has proven to be utterly groundless.

It's hopeless, how on earth will a bloated JS library compete with the beast that backs WebSQL? Sqlite, a native C application, just blows everything else out the water. Table joins, window functions? Can't even think the thought with anything built on IndexedDB.

Worse, in the same way that many languages these days compile to javascript, so too do server-side DSLs generate SQL (see Slick, Quill, Esqueleto, LINQ to SQL, etc.). Wouldn't it be nice to share the same model and sql statements on client and server? You know, offload complex aggregation tasks to a native C application designed specifically for this task, instead of manually doing it all by hand with reams of error prone boilerplate in javascript.

Interestingly Chrome and Safari, the browsers that dominate the market, after all these years still ship with WebSQL. Perhaps there's still hope...

> The notion that a WebSQL-like library would be created on top of IndexedDB to replace the deprecated API has proven to be utterly groundless.

Uh, we came here to discuss AlaSQL.js which does implement a decent alternative to the WebSQL standard and does store to IndexedDB, when asked. It's probably a better alternative given its additions of JS functions and JSON operators.

In the footnotes you can even find a WebSQL shim that recreates the old WebSQL API on top of AlaSQL (though it recommends just using the AlaSQL API if you can).

I don't know what the benchmark characteristics of it are, as I haven't yet tried to use this library, but I doubt it is "hopeless".

On the other side, SQLite itself directly runs very well with Emscripten (sql.js), and I've heard Web Assembly builds are getting really good, if you do need the "performance" of "real" SQLite.

All of which seems to be further proof that SQLite is much better in "user land" than in the Browser's security footprint.

> AlaSQL.js which does implement a decent alternative to the WebSQL standard

Well, it isn't at all if SQL is important to you (and corresponding performance gained through delegating to a native C application), or table joins, or efficient aggregation, or window functions etc.

AlaSQL.js is a DSL built on IndexedDB with all the limitations that implies; you can't share code between client and server (unless you're all-in on JavaScript and go with Node on the server); all clients have to download it (400KB isn't huge but it's also not a 0KB "download" like browser built-in Sqlite).

> SQLite itself directly runs very well with Emscripten (sql.js)

Oh, yes, now try writing to disk -- you can't, it's in-memory only, not to mention huge (2MB). I'm sure mobile devices will love parsing that, battery life must be wonderful.

> All of which seems to be further proof that SQLite is much better in "user land"

No proof has been given, severe limitations have been, however. Security is an issue, I'll conceed that, but otherwise I'll stand by "The notion that a WebSQL-like library would be created on top of IndexedDB to replace the deprecated API has proven to be utterly groundless".

> Interestingly Chrome and Safari, the browsers that dominate the market, after all these years still ship with WebSQL. Perhaps there's still hope...

Maybe... I'll have to start experimenting again... when I first looked at it, IE was still too many people to ignore.

WebSQL literally just had a remote code execution bug showing exactly why it was right that browsers decided against WebSQL


Uhm...all major browsers have a long history of remote code execution bugs, and almost certainly still have remote code execution bugs that aren't publicly known yet.

Heck, even if we just consider databases, the IndexedDB implementations in both Firefox and Chrome have had remote memory corruption bugs of the type that are often exploitable for remote code execution, although I don't know if anyone ever actually built such an exploit.

They all have different remote code execution bugs so those bugs can and should be fixed. The reason to not standardise on a specific single implementation is to make sure those bugs do not become part of the web platform

Specifically SQLite had a remote code execution bug, and since all browsers that shipped WebSQL used SQLite as the backing engine, they all inherited the vulnerability and exposed it to all web-pages out there.

And WebSQL was more or less specified to be backed by SQLite, given its "specification" for the SQL used was "whatever SQLite x.y.z implements" (which was one of the reasons it was rejected)

Honestly, I'm more surprised that a WebAssembly version of SQLite isn't more commonly used.

It would be huge, wouldn't it?

Looks like it's around 2.6mb for the not gzipped worker compiled to JS... so would presume it'd be a bit less for wasm, then gzipped to around 800k.


I'm actually in the midst of converting SQL.js to use WASM primarily instead of asm.js (https://github.com/kripken/sql.js/pull/255). By disabling some less-known features and optimizing for size, the WASM build is currently 631KB. When compressed (zip), they are a total of 270KB. https://github.com/Taytay/sql.js/pull/1

very cool.. much smaller than I'd expected... still as big as the entire application I'm currently working on ;-)

I still wish WebSQL just had broader support in general.

Agreed - dropping WebSQL for much weaker inferior IndexDB made no sense to me. Instead of going backwards, there should be a choice for persistent data - Text cookies or Simple KV Doc store or Relational db. I don't buy the dogmatic arguments against SQLite which is Open Source software of the highest quality - even if a major bug is discovered, there is always the option to disable it, anyway surely surely the likes of Google would have every incentive to help fix it quickly.

400KB is not much for a full database implementation, considering Google.com need's double as much to show a logo and a search field.

Yes. But that 400k is in addition to any other assets, payload, etc. That is the total page weight is higher. While Google tops out at a lighter side 400k.

> browsers

The blame lies entirely with Mozilla/Firefox: https://nolanlawson.com/2014/04/26/web-sql-database-in-memor...

Microsoft was also a dissenting voice against WebSQL. It was a team effort (because this is about standards), not just Mozilla's "fault".

SQLite is not a standard, and it would have been an IE6-level mistake to bake SQLite quirks and bugs into the web standard for the rest of time. (It doesn't matter that SQLite is open source, it's still a central source of failure that the web doesn't necessarily need in every browser for the rest of time.)

Just as "entirely" as the blame lies with those suggesting WebSQL not proposing something that meets the well-established and agreed on expectations for a web specification, and not fixing that issue when it was pointed out.

I suspect that a smaller builds could be generated. There's a lot of functionality like reading from a sqlite db or Excel file that you could probably drop to improve the package size depending on use case.

I am researching for a project idea I have and was actually looking up in memory SQL databases for node last night to store and sort data instead of creating a object and custom functions to filter it.

I came across https://nanosql.io/ also which I am leaning more towards. Not to the point yet to play with either yet but it seemed like nanosql had better documentation and also less issues opened on Github right now.

Kinda odd how when you look up things or think of things you see signs of them... Just wow'ed to see something I was looking at last night be #4 on the HN homepage... I know people talk about positivity and the law of attraction but starting to believe more and more in it where thoughts and focus becomes reality.

If it's on the backend, why not sqlite? It's pretty great!

Just temporary data. However SQlite does have a In-Memory Database support, almost forgot about that. But probably SQlite would be a bit bloated to use for this part anyways. This being native to JS and even typescript support just seem nice... However SQlite is more battle tested, was watching a tech talk about it once and they even use it for airplane computers!

SQLite can also be compiled to wasm which has some cool use cases for syncing with backend data stores.

Interesting that Mike Bostock posted this today: https://beta.observablehq.com/@mbostock/sqlite

I'm not sure how the two compare...

Persistence seems to be via files... So I suppose just as for Lovefield, all the data needs needs to be loaded before use? This also seems to make sharing between tabs tricky?

Does this use web workers to access the db? Otherwise I'd worry about queries blocking the main thread, especially as the database gets larger.

Maybe better for hybrid app dev.

Applications are open for YC Summer 2019

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact