
SQLite Is Serverless - alexellisuk
https://www.sqlite.org/serverless.html
======
nicholaides
I think a good under-appreciated use case for SQLite is as a build artifact of
ETL processes/build processes/data pipelines. Seems like lot of people's
default, understandably, is to use JSON as the output and intermediate
results, but if you use SQLite, you'd have all the benefits of SQL (indexes,
joins, grouping, ordering, querying logic, and random access) and many of the
benefits of JSON files (SQLite DBs are just files that are easy to copy,
store, version, etc and don't require a centralized service).

I'm not saying ALWAYS use SQLite for these cases, but in the right scenario it
can simplify things significantly.

Another similar use case would be AI/ML models that require a bunch of data to
operate (e.g. large random forests). If you store that data in Postgres, Mongo
or Redis, it becomes hard to ship your model alongside with updated data sets.
If you store the data in memory (e.g. if you just serialize your model after
training it), it can be too large to fit in memory. SQLite (or other embedded
database, like BerkleyDB) can give the best of both worlds-- fast random
access, low memory usage, and easy to ship.

~~~
sly010
I have been using SQLite as a format to move data between steps in a
complicated batch processing pipeline.

With the right pragmas it is both faster and more compact than JSON. It is
also much more "human readable" than gigabytes of JSON.

I only wish there was a way to open an http-fetched SQLite database from
memory so I don't have to write it to disk first.

~~~
SQLite
> I only wish there was a way to open an http-fetched SQLite database from
> memory so I don't have to write it to disk first.

The sqlite3_deserialize() interface was created for this very purpose.
[https://www.sqlite.org/c3ref/deserialize.html](https://www.sqlite.org/c3ref/deserialize.html)

------
oldgradstudent
> Of those that are serverless, SQLite is the only one known to this author
> that allows multiple applications to access the same database at the same
> time.

IIRC, MS Access allowed that, which explained a lot of its popularity.

~~~
wenc
MS Access “allowed” it but depending on the version it could be quite
problematic. We had a use case where Tableau connected to an Access file read
only (but which another program used as a data store and wrote to it often) on
an Windows file share, and once in a while the lock files would get screwy and
we would have to manually delete the lock files to get things working again.
Deleting lock files could be a huge chore because you have to figure out stuck
processes and kill them. Task Manager wasn’t up to the task, so had to use
SysInternal tools to help with that.

Access is really meant for single-user scenarios I feel. Maybe the locking
mechanism has gotten better but for multiuser access I tell people to use a
real SQL database.

~~~
m0xte
It was ok pre-windows 2000. I had access 97 it on 9 workstations on NT with
never a problem! Moment office 2000 and windows 2000 came out it went to hell.
Moved to SQL Server then. I am not sure what changed.

It was quite frankly the most productive custom business software package I
have ever used. Literally would take 10 people a week to do something custom
you could do in an afternoon with 1 person in Access. I suspect the same is
true now.

~~~
JasonFruit
We had a much heavier multiuser setup on Access 97, and it worked fine. We had
the same breakdown as you did, and had to do a registry edit on all users'
machines to keep it limping along until we could move to SQL Server, which was
the right thing to do anyway.

~~~
m0xte
Interesting. Nice to know it wasn't just me. There was no one to talk to about
it back then so you had to think on your toes.

SQL Server was a crap load easier to back up reliably

------
cle
> It is important to understand these two different definitions for
> "serverless". When a database claims to be "serverless", be sure to discern
> whether they mean "classic serverless" or "neo-serverless".

It's really not important to understand that distinction, because this author
seems to be the only one making it. Everyone knows what "serverless" means at
this point, and it's not an embedded DB.

~~~
jiofih
This such an air-headed comment to make. You must realize that page describing
how SQLite is “serverless” has been up probably longer than your entire adult
life. It is important in its context, they are not trying to “redefine” (lol)
the term.

~~~
cle
It doesn’t matter. It is not the generally accepted definition of serverless.
The meaning of a word is based on how it is generally used, and almost nobody
means this when they use the word “serverless”.

At this point trying to use the word in this way just creates a bunch of
unnecessary confusion. Call it something else so we can move on to more
important (and clear) discussions.

(Also there are a lot of assumptions and snarky comments in these responses
about my age, which is quite rude and pretty elucidating.)

~~~
mbreese
Their usage of serverless predates all of these cloud application offerings.
Just because a lot of people in this particular echo chamber like to think of
serverless as meaning one thing, there are plenty of others who have been
using the same word to mean something different for quite a bit longer.

Moreover, the cloud as a service definition isn’t even accurate. There still
is a server — it’s just not one the developer has to worry about.

------
threatripper
In case SQLite is not enough and you need redundant servers or clustering,
there's also database servers that use SQLite as storage engine:
[http://rqlite.com/](http://rqlite.com/)
[https://dqlite.io/](https://dqlite.io/)

~~~
Gys
So you run an app with sqlite on one server and sync to sqlite on another
server? What would be the benefit to using a separate db server like in the
'neo-serveless' setup?

~~~
threatripper
No, in this case you always have to use rqlite/dqlite because they manage the
network synchronization. They use SQLite as storage engine (one SQLite
database per server instance).

~~~
Gys
I understand that in those cases rqlite/dqlite is used. But that it just a
technical detail. My point is that I am running two servers: one with the app
and Xqlite and another one with Xqlite.

In case of a neo-serveless setup, I also have two servers: one with the app,
the other one with the db server.

So what are the benefits of the Xqlite setup? I looked into that before and
for one thing, Xqlite is slower (obviously) then just sqlite. So speed is not
a key benefit. I also will have to manage both servers myself.

At least for a seperate db server I have the benefit that I can buy that as a
service, incl management, backups and such.

~~~
tracker1
Not the author, or knowing of all the technical details... simplistic
replication structure and redundancy/failover without an expensive or more
complex RDBMS solution while still self-hosting the service.

There are still a lot of instances where you cannot use a cloud provider for
your app or database.

To be honest, I'd probably lean more towards a nosql database that has in the
box, relatively easy replication strategy, though that might mean 3+ db
servers for good performance. (RethinkDB, MongoDB, Cassandra/ScyllaDB,
Cockroach). Just depends on the budget and resources.

~~~
PeterCorless
Well, ScyllaDB is free and open source, so that should help the budget.
(Though we do have an enterprise version, base price is FREE!)

~~~
tracker1
I'd probably reach for Scylla wherever I would consider Cassandra (or
similar), though the similar bigtable/columnstore-like services in most cloud
hosts is generally still going to be easier.

The performance over C* is surprisingly good.

------
dana321
It does allow multiple applications to access the same database at the same
time, but when you do so it really hurts performance. I noticed this when i
wrote a web crawler in go and used sqlite as the backend.

As soon as i connect using the command line interface, it slows down
significantly.

Just something to bear in mind if you want to use it with multiple processes!

~~~
iampims
Even with WAL enabled?

~~~
FpUser
Let's say there are concurrent writes to a single table with few indexes.
Process of updating said table and indices eventually boils down to update of
a single B+ tree from a multiple threads. In general until _serialized_ it is
physically impossible without corrupting the tree. Sure there might be other
specially crafted implementation of table more friendly to concurrent writes
but there will be trade-offs. There are no miracles in the world.

As for particular case with WAL, all it does in this particular scenario is
act as a queue. If your database load is spiky then it can even out load and
make an impression of faster response. Under constant load it will not speed
up the things and will internally serialize all actual updates

~~~
catalogia
If you _need_ concurrent/parallel writes, SQLite is not the right tool for
you. You may as well lament that hammers are no good for driving screws.

~~~
FpUser
Can you please point me to where did I say anything about SQLite being right
tool for concurrent write? My point was that true concurrent writes to a
generic database table with indexes are are physically impossible in a general
case disregarding what database it is.

------
pm90
What I really want: SQLite storage backend driver for s3/gcs. No need for
disks then. I haven’t been able to find such a solution though; and am not
technically proficient enough in C (the Lang SQLite is written in) to do so
myself.

~~~
siscia
I am working on RediSQL[1] and I am about to launch a managed version.

The interface will be either HTTP or Redis protocol, you create your database,
and daily I will back it up on S3.

(If interested you can subscribe for updated here:
[https://simplesql.carrd.co/](https://simplesql.carrd.co/)

[1]: [https://redisql.com/](https://redisql.com/)

~~~
pm90
Redisql looks pretty sweet! To be frank, and this is only my personal opinion,
I don't think I would want to pay for an api, but rather run my own. The
business model of providing everything OSS but sending telemetry seems rather
intriguing; as a hobbyist user I am ok with such telemetry being collected. If
I were to run it in production for a business app though, I wouldn't even
bother considering the unpaid version for the following reasons:

1\. I do not want my production instance to shut down for WHATEVER reason.
This is just not an acceptable risk for most businesses. The only time a DB
can go down is something goes wrong.

2\. As an engineer, I understand that 3 counters that are not accurate aren't
a big deal. I can even look into the source and see that they really do as you
say. Justifying this to a security org will be a complete nightmare, as most
security orgs in enterprises are staffed with barely technical folks
masquerading as "security".

So, it seems like a pretty good way to coerce enterprises to pay up while
letting hobbyists continue using it. Very smart, I wish you the very best!

~~~
siscia
Thanks! That was exactly my reasoning.

Make it available and simple for hobbyist and small companies, ask money to
who can afford it to sustain the development.

------
geokon
a bit tangential, but when do you move form using in-application data
structures (maps, trees, vector/arrays) to using a database? Is it basically
when the data doesn't fit in memory? I've been programming for almost a decade
and I've never come across needing a database... (for context, it's ten years
without anything web related) I'm interested in them and I'd love to learn SQL
but I can't even think of a use case outside of managing user in some web
based application

~~~
closeparen
SQL queries tend to be much smaller than their equivalent data-structure
traversal procedures. This can be beneficial even when you still want the data
in your process's address space, hence embedded database engines like sqlite.
Libraries like Linq can also provide the same expressive power over a
programming language's native objects and collections.

As to why you'd want a separate DB server process: long lived mutable
structures tend to drift into unexpected states, which is why the standard
computer troubleshooting procedure since forever is to restart. Database
management systems specialize in not having that problem. You deploy one
that's widely used and hardened over many years, change it infrequently, and
operate it with great care. Or pay a cloud provider to do that.

Then _everything else_ gets to outsource the burden of persistence to it, and
the vast majority of the workloads you develop and operate are stateless.
These are drastically more convenient and more resilient to mistakes. They're
effectively "restarted" with each API request, and you can stand them up, tear
them down, migrate them to new machines, etc. as much as you want without fear
of data loss.

~~~
FpUser
" _long lived mutable structures tend to drift into unexpected states_ "

Why are those "tend to drift" ? As an example I wrote sort of like game server
for one of my applications. Internally it has those exact forever lived
mutable structures. I've never observed it to drift into any unpredictable
state. Works like a charm and running for many month. I only reboot it when I
need to update it to a new version.

The only catch here is that all data fit into RAM. With the amount of RAM
modern computers can be stuffed with I do not really see if my server would
ever run out of it.

Of course it backed up by database but it merely serves as a persistence layer
for this particular application.

~~~
dragontamer
Consider why 3rd normal form exists to begin with.

In the real world, let's say you have a shipping address and a billing address
for a customer, and they are usually (but not always) the same.

Eventually, a customer moves, changing both of their addresses. But the user
forgets to change their billing address with their delivery address.

A proper database would have a 'billing address is same as delivery address'
logic, following the principle of DRY (don't repeat yourself).

\----------

There are lots of examples here of what can go wrong when you repeat yourself
in a database application. The user may have an error when repeating
themselves over the dataset (delivery address is correct, but zip code on
billing address has a typo).

Dealing with these issues at scale, with hundreds of thousands of customers,
is certainly a problem. Normal forms can formalize these issues and help the
business owner avoid the problems.

Where do you verify the existence of zip codes and cities? Where do you check
for typos? How do you prevent contradictions on the submitted information?

Your human customers will make many mistakes. Your logic must hold up even in
the presence of faulty data.

~~~
FpUser
_" A proper database would have a 'billing address is same as delivery address
logic"_

The database does not have logic like this. It has to be implemented by stored
procedure. When I have application server all such logic (if applicable) is
handled by code in much more performant way. No data goes to a database
directly. Everything passes through the app server along with the validation
data transformation etc, etc. As already said the database in this particular
case is nothing more but persistence layer.

Again we can put all kind of theoretical speculations but as I already said,
my particular server does not have data drifting to some faulty states.

~~~
dragontamer
>> "A proper database would have a 'billing address is same as delivery
address logic"

> The database does not have logic like this.

Of course it wouldn't have 'logic', databases are just stores of data.

You'd have one 'address' table, with probably a int-primary/surrogate key.
Then the 'delivery' and 'billing' address would be an int, pointing to the
address table.

Furthermore, the billing and delivery address would be foreign keys, so the
internal database logic would keep the tables in sync with no application code
required.

With the data organized in this manner, the application code becomes logic
free and braindead easy to write. Or at least, corner cases become easier to
handle and more explicit. (Say two customers share the same address, do you
allow repeats in the address table? Or do you allow customers to tie address
information together? Either way, your decision rests on how you define the
primary key)

> When I have application server all such logic (if applicable) is handled by
> code in much more performant way.

The most performant way is no logic at all. A proper database removes a lot of
checks, the storage format itself naturally creates logic free code.

Some application logic is necessary of course. But you can minimize the logic
needed by thinking about data layout.

------
dacort
SQLite is amazingly prevalent as well. Your own phone probably has hundreds of
SQLite databases on it. One challenge with SQLite, though, is you have to
download the whole thing to make use of it.

Shameless plug, but I made a fun side project that allows Amazon Athena to
read SQLite databases from S3. [https://github.com/dacort/athena-
sqlite](https://github.com/dacort/athena-sqlite)

------
bane
I love SQLite, but people approach it from a classic RDBMS angle which
confuses them.

Here's the deal: SQLite is a file format with a nice API that uses SQL as the
paradigm for reading/writing to the file.

That's it. Stop overthinking it.

Can you write a microservice that stores its data in a big JSON file that
you've built some code around to read/write to? Yes. It's just a file, but you
have to build all the read/write methods. SQLite is not really any different,
except the read/write work is already done, and you use SQL to format the data
values and encode the read/write logic instead of the language you are working
in.

The file format has some cool extensions like text indexing, geospatial, etc.
But it's no more a RDBMS than reading and writing a JSON file is.

"But there's indexes!" Yes, just liked you might build an index on your JSON
file and read that before reading the JSON file to know where things are
faster -- and then you have to write all the code to do that. SQLite is just a
file format, where you can also build indexes and all the code for that is
already done for you.

It's just a file format with an API that's similar to the ones you'd use for
any regular old RDBMS and uses SQL as the domain language to read/write data.

It's just a file format. Anything you can do/can't do with a file format you
do with SQLite.

It's just a file format. A nice convenient one that you are probably better
off using than most other things for most purposes.

It's just a file format.

 _edit_

I'm glad this comment is getting such a response. I'm not trying to be mean,
just help clarify thinking.

Here's two thought experiments:

1) If SQLite didn't require you to use SQL as the read/write logic and was
called "datalite", and instead just forced you to use C function calls,
exactly like you would if you were working with literally any other file
format on the planet, would you still be confused as to what it is?

2) Do you consider reading and writing to any other file format anywhere in
the hierarchy of RDBMSs? Consider Python's csv module. Is that an RDBMS? Let's
move away from tabular data, how about python-docx?

~~~
derefr
Consider: it’s totally possible to strip down Postgres until all you have left
is an embedded RDBMS of the style of SQLite. (I’m not sure why nobody has done
this yet, actually.) Would you call the result “just a file format”?

Such an instance of “embedded Postgres” would still have a huge sprawling
catalog (PG_DATA) directory attached to each use of it, so it wouldn’t be
contained to a single file. But neither is SQLite contained to a single
file—SQLite maintains a journal and/or WAL in a second file.

And, yes, this “embedded Postgres” would require things like vacuuming. But...
so does SQLite. Have you never maintained an application that maintains a
long-lived “project” as a single SQLite file, where changes are written into
this project file repeatedly over a long period? (Think: the “library”
databases of music/photo library management software.) SQLite database files
experience performance degradation from dead tuples too, and need all the same
maintenance. Often “database version migrations” of such software is written
to either rewrite the SQLite file into a clean state, or—if it has the
possibility of being too big for that to be a quick task—to call regular
VACUUM-like SQL commands to clean the database state up.

——

Now, I get what you’re trying to say; the point that you’re trying to
make—that SQLite might be a relational _database_ , but it’s not a relational
database _management system_ in the sense of sitting around online+idle where
it can do maintenance tasks like auto-vacuuming. Unlike an RDBMS, SQLite
doesn’t have its own “thread of execution”: it is a library whose functioning
only “happens” when something calls into it. By analogy, regular RDBMSes are
like regular OS kernels, while SQLite is like a library kernel or exokernel.

But that doesn’t mean that SQLite _is_ a file format! It can be _used_ as one,
certainly, but what SQLite _is_ is exactly the analogy above: the kernel of an
RDBMS, externalized to a library. As long as you “run” said kernel from your
application, and your application _is_ a daemon with its own thread of
execution, then your application _is_ an RDBMS.

This can be seen most directly in systems like ActorDB, that simply act as a
“transaction server” routing requests to SQLite. ActorDB is, pretty obviously,
an RDBMS; but it achieves that not due to its own features, but 99% due to
embedding SQLite. All it does is call into SQLite, which already _has_ the
“management system” part of an RDBMS built in, just not called unless you use
it—just like exokernels often already have things like a scheduler, just not
called into unless you as the application layer do so.

~~~
bane
SQLite is a file format with a familiar API and uses SQL as the logic for
searching/adding data to the file.

Approach it exactly the same way you'd approach using a CSV file and all the
confusion and overthinking about it goes away. Approach it as a stripped down
RDBMS and you end up with all kinds of questions about support for this or
that RDBMS familiar service.

You can write your own SQLite file reader/writer. Here's the specs (includes
the specs for the Journal and WAL files and semantics as well)
[https://www.sqlite.org/fileformat.html](https://www.sqlite.org/fileformat.html)

Here's an example of somebody who's done this.
[https://sqljet.com/](https://sqljet.com/) \- this is not a wrapper on the
sqlite C code, this is a re-implementation of that code that is binary
compatible with SQLite files.

The Journal file only exists as a temporary file until transactions complete.
The .sqlite file you make is the entire atomic file that follows the SQLite
file format. The Journal has its own file format. Same goes for the Write-
Ahead log.

RDBMSs also manage connection queues, account management, rights and
permissions, and so on. Many overcome various OS limitations by providing
their own entire file management, fopen(), virtual memory and other subsystems
that are tuned to their workloads.

SQLite is a file format. SQLite uses familiar relational paradigms to make it
easy to read/write data to the format without having to learn yet another API
and domain language. The API code is extraordinarily well tested, and it makes
simple complex logic like transaction journaling, indexing and so on.

>Consider: it’s totally possible to strip down Postgres until all you have
left is an embedded RDBMS of the style of SQLite.

No! SQLite is not an embedded RDBMS. It's a file format.

If there was a library you could import, and it provided methods to read and
write directly to files that PostgreSQL could read/write to and there was
nothing else to install, no runtime, no daemons, no servers, etc., then we
could pass around self-contained PostgreSQL files to each other. Then
PostgreSQL files would be a file format as well.

Have you ever used a library to read/write from a CSV, JSON, JPEG? It's no
different than doing so for a SQLite file!

SQLite is a file format.

~~~
happymellon
File formats don't have an API.

> SQLite is a file format with a familiar API and uses SQL as the logic for
> searching/adding data to the file.

That description is for a library, not a file format. SQLite is a library,
that saves to a convenient format and allows you to query the file using the
SQL syntax. File formats dont have "logic".

You are arguing the equivalent that Word is a file format. While there is a
Word file format, Word itself is an application.

~~~
bane
> File formats don't have an API.

So when you read/write to any other file format, you just read/write bytes
directly to/from disk and re-implement the parsing and read/write logic in
your own code every time?

> File formats don't have "logic".

Every file format has logic otherwise it's just random entropy in an
arbitrarily long byte stream on a disk. How to read/parse and interact with
that format depends entirely on the logic and scheme for that file. For
example, many file formats have an index of some kind that you must read and
parse before you can figure out where the other data lives, compressed file
formats often store a dictionary, image formats often have a
compression/decompression logic that must be followed for reading/writing to
them.

> You are arguing the equivalent that Word is a file format. While there is a
> Word file format, Word itself is an application.

.docx is the file format for Word documents. There are many APIs and programs
that can read/write to/from .docx files.

------
mettamage
Does this mean that you can hack some database storage (w/ sqlite) together on
frontend only hosting platforms like Github or Netlify?

I think not, but I wonder if some hack is available by virtue of it simply
being a file that you can read (and somehow) write to.

The best I came up with: let's say you have a toy project, and you call the
Github API and replace the file upon every write. Implementing a read is
easier as you know where the file is located. This hack shows that you somehow
need write access to get any form of performance out of it, because this hack
is super slow.

~~~
aikah
> Does this mean that you can hack some database storage (w/ sqlite) together
> on frontend only hosting platforms like Github or Netlify?

You can run SQLlite in the client via webassembly therefore open a SQLite file
in the browser to query it yes, you just can't write anything in it and expect
it to persist somehow on the static hosting service itself.

> The best I came up with: let's say you have a toy project, and you call the
> Github API and replace the file upon every write. Implementing a read is
> easier as you know where the file is located. This hack shows that you
> somehow need write access to get any form of performance out of it, because
> this hack is super slow.

No you'll need a server for that, you can't make random HTTP requests to any
server in the browser, because of CORS/SAME ORIGIN policies.

"SQLite is serverless" is meaningless buzzword. It just means that SQLite is
equivalent a flat file where you'd shove some data, just that you can use SQL
to query that file instead of having to index data in it yourself.

~~~
filleduchaos
"SQLite is serverless" is by no means a meaningless buzzword. The term had a
clear meaning before it was coopted by the current web dev fad. SQLite does
not operate on a client-server architecture the way e.g. MySQL or PostgreSQL
do.

~~~
mrmekon
Just to back that up, it had a clear meaning because -less is a valid suffix
to append to English words. When grandma runs out of cookies she is cookieless
(when a website doesn't use cookies it is also cookieless). It doesn't have to
be "in the dictionary" to make sense in conversation.

A quick search of usenet shows "serverless" being used in 1994. It wasn't a
term or a buzzword, it wasn't common, it was just English:
[https://groups.google.com/d/msg/comp.os.linux.misc/r76oNl98C...](https://groups.google.com/d/msg/comp.os.linux.misc/r76oNl98C1k/CkuR8EEuYV0J)

~~~
filleduchaos
Yes, this is what I was trying to say. It wasn't a thing people would throw
around like a buzzword but if someone used it in conversation and especially
in context (like the SQLite page, which was written over a decade ago, does)
people would understand what you meant.

------
greatjack613
This document is golden, serverless means an application reading and writing
to the database in the process directly

------
ashildr
I find this article valuable. I made a mental note to check if SQLite is
enough every time I consider one of the “serverless” options.

------
verdverm
Choose your own definitions... but SQLite is not serverless by common
parlance.

Author creates two definitions for serverless which don't match the common
usage. Serverless is more about DevOps / deploy experience than how the
program leverages OS processes internally.

Apparently MS and AWS are ISPs?

Maybe SQLite could be serverless if you defined it as incapable or running as
a server on its own?

~~~
gpvos
Serverless used to have a very clear definition: not having or using a server,
so SQLite is a perfect example of serverlessness. I was extremely confused
when the newfangled definition (called neo-serverless on TFA) showed up, or
your definition for that matter. Who ever thought up theses confusing meanings
for a word that used to be perfectly clear?

You don't need an ISP to have a server. Any computer or program that listens
to a network port is a server.

~~~
aksss
Or is it any program that _responds_ to comms on a network port. The "serve"
part of "server" ;)

------
chmaynard
I don't think we fully appreciate just how pervasive SQLite is in our
computing devices. Apple recognized its power and utility and embedded it in
macOS and iOS, and Google apparently followed suit in Android. The
universality of SQLite is due in no small measure to its stellar quality and
the trustworthiness of its author, D. Richard Hipp.

------
eb0la
Serverless, like all embedded databases ;-)

In the sense you don't need to provision an additional piece of infrastructure
to power your application :-)

------
zubairq
Yep SQLite is in fact serverless. We based our open source web based IDE on
SQLite and it works fantastically. The best part is that you can even take a
SQLite database and use sql.js and run it offline in the browser!

------
peterwwillis
_" Microsoft Azure Cosmo DB and Amazon S3 are examples of a neo-serverless
databases."_

Can we back away a bit from the bandwagoning of misused terminology?
Serverless literally means "running your apps on somebody else's server". S3
is not a server you run your apps on, it is SaaS that you manipulate through
an API - you don't put your apps on it. If S3 is serverless, then literally
every network service of any kind is serverless.

~~~
tracker1
I always preferred XaaS nomenclature myself. "XaaS with autoscaling"... is
probably better than "Serverless"

------
Gys
The main problem that I see for using sqlite is exactly for it being 'Classic
Serverless'. Because how does one keep an up te date backup? Deploying to
Heroku, Dokku, AWS Lambda and such means the sqlite file will be lost on a
crash or new deploy. Even a VM can crash. Export to S3 on every write? Maybe
if changes do not happen often, so only for specific use cases (and actually I
think you should just generate static html in that case).

I use sqlite for local tests for cases where the live app has a 'neo-
serverless' database. It is very, very fast so the tests run almost instantly.

~~~
throw0101a
> _Because how does one keep an up te date backup?_

[https://www.sqlite.org/backup.html](https://www.sqlite.org/backup.html)

There's a ".backup" command:

* [https://sqlite.org/cli.html#special_commands_to_sqlite3_dot_...](https://sqlite.org/cli.html#special_commands_to_sqlite3_dot_commands_)

Alternatively, given that it's ACID, you could just take a snapshot of the
file system/volume in question, and do a recovery on restore.

Edit: SQLite also has WAL files, so presumably one could just use tar/rsync to
create the backup, and only the last file would be 'corrupted', so you'd lose
the last (few) transaction(s):

* [https://www.sqlite.org/wal.html](https://www.sqlite.org/wal.html)

~~~
bsenftner
Ding! correct answer. In addition, the backups can run in the background,
simultaneous to active use in other threads. When the backup is initiated, it
is a transaction, so it maintains knowledge of what portion of the database is
new since the backup began and does not include that data. Which is what you
want for backup integrity.

------
WolfOliver
I like the definition of classical- and neo- serverless. Does anyone knows
more articles about serverless in terms of classical serverless?

------
hinkley
> Neo-Serverless: The database engine runs in a separate namespace from the
> application, probably on a separate machine, but the database is provided as
> a turn-key service by the hosting provider, requires no management or
> administration by the application owners, and is so easy to use that the
> developers can think of the database as being serverless even if it really
> does use a server under the covers.

I’m having a failure of imagination here.

What would I use a database for that I could reasonably assert to my peers and
superiors that _no_ maintenance whatsoever is required? Backups and restores
count as maintenance. Multi region is now common, if not pervasive.

Are there public datasets that are so common that it would be worth it to
provide it as a service? What other service would behave like S3 but look like
SQLite?

I get that one might be safe to assume that “serverless” isn’t just pure
functions. It could reach out to other services that are not serverless and
still not consume (further) resources on a set of machines while not in use.

But a severless database... I’d have to have something aggressively read-
mostly, written to S3 at intervals and read from serverless processes. But is
that a new thing or reading data from S3?

------
bullen
I integrated my database with the app server, but I still open a socket
internally to access it.

So my system is also "serverless" in that meaning.

Acronyms used to be confusing, this is just ridiculous.

[http://root.rupy.se](http://root.rupy.se)

------
cluoma
I use SQLite as a backend for my website and absolutely love it. Not having to
setup a full dbms just for a small blog is a dream, and backups are as easy as
copying a file.

------
tiku
CSV is also serverless..

~~~
bob1029
CSV doesn't have a write-ahead log.

~~~
masklinn
I don't think sqlite had WAL when that page was created.

~~~
threatripper
I think bob means transactions.

~~~
bob1029
Transactions, durability and acceptable performance with concurrent access.

------
pachico
I currently use it to build a read-only rest service for GeoNames gazetteer.
Each instance contains a copy of the database which means its really linearly
scalable.

------
imafish
My understanding of serverless = easily scalable, managed service.

But somehow the word annoys people. Maybe we should find a better word?

~~~
rumanator
> My understanding of serverless = easily scalable, managed service.

There is already a term for that concept: managed services. There is no such
thing as a managed service that's designed not to be scalable. Some
implementations may be better at scaling than others, but that's it.

The serverless buzzword is pure marketing.

~~~
nck4222
On the one hand, the term serverless is really frustrating to me.

On the other hand, there's a certain amount of amusement I get from seeing
"the cloud" become a marketing buzzword in the mid-late '00's, and now seeing
the same thing happen with "serverless."

When you look at the implementation of the two "technologies", they're about
95% the same. Yet somehow they're pitched as these big revolutions.

In another decade when terminals or p2p become popular again we'll be hearing
about some new buzzword like "Terran" computing, or "social" architecture or
something.

------
macdice
"There's no server process" vs "there's no server on your itemised bill".

------
m3kw9
I don’t get it, then any db can be classic serverless if they run in the same
computer as the app?

~~~
jasode
_> any db can be classic serverless if they run in the _same computer_ as the
app?_

Not just the _" same computer"_ ... it's the _same process id (PID)_.

Extract of relevant text from that webpage:

 _> Classic Serverless: The database engine runs within the same process,
thread, and address space as the application. There is no message passing or
network activity._

In other words, when you compile and link "sqlite.c" into your own executable,
the same PID (process) that handles text input and paints pixels on the screen
-- is the same PID that writes to the sqlite database file. It's all the same
_process_. That's what they mean by "classic serverless".

In contrast, if you make a Go executable that writes to MySQL/PostgreSQL db
and make them both run on the same physical computer, that's not "classic
serverless". It's because when you enter _" ps -aux"_ to list all running
processes, you see _separate PIDs_ for the Go executable and the MySQL db
engine.

Other jargon used might be _" in-process"_ vs _" out-of-process"_ or _"
embedded"_ vs _" external"_. SQLite is sometimes characterized as "in-process
embedded database" but MySQL is an "out-of-process" db.

------
loukrazy
Isn’t it easier to say “embedded” and “severless”

~~~
manigandham
Not in this thread apparently. I guarantee nobody here as ever used the term
"serverless" over "embedded" or "in-process" in their entire career but
apparently the purity and nostalgia of SQLite overrides it.

~~~
filleduchaos
This article was written in 2007, so yes, people have been using the word
"serverless" before managed service providers decided to take it up as a
buzzword.

~~~
manigandham
Nobody used it before. Even the about page says: "SQLite is an embedded SQL
database engine."

"serverless" is a marketing term, then and now. Not sure why there's such a
big defense of it. If you have to argue this much over the provenance of a
term that clearly isn't used the same way today then it's a good sign that
it's not very useful.

~~~
catalogia
> _Nobody used it before._

The archive.org links already provided to you prove that you're flat wrong
about that. SQLite used the term many years before it was a webshit buzzword.
They used it in the intuitive straight forward english sense; somebody who has
no hat is hat _less_ , somebody who has no home is home _less_ , and a
database system that has no server process is server _less_. In 2007 when it
was written, nobody would have batted at eye at this term, the meaning would
have been immediately clear to anybody who had any familiarity with databases.

~~~
manigandham
Just because a single page on the internet used it does not mean the term was
used in the industry. It also doesn't matter if it technically makes sense,
although it's a stretch (because there is a server process, you just share
it).

It's a marketing term, and a poor one at that. That's why SQLite even
describes itself as a embedded, in-process database without client/server
architecture. Because that's the common jargon.

I'm surprised at the endless defense of a marketing buzzword and the argument
over which marketing definition is the "real" one. This is bikeshedding at its
finest.

~~~
catalogia
Nobody was saying it was a buzzword in the industry back then
(actually...[1]). It was being used as a normal English word, not some
misleading marketing drivel like the new use of the term. Somebody who has no
shirt is shirt _less_. An MRE heater that doesn't catch fire is flame _less_.
Somebody who isn't witty is wit _less_. Somebody who doesn't have a clue is
clue _less_.

Are you starting to notice the pattern here? "less" is a suffix that can be
applied to nearly any noun to describe the trait of lacking that noun. When
this is done, the meaning is clear to native English speakers who've never
heard that combination before. When this article was written in 2007, the
meaning was clear. SQLite didn't muddy anything, didn't redefine anything.

[1] The term 'serverless' was in fact in use in the decade and a half prior to
2007:
[https://books.google.com/ngrams/graph?content=serverless&yea...](https://books.google.com/ngrams/graph?content=serverless&year_start=1950&year_end=2008&corpus=15)
I don't know if the term was being used _primarily_ in a technical context,
but I have little doubt the term was being used to describe something that did
not have a server. In fact I'm just about certain that spike was the term
being used in a tech context. I've found you an example of the term being used
in 1995 (DOI 10.1145/224057.224066):

> _A serverless network file system distributes storage, cache, and control
> over cooperating workstations. This approach contrasts with traditional file
> systems such as Netware, NFS, [...] where a central server machine provides
> all file system services._

This is not precisely the same way that SQLite has used the term. Rather,
they're using the term in an intuitive natural way that fits their context,
just as SQLite used it in 2007.

Here is an example from 2007 (DOI: 10.1109/TNET.2006.886289):

> _Abstract—We explore exploits possible for cheating in real-time,
> multiplayer games for both client-server and serverless architectures._

Here we have the term "serverless" clearly being contrasted with "client-
server", which seems _very_ similar to the way in which that SQLite document
used it.

------
JDiculous
This is not new or unknown information, why is this on the front page?

~~~
klez
Because other people deemed it interesting.

~~~
JDiculous
The fact that SQLite is serverless is common knowledge and not interesting
unless you're brand new to databases

------
jankotek
I think better name (and coined since 1990ies) is __embedded db engine __.
MySQL, Firebird (InterBase) etc can also be linked into binary and run in the
same process, without any type of sockets.

------
k__
Two definitions of serverless?

I heard many, but what they're writing isn't one of them.

As far as I can tell SQLite was always called an embedded database, and never
a serverless one.

~~~
masklinn
The page is more that 10 years old. And it uses "serverless" in the sence of
server-less aka not having a server. Which is pretty damn sensible
terminology, as opposed to the newer "serverless" meaning" runs on some
utility server outside of your care or control".

~~~
k__
I know, but have you anyone calling "embedded databases" "serverless
databases" in the last 10 years?

It's like someone found this page and felt very smart about it, because they
stick it to the serverless crowd.

~~~
SQLite
"embedded database" does not imply "serverless database". Other embedded
RDBMSes run servers; they just run the server as a separate thread rather than
a separate process. SQLite is different in that there is no separate thread of
control. SQLite runs in the same thread as the application that calls it.
There is no separate thread hanging around to clean up or handle background
tasks after an SQLite function call returns.

------
Eikon
> SQLite is an example of a classic serverless database engine. With SQLite,
> there are no other processes, threads, machines, or other mechanisms (apart
> from host computer OS and filesystem) to help provide database services or
> implementation. There really is no server.

Well, it's bending the overall consensus defining serverless as a managed /
and or stateless service.

I never saw anyone using the term "serverless" to mean "embeded".

Using this definition, anything and everything that is not requiring a
specific server to be served / distributed can be described as serverless.

~~~
IAmEveryone
There's a section added in 2018 to deal with the apparent confusion.

It'd be a better idea to just delete the page. It may have been written long
before the meaning changed, but it's pointless to fight a losing and
completely insignificant battle over language.

~~~
chungy
Hard disagree. The "neo-serverless" version has always been extremely
confusing to me. I expect serverless to mean the absence of a server.

In SQLite's particular case, it's subverting the expectation that has
persisted since the beginning of time (of databases) that a database must be
managed by a server.

------
manigandham
Why even write this? It's yet another convoluted use of the term and
meaningless for SQLite of all things.

Re: downvotes - what are people disagreeing with? That the term is not
convoluted? That it's actually useful? That it helps to have more sub-
definitions in an industry known for overloaded terms? I guarantee not a
single person here has used "classic serverless" over "in-process" or
"embedded" in their entire career.

~~~
jamessb
It describes an important architectural feature of SQLite.

Also, this page was first written in 2007 (or perhaps earlier) [1], long
before 'serverless' was applied to things like Amazon Lambda.

[1]:
[https://web.archive.org/web/20071115173112/https://www.sqlit...](https://web.archive.org/web/20071115173112/https://www.sqlite.org/serverless.html)

~~~
manigandham
Adding an original timestamp to the page would've been more helpful than the
updated section, or just deleting the page entirely. Or just title it as
"SQLite is not client/server".

Nobody goes around talking about "classic serverless" instead of "in-process"
which has been around for decades.

