
Show HN: JSONlite – A simple, serverless, zero-configuration JSON document store - nodesocket
https://github.com/nodesocket/jsonlite
======
CJefferson
If you are going to invoke the sacred name of 'sqlite', then I expect to see
rock-solid code and quality testing.

Instead, there are two tiny tests, and I found a couple of issues in 30
seconds of looking at the code.

EDIT: I should prove my point.

1) Race condition on calculating uuid (obviously won't be too serious for a
good uuid implementation)

2) No check that file is written successfully.

3) No check when json is invalid, just silently swallowed, or filesystem full.

4) Makes one file a json file, will scale terribly past a few thousand json
files.

~~~
nodesocket
Good point on 2 and 3. Should just involve checking the previous commands
return code before echoing the uuid. I'll get that patched up and ship a new
version in a bit.

Not sure what you mean exactly by 1 and 4 though.

~~~
CJefferson
With 1, you are trusting the directory returned by uuid will be unique --it
should be, but it might not be, in particular if it is based on time and I run
in two threads.

With 4, I find once a directory has about 100,000 files in it, things get bad,
from the simple (ls * won't work) to the nastier (git starts using huge
amounts of space, you'll hit github's size limit even though all your files
are quite small).

~~~
sz4kerto
I would not let anyone close to the code I'm working on if he wanted to verify
whether a properly generated UUID clashes with some previously generated UUID.

You could just as do stuff like "int x=1; assert(x == 1);" and so on. And it
would be just as futile.

~~~
CJefferson
In this case, you are relying on what installed program or function is called
when you invoke uuid in the shell. It seems reasonable to me to check if it is
having sensibly, if the reliability of your program counts on it. I have no
idea what dodgy uuid programs you might find installed.

------
mtolan
I think your title might not be the best;

* Serverless typically implies something accessible from more than the host you're on

* It's not zero configuration, there's at least 1 configurable parameter (albeit with a sensible default)

* I'm actually not even clear why this is a document store limited to JSON, other than you're piping it though a JSON python module.

Having done some things like this in the past, as you continue, you'll
probably want to create subdirectories based on the first N characters of the
UUID, but I'm not sure you wouldn't get considerably more value of just
putting your json blobs into an AWS Dynamo table.

~~~
edraferi
I agree that "serverless" now mainly means "Function as a Service." The goal
here is convey that there is no server component of the database, similar to
SQLite and unlike PostgreSQL.

What is the correct terminology to highlight this architectural distinction?

~~~
deif
What's wrong with 'local'? Serverless is a terrible word to describe this.

~~~
evanweaver
I always thought they were "embedded" databases.

------
noonespecial
Hint: If you really want to use your filesystem as a database (and don't mind
the haters, there's a long and storied history of doing just this), make sure
you break it up into many subdirs usually based on the first few characters of
the uuid.

Example: ./jasondir/aa/bb/cc/aabbccdd

You won't like what happens when you put 100k files in one directory.

~~~
TheGreatPotatoe
There was an old document management system called Keyfile that did just that.

~~~
manojlds
Git does it too.

------
sigmonsays
This is 200 lines of bash using the file system as a database. The term
document store implies a database which this is not. How is this different
than just writing files to a directory?

What are the use cases of this project?

~~~
pmelendez
"A database is an organized collection of data". I think writing files in a
directory with a convention falls in that category.

[1][https://en.m.wikipedia.org/wiki/Database](https://en.m.wikipedia.org/wiki/Database)

~~~
camus2
The point is the project seems a bit low-effort. The most complicated thing in
that bash script is argument processing.

What is the added value of that project? does it come up with something new or
interesting? a query language for data? an efficient data storage? no, it just
store json files in directories. Sure the file system is already a database,
but that project doesn't add anything to the file system.

------
Waterluvian
I think there is a lot of feedback, mostly constructive criticisms. I would
encourage you to embrace a lot of what everyone is saying and not to take it
personally. You've done far better than most developers: you authored
something and shared it with the world. To this day I'm still too afraid to
show HN or Reddit my work.

------
camus2
Serverless means anything really, a cli app is now serverless. I didn't know I
was writing serverless applications for 20 years then. Serverless was already
a bad buzz word, it gets more and more meaningless by the hour.

By the way sqlite just got support for JSON :

[https://www.sqlite.org/json1.html](https://www.sqlite.org/json1.html)

off-topic : I'd like to see more books on database implementation for
beginners. Of all the crap load of CS books that come out year after year this
is a matter with very little literature, as most papers on the subject are
research papers. Seems like an excellent topic to teach distributed
programming.

~~~
dchest
No, serverless database is not a recent buzz word, but a term frequently used
to describe DB that doesn't need a separate server running to which clients
connect: compare MySQL, PostgreSQL against BDB, LMDB, SQLite.

~~~
camus2
"serverless" means so many things it really means nothing, that's like "cloud
computing" and that's the definition of a buzzword. Sqlite is an embedded
database and has nothing to do with your favorite buzzword.

~~~
dchest
Richard Hipp called SQLite "serverless" long before you started trying to be
cool by hating buzzwords
[https://sqlite.org/serverless.html](https://sqlite.org/serverless.html)

------
PythonicAlpha
I currently don't see the similarity to SQLite, that is implied in the readme.

SQLite stores all its data (every table and all entries in tables) in one
single file. JSONlite does store every JSON entry in one particular file, as
much I saw. So, JSONlite uses the file system (one directory) as data storage
-- in contrast to SQLite!

------
jwilk
> JSONlite sandboxes the current working directory similar to SQLite.

Huh, what does that mean?

------
dunkey123
I don't like that it is just a composition of bash and the filesystem. I'd
just do this:

id=$(uuidgen)

echo '{"hi":"mom"}' | jq . > id

cat id

------
agounaris
The "serverless" starts being used and abused the same way "microservices" and
"rest" are.

------
NelsonMinar
Why not just use SQLite? Its JSON1 extension is pretty good, and then you get
all the rest of SQLite's goodness as part of the package.
[https://sqlite.org/json1.html](https://sqlite.org/json1.html)

------
nerdponx
There's already an R package called jsonlite:
[https://cran.r-project.org/package=jsonlite](https://cran.r-project.org/package=jsonlite)

------
bvinc
Wouldn't you be much better off storing JSON in a SQLite table?

~~~
PretzelFisch
How would that be an improvement? you still wouldn't be able to readily query
and build indexes? Or does SQLite handle that now?

~~~
camus2
[https://www.sqlite.org/json1.html](https://www.sqlite.org/json1.html)

I really don't get why this made it to HN first page, especially when plenty
of other show HN demonstrate way more efforts than writing a short bashscript
yet never make it. I suspect it has to do with the number of buzzwords that
were inserted into the title.

------
drinchev
This reminds me of NodeJS configstore[1].

I would be really happy to learn what are the use-cases for this tool, since
using it via bash is actually an overkill.

1:
[https://github.com/yeoman/configstore](https://github.com/yeoman/configstore)

------
vikingcaffiene
First I want to say this is really cool. Thanks for taking the time to make it
and I'll definitely give a test drive. That said, it's not zero configuration
and your title is kinda misleading.

------
laughfactory
Also the name of a very popular R package. Maybe call it JSONdb? Or JSONstore?

------
Tepix
Cool! Can you add an append function?

------
sscarduzio
lol why is this so high in the front page? BTW this is __NOT SERVERLESS
__therefore click bait in my book

