
Show HN: IronDB – a resilient key-value store for the browser - grun
https://github.com/gruns/irondb
======
bazizbaziz
> When a value is retrieved via its key, IronDB... > Looks up that key in
> every store. > Counts each unique, returned value. > Determines the most
> commonly returned value as the 'correct' value. > Returns this most common
> correct value > Then IronDB self heals: if any store(s) returned a value
> different than the determined correct value, or no value at all, the correct
> value is rewritten to that store. In this way, consensus, reliability, and
> redundancy is maintained.

I'm not sure any of these properties are ensured the way anyone would want.
Why do this? What are the failure modes this protects against? If only one of
the backing stores is active when a result is written, and then the others
later become active with no data, is blank data returned for the prior result?
It seems like recording timestamps would fix this problem nicely on a single
system and make this thing overall quite reliable.

~~~
grun
Cookies are frequently cleared by users, and the other datastores --
IndexedDB, LocalStorage, and SessionStorage -- can be unceremoniously purged
by the browser under storage pressure.

See my other comment, here:
[https://news.ycombinator.com/item?id=18297177](https://news.ycombinator.com/item?id=18297177)

I'll explicate this in the documentation. Thank you for your feedback,
bazizbaziz.

~~~
bazizbaziz
Thanks for the response! FWIW I'm overall really interested in this, as I
maintain an application that uses localStorage to keep very important data
while the app is offline until it can be uploaded.

I recently tested localForage [0] to see if it could be more reliable and get
around storage limitations of localStorage. Unfortunately, LocalForage has a
very annoying problem where it picks a _single_ storage backend to use, but
the one it chooses can switch on page reloads (yes, even on the same
device/browser.) and this switching causes data loss as keys stored in the
other backends are unaccessible. I'm very interested to see if IronDB can help
here! Thanks for working on this.

[0]
[https://github.com/localForage/localForage](https://github.com/localForage/localForage)

~~~
grun
Wonderful.

Don't hesitate to let me know if there's anything else I can help with,
bazizbaziz.

------
gaimrox
Confused. Why would you name your K/V database irondb when there is already a
time-series database called exactly that?

Think this is their site: [https://www.irondb.io](https://www.irondb.io)

~~~
thanatos_dem
Yup, I was wondering how the hell a timeseries database would run in the
browser.

Nope, this is just syntactic sugar on top of calling (up to) 4 JavaScript
functions.

------
kreetx
Maybe this is useful tool, but several things strike as weird to me:

\- save all values everywhere: why not select the one that would last the
longest (that people delete least often)? Or maybe two, if you want a cookie
but also want to survive over "delete all cookies". But then again, if a user
wants to delete cookies then you really shouldn't try to keep it.

\- "Data is stored resiliently but can also be voluntarily purged if the user
designedly clears cookies and application storage." So you delete data when
the user wants to delete data. Which database doesn't have this feature?

\- doesn't use Flash, Silverlight, or Java -- which do?

\- "healing" is usually used in the context of distributed computing, but here
it means that it gets around user wanting to delete stuff from it's browser,
but perhaps isn't in-the-know enough to delete everything but just the
cookies.

No offence to the author but all this sound malicious. Perhaps this is why
evercookie isn't maintained?

~~~
grun
> save all values everywhere: why not select the one that would last the
> longest (that people delete least often)?

For improved resiliency.

You can also configure IronDB to only use any two datastores of your choice,
if you so desire: IronStorage's constructor takes an Array of storage
implementations of your choice. See
[https://github.com/gruns/irondb#api](https://github.com/gruns/irondb#api).

> So you delete data when the user wants to delete data. Which database
> doesn't have this feature?

A database where the data therein can be deleted without warning. Browsers
unceremoniously delete IndexedDB, LocalStorage, and SessionStorage under
storage pressure. See [https://developers.google.com/web/fundamentals/instant-
and-o...](https://developers.google.com/web/fundamentals/instant-and-
offline/web-storage/offline-for-pwa#how_does_cache_eviction_work)

> doesn't use Flash, Silverlight, or Java -- which do?

Evercookie, a similar library, uses Flash, Silverlight, and/or Java.

See
[https://github.com/samyk/evercookie](https://github.com/samyk/evercookie).

> No offence to the author but all this sound malicious. Perhaps this is why
> evercookie isn't maintained?

No offense taken.

Thank you for your feedback, kreetx. Don't hesitate to let me know if I can
answer any other questions, or if there's anything else I can do for you.

------
tptacek
It's not clear to me how this is more privacy-respecting than Evercookies,
except that it doesn't use browser extensions.

~~~
grun
Evercookie's goal is to store data indelibly, regardless of user intention.

IronDB's goal is to store data reliably, e.g. in the face of storage eviction,
but not against user intention. If the user clears all browsing data, that
willful action is respected.

Thank you for your feedback. I'll clarify this in the docs.

~~~
laurent123456
Why do we need a library to store data reliably? Aren't browsers API already
reliable? Does the LocalStorage API for example sometimes fails to store its
data properly?

~~~
grun
Clearing cookies is a common user action. Even for relatively non-technical
users.

And under storage pressure, browsers evict data stored in IndexedDB,
LocalStorage, and/or SessionStorage. See

[https://developers.google.com/web/fundamentals/instant-
and-o...](https://developers.google.com/web/fundamentals/instant-and-
offline/web-storage/offline-for-pwa#how_does_cache_eviction_work)

IronDB is resilient in the face of such events.

Thank you for your feedback. I'll add an explanation about such in the
documentation.

~~~
staticvar
If anyone could point to an up to date docs around storage eviction in
browsers it would be greatly appreciated. It's been a big hit to our team's
confidence in the "use the platform" mentality that this is not accurately
documented.

~~~
edoceo
I remember reading a 5MB limit some place (ages ago) and I'm not able to find
the reference at the time.

My team has been assuming that is true (for local,session)Storage and so far
haven't been bitten.

Only occasionally have we even gotten close to that size. You can also shove
loads of data in the DOM.

Edit: one of the old refs - 10MB

[https://www.html5rocks.com/en/tutorials/offline/quota-
resear...](https://www.html5rocks.com/en/tutorials/offline/quota-research/)

------
proneb1rd
What’s the point of storing the same data in three distinct storages? What
kind of disaster are you trying to avert?

ps: this will back fire real bad when safari and friends add permission
prompts for each specific storage that this library fills up in a for loop.

~~~
latchkey
The usecase I've seen more recently is that paywall news sites now store the
number of page reads in multiple places. You end up having to clear more than
just your cookies to 'reset' the read count.

------
ggregoire
What are the differences with localForage (or similar solutions)? The fallback
to cookies if no localStorage?

[https://github.com/localForage/localForage](https://github.com/localForage/localForage)

~~~
grun
localForage only uses a single datastore. E.g. LocalStorage or SessionStorage
or IndexedDB. Thus, stored data is lost forever if the browser unceremoniously
purges that datastore due to storage pressure.

[https://developers.google.com/web/fundamentals/instant-
and-o...](https://developers.google.com/web/fundamentals/instant-and-
offline/web-storage/offline-for-pwa#how_does_cache_eviction_work)

IronDB redundantly stores data in multiple datastores -- cookies, IndexedDB,
LocalStorage, and SessionStorage -- and self-heals if the data in any thereof
is deleted or corrupted. For example, stored data is safeguarded in the event
the user clears their cookies or IndexedDB is purged due to storage pressure.

tl;dr: IronDB is resilient in the face of data deletion. localForage isn't.

~~~
lgessler
It's kind of ridiculous that a library needs to exist to guard against this
browser behavior, although I empathize with the browser vendors' argument,
which is that browser storage should essentially be used for caching and that
browsers should somewhat aggressively defend against disk space consumption
that most users don't even know about.

It would be nice to have an in-between where if a user authorized an app to
permanently consume up to X amount of disk space the browser would allow it.
But pulling off that user interaction well (i.e., getting the user to make an
informed decision rather than blindly clicking yes or no) seems quite hard.

