
ImmortalDB – A resilient key-value store for the browser - tosh
https://github.com/gruns/ImmortalDB
======
air7
While I appreciate the work done I feel this is conceptually problematic and
is somewhat nefarious in nature, just like "evercookie" that it's based on.

> clearing cookies is a common user action, even for non-technical users. And
> browsers will unceremoniously delete IndexedDB, LocalStorage, and/or
> SessionStorage without warning under storage pressure. ImmortalDB is
> resilient in the face of such events.

When a user (especially a non-technical one) clears cookies, their sole
objective is to remove their identity and "start fresh". That's the expected
outcome and this library violates that expectation. Also, the reason the
browser "unceremoniously deletes" some storage areas is because they are not
meant to hold 100% persistent data. So in both cases, the "right" thing to do
is respect the end goal of the action and _not_ restore it.

Furthermore, this library should not be used for sensitive data but likely
will. This is because sensitive data should not be stored in local storage but
rather in cookies [0] as they have better security options with "httponly"
mitigating leaks via XSS and scope narrowing with Domain and Path. Local
Storage is available to any script on the domain including any 3rd party
script in the page.

> Strikes an equitable balance between reliability and respect for the user.
> That's a fancy way of saying "Doesn't fully respect the user".

[0]
[https://www.owasp.org/index.php/HTML5_Security_Cheat_Sheet#L...](https://www.owasp.org/index.php/HTML5_Security_Cheat_Sheet#Local_Storage)

------
bigiain
> Strikes an equitable balance between reliability and respect for the user.

Is it just me, or is that a novel interpretation of "respect for the user"?

"I'm going to continue to track you even after the usual and expected method
of preventing tracking in the browser is used, but if you jump through an
unexpected hoop that's not documented to the user, then I'll grudgingly delete
my tracking data..."

~~~
dspillett
It depends on how the data is used and if an easy delete option exists. There
is a valid use case for working around _browser_ behaviour not user behaviour:

 _> browsers will unceremoniously delete IndexedDB, LocalStorage, and/or
SessionStorage without warning under storage pressure_

If you have an app that stores a small amount of data client-side
(preferences, data to allow operation while unable to connect) but is not used
very often it's stored state could be evicted because of a few large apps
because the browser drops data on a "last used" basis because it has no way of
knowing that this bit of data might be more important than the data touched
more recently. The user might not want a small amount of practical data for an
app that they used on their morning commute being evicted because they've
touched some large games over the weekend. For a large device this isn't going
to be an issue as you are unlikely to be that limited for space, but for a
cheap phone with limited storage it could be a real inconvenience.

Any app doing this should have a very easy and obvious "purge my data from
this device" option, to maintain the user control. In fact to do it properly
right, the local storage should be opt-in ("Do you want to keep some data
locally to speed up this app and allow it to be used offline? Yes / Not now /
No and don't bother be again") with the app continuing to function without the
local data cache.

 _> I'm going to continue to track you even after_

Of course it _will_ get used for this, it would be naive to think otherwise,
it is after all essentially a structured evercookie, but that doesn't stop the
idea having good uses too. The existence of arsonists doesn't mean you need to
ban all uses of fire.

------
eridius
> _And browsers will unceremoniously delete IndexedDB, LocalStorage, and /or
> SessionStorage without warning under storage pressure._

I get why it's doing this, but it strikes me as fairly user-hostile to react
to the browser deleting data under storage pressure by duplicating the data so
it now takes up 4x the space it used to.

~~~
woah
What if it is something small but very important. That seems like it would be
the opposite of user-hostile.

~~~
mosselman
So it is not user-hostile when you don’t respect the user’s attempt to delete
data? It seems condescending at the very least.

------
wtmt
The real world equivalent that I can think of is being stalked even after
moving to a far off location taking care not to leave a trace but
inadvertently left one or two things around.

With many user hostile and invasive solutions, browser makers should take a
stronger stance on clearing all local data simultaneously (cookies, local
storage, IndexedDB, etc.). If someone wants a resilient client side data
store, apps are one solution. Websites shouldn't be doing this or allowed to
do this in a (standalone) browser.

------
coderintherye
This seems...really valuable, mainly for me is the case when we want to store
a value for a user locally but they have browser restrictions preventing it.
Primarily, this is the case with corporate users running within their
environment where the domain admin has misconfigured their PCs and thus
unintentionally disallowed localStorage (I've experienced working with two
large corporations, I literally had to get one of them to send me the laptop
to figure it out cause they didn't think they had disallowed local storage, it
had happened unintentionally).

~~~
saagarjha
> Primarily, this is the case with corporate users running within their
> environment where the domain admin has misconfigured their PCs and thus
> unintentionally disallowed localStorage

Safari in Private Browsing also used to do this too, which happens to
completely break a surprising number of websites.

~~~
madeofpalk
Safari didn't remove or 'break' localStorage, it simply responded with
QuotaExceededError, indicating there's no more space available.

------
musicale
Automatically recreating cookies whenever users delete them is the opposite of
"respect for the user."

~~~
dspillett
I think people are reading this backwards. It isn't just using local storage
and indexeddb to protect cookie values from being wiped (by the user), I can
see it being more useful for using cookies as an extra store to protect
localstorage/indexeddb from being cleared by the browser.

The user can still clear the data manually (clear cookies and other local
data) or through a properly written app (which implements a "remove my records
from this device" option, and preferably makes the local storage opt-in
anyway).

If you are thinking of using this maliciously then you wouldn't use this:
you'd use the already existing evercookie instead as it survives more (at the
expense of potentially imparting more load on the users browser).

------
grun
Hey guys. I'm Ansgar. I built ImmortalDB.

Don't hesitate to let me know if you have any questions or comments about
ImmortalDB. Feedback is how good products become great.

~~~
rufio
what caused you to write this library? do you believe it will be used for
nefarious purposes?

thank you for open sourcing your work

~~~
grun
> what caused you to write this library?

On smaller devices, a few kb of important data in LocalStorage was
periodically deleted without user intervention.

> do you believe it will be used for nefarious purposes?

If one is nefarious of heart,
[https://github.com/samyk/evercookie](https://github.com/samyk/evercookie)
supersedes ImmortalDB.

------
martin_drapeau
What is the storage limit? Would be nice to offer some kind of helper function
to return that. For instance, I want to store a base64 encoded image. I
believe there is a 10MB limit imposed by Chrome on local storage (or something
similar). Bypassing that restriction locally would be great. Could be a nice
feature of ImmortalDB.

~~~
tracker1
That won't fit in cookie space... this is really for redundant controls for
session/access tokens etc... anything large like that you're better off using
caching and a service worker to handle repeated requests. Also, cookies are
sent with every request to the server, and I don't think you'd want the user
sending 10MB to your server for every request.

Also, if you are storing larger strings in local/sessionStorage, websql or
indexdb, you should use lz-string[1] to try to minimize storage impact.

Aside: if you want something with a nicer interface for underlying storage you
may want to look at pouchdb[2], which has a localized couch interface and
supports remote synching to coudchdb, which is pretty nice.

[1] [https://github.com/pieroxy/lz-string](https://github.com/pieroxy/lz-
string) [2] [https://pouchdb.com/](https://pouchdb.com/)

~~~
martin_drapeau
Yeah, about those cookies - I don't want them
([https://github.com/gruns/ImmortalDB/issues/6](https://github.com/gruns/ImmortalDB/issues/6)).
I just want to permanently store large data in the browser. ImmortalDB sounded
like the right thing to abstract away everything and make it sticky.

~~~
dspillett
You can tell it not to use cookies, to get around the problems of them being
more limited in size and being sent on each request so wasting bandwidth.

I'm guessing a mix of localstorage and sessionstorage isn't al that useful (I
can't imagine a circumstance where localstorage is being cleared when
sessionstorage isn't also or hasn't already been) but localstorage+indexeddb
might give you some useful level of protection.

------
applecrazy
Cool project. I wonder how it persists after clearing browser
history/cookies/LocalStorage.

~~~
svdr
It doesn't: "Data is stored reliably but can also be voluntarily purged if the
user designedly clears cookies and application storage".

~~~
applecrazy
Then that’s inconsistent with the beginning of their readme:

> For example, clearing cookies is a common user action, even for non-
> technical users. And browsers will unceremoniously delete IndexedDB,
> LocalStorage, and/or SessionStorage without warning under storage pressure.

> ImmortalDB is resilient in the face of such events.

I guess that’s a good thing. I was worried about the next browser supercookie
being made with a library like this.

~~~
penagwin
It's based on Evercookie
[https://github.com/samyk/evercookie](https://github.com/samyk/evercookie)
which is a little less... friendly.

~~~
vokep
>Its goal is to identify users after they've removed standard cookies and
other privacy data such as Flash cookies (LSOs), HTML5 storage, SilverLight
storage, and others.

AKA

>Its goal is to identify users after they've taken actions to not be
identified

 _sigh_

------
tosh
I just wondered why there is only support for IndexDB and not for WebSQL.
Turns out all modern browsers support IndexDB by now :)

~~~
tracker1
Though websql, where available, is often faster. It's one of the more common
overrides for pouchdb to override the user order in favor of websql where
available.

~~~
barbecue_sauce
Isn't WebSQL deprecated?

~~~
virtualwhys
Yes, and 10 years after its deprecation, WebSQL is still shipping with Safari
and Chrome (worth pointing out that these 2 browsers completely dominate the
market).

Interestingly, neither Apple nor Google were pro-deprecation; Mozilla, and to
some degree Microsoft, spearheaded the move to standardize on IndexedDB.

Moving forward now that Edge is based on Chrome, maybe there's hope for WebSQL
after all. Not that it will be undeprecated, but if 98% of the browser market
supports WebSQL then it's effectively a non-standard standard.

~~~
lmorchard
The "standard" would have basically read "link your browser with SQLite v3.x.x
or an exact bug-compatible clone".

An attempt was made to break down the SQL dialect and exact behavior of that
version of SQLite, but the objection was that it's not really great to reverse
engineer a standard from a single vendor implementation even if that vendor's
product is open source. (eg. what happens when sqlite changes? do browsers
stay fixed on an ancient exact version?)

------
recursive
Can anyone explain why there's a leading semi-colon in the inline script
example? AFAICT it doesn't do anything.

~~~
daleharvey
Some people for some reason write javascript without semicolons, if they do
that you cant safely concatenate javascript files together since the file2 may
modify file1's last expression, putting a ; at the start guarantees to
complete the last files expression if they havent already done so. A bunch of
concatenators will do this automatically / by default

~~~
jacobwg
You are correct that this is an issue when writing JavaScript without
semicolons, but it's also an issue if you place an IIFE (immediately invoked
function expression) directly after a function, even if you're using
semicolons:

    
    
      function hasSemicolons() {
        console.log('hello world');
      }
    
      (function() {
        console.log('this is broken, JS will call hasSemicolons()');
      })()
    

In the ImmortalDB example, since it's a standalone <script> tag this is just
extra defensive against a minifier that would join the tag with other tags,
but just wanted to call out that there's a reason to include the leading
semicolon in front of an IIFE whether or not you write your JavaScript with
semicolons.

See also: [https://stackoverflow.com/questions/1873983/what-does-the-
le...](https://stackoverflow.com/questions/1873983/what-does-the-leading-
semicolon-in-javascript-libraries-do)

~~~
recursive
So I tested this out in Firefox. I think you're wrong. I never see 'hello
world' even if I remove all the whitespace between the functions.

------
platform
nice.

I assume, still, it should not be used to store JTW refresh tokens, or
anything where secure storage is required.

~~~
geofft
I don't see why not - at least if you're comfortable with client-side code on
the website having access (implicit or explicit) to the token. Certainly
nothing you store on the client side is secure against an untrusted user, so
you should only store the user's own secrets, not things you wish to keep
secret from the user. And anything you intend to make _use_ of fro the client
side must be usable from within the website, so anyone who can get code
execution on the website (e.g., via XSS) has already gained access to whatever
you're trying to protect.

I see a handful of blog posts on the internet arguing that HttpOnly cookies
are a more secure storage place than web storage, because HttpOnly cookies are
only vulnerable to CSRF attacks and not XSS and CSRF is easier to defend
against. This argument is, as far as I can tell, completely flawed: if an
attacker is able to execute XSS, they're then able to use that XSS to execute
same-site forged requests - they no longer have to bother with the "cross-
site" part of CSRF. They can just fetch your anti-CSRF token with a normal XHR
after having executed an XSS, and then they can make arbitrary same-origin
requests against your website, each of which gets sent with the HttpOnly
cookie.

~~~
platform
JWT refresh tokens are equivalent to passwords. (they are active for days at a
time).

I do not know of many web apps that store passwords in browser storage.

Unlike passwords, JWT refresh tokens are created programmatically (not entered
by user interactively).

Perhaps,I am just missing a link, but I though browsers do not allow an
application to programmatically store passwords securely

~~~
api_or_ipa
If you don't persist JWT in localStorage or sessionStorage, how would you
handle page reloads? You'd have to rely on a session cookie and get a new JWT
from your auth service. You'd solve XSS but get CSRF.

Storing a JWT in localstore isn't the same as storing a password in plain
text. JWTs are indeed non-revocable, but they are also non-durable (ours
expire after 24 hours) and cannot be renewed without a refreshToken. Most
people also do not rely on JWTs to write important information (like password
reset) because there are more secure alternatives for these specific use
cases.

As an aside, if someone gets script access to your SPA, you're already very
far up shit creek. If your user types in any sensitive information like a
password to login you've lost the war. Securing the JWT is the very least of
your concerns at that point.

