
Local-first software: you own your data, in spite of the cloud - godelmachine
https://blog.acolyer.org/2019/11/20/local-first-software/
======
herf
I spent a lot of time with photos (Picasa) trying to do this peer to peer -
this is what we built in the 2002 era. Here are a few issues:

1\. Identity is hard to do on the LAN, so any sharing ends up using the cloud
to figure out who has access. Similarly, identity is hard to move around, so
representing your Facebook comments feed outside Facebook is difficult to do.

2\. Any time you have a "special" server that handles identity, merging, or
any other task, it ends up in the _effective_ role of master, even if the rest
of the parts of your system are peers. You want all your collaborations in
Dropbox to survive their infrastructure being offline? It's tough to do.

3\. p2p stalled a bit in the mid-2000s when storage and bandwidth got much
cheaper--in a period of just two years (2002-2004), it became 100x cheaper to
run a cloud service. But what continued to stall p2p was mobile. Uploading and
syncing needs to run in the background, and if you're on a limited bandwidth
client or a battery-limited device like iOS, sync can effectively diverge for
months because the applications can't run in the background. So changes you
"thought" you made don't show up on other devices.

4\. For avoiding mass surveillance, what we are missing from this older time
is the ability to make point to point connections (between peers) and encrypt
them with forward secrecy, without data at rest in the cloud. Even systems
that try to do some encryption for data at rest (e.g., iMessage) keep keys
essentially forever, so data can be decrypted if you recover a private key
later on. A system that only makes direct connections between peers does not
have this issue.

5\. Anytime you have multiple peers, you have to worry about old versions
(bugs), or even attackers in the network, so it's fundamentally harder than
having a shared server that's always up to date and running the latest code.

~~~
koheripbal
With regards to identity management, maybe there should be a formalized
integration between browsers and password managers such that the concept of
"registration" goes away and new logins just automatically create user
accounts with default permissions, according to email address.

~~~
mceachen
Centralized identity management was the grand promise of OAuth. It seems that
somehow ended up, in most practical applications, as a "login with Facebook"
button.

There are a number of startups working on simplified or "passwordless" auth,
but it seems that none have substantive traction. I'd love to be proven wrong,
here, though!

~~~
bdcravens
Before that there was Microsoft Passport. Federated login for the web is a
“problem” various parties have been working on for 20 years.

~~~
jandrese
There's a fundamental problem of "I want my credentials to be validated by
some central service, but I don't want to give some big faceless company my
information." The centralized login service can be used to track your activity
all across the net. It's a lot of power to give to some people you don't know.

Worse, people's fears in this area are completely justified. Precious few
companies have proven themselves to be contentious with our personal data.
Some go as far as to repackage and outright sell said data for personal gain.

A decentralized blockchain-like system might work for this, but as far as I
know it has not been attempted.

~~~
dane-pgp
Actually, the problem of centralized login services tracking users across the
net was considered and partially solved in 2010 by a team from Google and MIT.
This was at a time when OpenID was the most popular federated login system, so
the proposal was called PseudoID.

Unfortunately this idea didn't gain more traction, and there are only a few
references to it online now, such as this research paper and a YouTube video
by one of the authors:

[https://ai.google/research/pubs/pub36553](https://ai.google/research/pubs/pub36553)

[https://www.youtube.com/watch?v=fCBPuGsO_I4](https://www.youtube.com/watch?v=fCBPuGsO_I4)

Also, it didn't address all the methods by which a malicious identity provider
could track the user, so it would probably have to be extended by having
support added in the browser.

------
jka
As far as possible, I'm following a local-first methodology for a recipe
search, meal planner, and shopping list application:

[https://www.reciperadar.com](https://www.reciperadar.com)

There's a 'collaboration' mode which allows peer-to-peer sharing of a session
via CRDTs over IPFS. My partner and I select our meals for the week, and then
when one of us is doing the shopping, we can mark ingredients as found -- the
other person's view reflects those updates in near-real-time.

If either of us lose connectivity, we can continue to use the app, and when
data access is restored, those changes are synced with the shared session
(with automatic conflict resolution). All data in the shared session is
encrypted, and the collaboration link contains the keys.

Much of this functionality is thanks to peer-base, which is an experimental
but extremely useful library:

[https://github.com/peer-base/peer-base](https://github.com/peer-base/peer-
base)

A side-benefit of this approach is that _all_ user data can be stored locally
(in browser localStorage) - there are no cookies used in communication between
the app and server.

~~~
jimpick
(peer-base maintainer here)

This is really cool!

I found the share link, and just tried it between two Chrome browsers, and it
worked great!

Thanks for using peer-base! There's a lot of great work happening with js-
libp2p recently that would be awesome to incorporate ... I'm hoping to get
active developing on it again in the new year. I've got so many ideas for
improvements.

~~~
jka
Thanks a ton for developing the library :)

If & when you're looking for any more contributors and/or testing, let me
know; I'd be glad to pay it back.

I'm hoping to open source the reciperadar application's stack soon and happy
for it to be part of any ecosystem of examples (jbenet's _peer-chess_ was a
big help to me getting started).

------
jmathai
I've been trying to document my "local-first" approach to managing photos.
I've made it a ways through but am not sure when I'll finish. Posting here
since it is relevant.

A Pragmatic Photo Archiving Solution:
[https://docs.google.com/document/d/1JzqT-
DJFlS2e8ZC00HrsQITq...](https://docs.google.com/document/d/1JzqT-
DJFlS2e8ZC00HrsQITq_ukORcH9icXTGKeWQYU/edit?usp=sharing)

It's the culmination of software I've written [1] + a workflow that's resulted
from it [2, 3, 4, 5].

[1] Elodie -
[https://github.com/jmathai/elodie](https://github.com/jmathai/elodie)

[2] Understanding my need for an automated photo workflow -
[https://medium.com/vantage/understanding-my-need-for-an-
auto...](https://medium.com/vantage/understanding-my-need-for-an-automated-
photo-workflow-a2ff95b46f8f)

[3] Introducing Elodie; Your Personal EXIF-based Photo and Video Assistant -
[https://medium.com/vantage/understanding-my-need-for-an-
auto...](https://medium.com/vantage/understanding-my-need-for-an-automated-
photo-workflow-a2ff95b46f8f)

[4] My Automated Photo Workflow using Google Photos and Elodie -
[https://medium.com/@jmathai/my-automated-photo-workflow-
usin...](https://medium.com/@jmathai/my-automated-photo-workflow-using-google-
photos-and-elodie-afb753b8c724)

[5] One Year of Using an Automated Photo Organization and Archiving Workflow -
[https://artplusmarketing.com/one-year-of-using-an-
automated-...](https://artplusmarketing.com/one-year-of-using-an-automated-
photo-organization-and-archiving-workflow-89cf9ad7bddf)

~~~
neLrivVK
Just read through your google doc, interesting! But what about additional
family members, with their own cameras, and no interest in any clever workflow
activities :) I'm currently using Google Photos as my main service, and it's
working good enough for now: each family member has the Google Photos app
which uploads pics automatically, to their own account. We all share our
Google Photos with each other. This way I (as main curator) have access to
everyone's pics, without anyone having to do anything. Google lets you store
the original size pics, so that is great (not like iCloud that resizes all
pics!). Google also adds face recognition, which is _very_ practical, and also
provides a good interface for everyone to view the pictures. Regarding
safekeeping: I use the Google Drive interface to backup all my photos to my
local linux storage (combination of rsync and
[https://github.com/astrada/google-drive-
ocamlfuse](https://github.com/astrada/google-drive-ocamlfuse) to mount Google
Drive). This way I always have all original photos locally. Finally I backup
everything offsite using BackBlaze.

All this relies heavily on Google Photos, but I have my own local backup of
all original files. So if I need to change service, it should just be a one-
time effort to migrate.

~~~
emilburzo
> I use the Google Drive interface to backup all my photos to my local linux
> storage

I thought this was killed recently[1]

(this = google photos appearing in google drive)

Does it still work for you?

If yes, how? :)

[1] [https://www.blog.google/products/photos/simplifying-
google-p...](https://www.blog.google/products/photos/simplifying-google-
photos-and-google-drive/)

~~~
jmathai
Uploading to both services is still supported through the Backup and Sync app.
But once uploaded they are independent copies and deleting from one doesn't
delete from the other. I also expect that this support will be deprecated at
some point in the future.

Not keeping Drive and Photos in sync really killed it for me. I ended up
switching from Google Drive to Dropbox but I still use Google Photos.

I have photos added to my library in Dropbox automatically added to my Google
Photos library and this has worked well so far. [1]

[1]
[https://github.com/jmathai/elodie/tree/75e65901a94e14e6fd1ff...](https://github.com/jmathai/elodie/tree/75e65901a94e14e6fd1ff7ad7a6e0f1c42edccc4/elodie/plugins/googlephotos)

~~~
emilburzo
I see. I was hoping there was still some way to quickly backup from google
photos directly to linux.

~~~
mceachen
You can use
[https://github.com/mholt/timeliner](https://github.com/mholt/timeliner) (open
source)

------
atoav
When I select software these are among the list of things I am looking for
generally:

\- file formats that won’t lock you in or are even openly hackable (allows you
to automate things)

\- no clouds that will break the software once it is gone

\- local storage with custom syncing or backup options

\- strictly no weird data collection or “We own the rights to your data”-Type
of terms

So if I get the slightest feeling of a _lock in_ or unnecessary data
collection you are scaring me away, because mentally I would then already look
at the time after you decide to scrap your cloud or abandon your file formats.
The data collection bit shows me your users aren’t front and center but
something else is which makes your product even less of a good choice.

If your product runs on the web, allowing for self-hosted solutions is also a
big plus.

~~~
jwr
While I fully agree with your selection criteria, please consider the other
side of the equation, because engineering (and the world) is all about
compromises.

I am the author of a SaaS app ([https://partsbox.io/](https://partsbox.io/)).
I export in open formats (JSON), there is no lock-in, it's easy to get all of
your data at any time. But the app is online and will remain so. Why?
Economics. Maintaining a self-hosted solution is an enormous cost, which most
people forget about. You need to create, document, maintain and support an
entirely different version of your software (single-user, no
billing/invoicing, different email sending, different rights system, etc). And
then every time you change something in your software you have to worry about
migrations, not just in your database, but in all your clients databases.

I am not saying it's impossible, it's just expensive, especially for companies
which are built to be sustainable in the first place (e.g. not VC-funded).
Believe me, if you don't have VC money to burn, you will not be experimenting
with CRDTs and synchronizing distributed data from a multitude of versions of
your application.

I regularly have to explain why there is no on-premises version of my app. The
best part is that many people think that an on-premises version should be
_less expensive_ than the online version, and come without a subscription.

~~~
gdfasfklshg4
Worth a thought. How does desktop software achieve backwards compatibility?
For example Libreoffice can work with arbitrary datastores from the 1990's.
Meanwhile with modern web based software we struggle to maintain compatibility
within a single datastore.

~~~
fonosip
We are working on a self-hosted Nextcloud, Onlyoffice solution. Specs here
[https://ba.net/office](https://ba.net/office)

------
mbalex99
Martin Kleppmann is a major inspiration for our startup, Ditto.

We take the local-first concept and p2p to the next level with CRDTs and
replication. But what we really do is leverage things like AWDL, mDNS, and or
Bluetooth low energy to sync mobile databases instances with each other even
without internet connectivity. www.ditto.live

Check it out in action!

[https://youtu.be/1P2bKEJjdec](https://youtu.be/1P2bKEJjdec)
[https://youtu.be/ITUrk_rjnvo](https://youtu.be/ITUrk_rjnvo)

We found that CRDTs, local first, and eventual consistency REALLY shines in
the mobile phones since they constantly experience network partitions.

~~~
Jyaif
Very interesting. Will the upcoming server support be end-to-end encrypted? In
other words, will the server be able to read the data?

------
radium3d
[https://github.com/syncthing/syncthing](https://github.com/syncthing/syncthing)

Syncthing solves a large part of syncing data between devices using your own
VPS, server(s), etc. If your VPS provider goes out of business, you can then
just fire up a new VPS and hook it back up to your local machine(s).

~~~
Fnoord
Cryptomator [1]. Cross-platform, allows you to encrypt your data in the cloud,
and access it transparently.

Thing is, like Syncthing, it lacks a collaborative feature. Nextcloud has it,
but only if you have the Nextcloud accessible (I want to host only on LAN).
Something like IPFS (or Tor) is a solution to such problem.

[1] [https://cryptomator.org/](https://cryptomator.org/)

~~~
StavrosK
> it lacks a collaborative feature

It does not, you can share folders with anyone you want without them even
needing an account.

------
daleharvey
I may be biased as the maintainer of PouchDB but you can do all this today
(and for the last 5+ years) with PouchDB.

The comment about CouchDB and the "difficulty of getting application-level
conflict resolution right" I am not really certain how it applies, You dont
have to handle conflicts in Pouch/CouchDB if you dont want to, there is a
default model of last write (actually most edits) wins but you can handle them
if needed

~~~
adamwiggins
Hi Dale, I'm one of the local-first paper coauthors. I'm a fan of PouchDB
(thanks for that) and the whole CouchDB lineage--the CouchDB book[1] was an
early inspiration in my exploration of next-gen storage and collab models.

I've been down the CouchDB/PouchDB path several times with several different
engineering teams. Every time we were hopeful and every time we just couldn't
get it to work.

As one example, I worked with a small team of engineers to implement CouchDB
syncing for the Clue Android and iOS mobile apps a few years back. Some of my
experience is written down here[2]. After investing many months of engineering
time, including some onsite help from Jan Lehnardt[3] we abandoned this
architecture and went with a classic HTTPS/REST API.

Other times and with different teams we've tried variations of Couch including
PouchDB with web stack technology including Electron and PWA-ish HTML apps.
None of these panned out either. Wish I could give better insights on why--we
just can't get it to be reliable, or fast, or find a good way to share data
between two users (the collaboration thing is kind of the whole point).

[1]: [https://guide.couchdb.org/](https://guide.couchdb.org/)

[2]: [https://medium.com/wandering-cto/mobile-syncing-with-
couchba...](https://medium.com/wandering-cto/mobile-syncing-with-
couchbase-6f076d8c7e08)

[3]: [https://neighbourhood.ie/couchdb-
support/](https://neighbourhood.ie/couchdb-support/)

~~~
emilecantin
I'm currently working on an app using PouchDB, and the approach I've taken is
using one database per "project" in my app. I'm not there yet, but I'll use
another database to manage users & access control. These aren't things you
want to sync anyway; I might even end up with a regular SQL database for this
(I haven't decided yet).

I hope this approach avoids most of the pitfalls you mentioned.

Your Git analogy is also spot-on, but I think you don't take it far enough.
Creating a repo is cheap, and I believe CouchDB databases are, too (altough
I'm still very new at this). You seemed hesitant to create too many.

Good point about notifications, though. I think you'll still have to have a
server process that manages that kind of thing (and probably inserts
notifications in other users's databases).

------
have_faith
> you OWN YOUR data

Like most people here I'm fairly hard line when it comes to personal data
abuses but I still struggle with the concepts of _owning_ data about yourself.
It's a confusion I see amongst less technically literate people when a well
meaning person explains to them the importance of some latest data breach and
they try to understand the concept that they owned this data, it was theirs
but now it has been "stolen" or abused in some way.

I would consider going as far to say that framing the data as owned by you is
a bad approach, but maybe I'm just being pedantic about the language. Company
A does have data about me, but I don't own it, and they have responsibilities
to protect it (or delete it if requested), but I don't see any ownership in
the equation, especially when the nature of the data can become quite abstract
while still maintaining some reference to _you_.

Not to take away from the intention or sentiment of framing it that way
though, I'm just musing.

~~~
cannonedhamster
The problem fundamentally is yes it's your data. It's what makes you, you. The
reason this is important is because if you can't control this data it can be
used against you. At some point in the near future if not already you'll have
job applications rejected based on third party data you didn't sign up for and
can't opt out of. It will impact your healthcare, your loans, what prices you
see when you shop and what items are shown. In essence what makes you, you is
used solely for controlling you and the options available to you.

~~~
infinite8s
The issue then isn't the data, but the fact that it could be used in those
ways. Because if it's legal for companies to use that data (even if it's
owned/controlled by the user), they will incentivize people to share it in
order to give them better prices/service, and the end result will be the same.

------
ricg
_" It should support data access for all time."_ \- This is key for me after I
had to convert my notes more than once between formats after the original
app(s) went into extinction (beloved Circus Ponies Notebook).

That's why I'm designing any new apps around a file format that can be
accessed even without the app.

I have a "local-first" Kanban/Trello-style app, "Boards"
([http://kitestack.com/boards/](http://kitestack.com/boards/)), that uses
zipped HTML files (to support rich text with images). No collaboration and
cross-device support just yet, but it works without a network and saves
everything locally.

~~~
threatripper
Boards seems to be Mac only.

"A new Mac app to boost your productivity in school, at work, and for personal
projects.", "For macOS 10.14 Mojave and later"

~~~
ricg
Yes, currently Mac-only, no cross-device support yet, but an iOS app is in
planning.

------
milansuk
>It should be fast. We don’t want to make round-trips to a server to interact
with the application.

The cloud apps are not slow only because of moving data, but there is also a
problem that an average server is fast(16cores CPU + 64GB RAM), but If it's
used by let's say 100users, It means one user has only 0.16core + 0.64GB
memory. So an average laptop(4cores/4GB) or phone(4cores/1GB) is way faster.
Basically people buy billions of transistors to use them only as a terminal to
the cloud. Not to mention privacy risks.

A week ago, I did showHN for skyalt.com. It's a local accessible database(+
analytics, which is coming soon). I'm still blown away how fast it is, that
you can put tens of millions of rows to single table with many columns on
consumer hardware and you don't pay for scale or attachments.

~~~
jdnenej
That's not a fair comparison since most of the memory usage is just loading
the app in to memory and then everyone is sharing the same app already loaded.
Web apps don't have to be nearly as slow as they are. It's just that it's
easier to make a slow app than a fast one. Also desktop apps are becoming
super slow and bloated now thanks to electron.

~~~
vpEfljFL
> Also desktop apps are becoming super slow and bloated now thanks to
> electron.

I can't quite get this point. From my perspective software engineers
love/adore electron applications.

Look at VScode as the example:

\- electron based javascript application

\- telemetry included

\- proprietary build with "open core"

It is literally the most popular code editor right now (p.s. I don't use it).
Why as a tech savvy user you will use something you don't like for 5-10 hours
each day to do your work?

Only answer I can see that electron is not an issue here.

~~~
thu2111
Obvious answer: it's the only IDE-like thing that's both well supported and
caters to the huge population of JavaScript-only developers, who generally
don't want to use IntelliJ or similar products because they can't customise it
using only web-stack skills.

That is, if JetBrains had made JS plugins first class citizens of their
products, possibly VS Code wouldn't be as popular as it is.

~~~
jdnenej
I use vscode even though I don't particularly like it. Imo rubymine is a
better editor but vscode does 90% of the job while not costing $300/year.

~~~
thu2111
I'm curious where you live. Is $300/year a big expense for you? Computers and
developer salaries are usually much higher.

------
gobengo
The blog post doesn't mention this, so I thought I'd point this out. One of
the paper's authors is Martin Kleppmann, who wrote the very good
[https://dataintensive.net/](https://dataintensive.net/) book.

~~~
ignoramous
Kleppmann co-created/major contributor to Apache Kafka along with Jay Kreps
and Neha Narkhede. He also co-founded Rapportive, a YC company acquired by
LinkedIn, along with Rahul Vohra, who is presently the CEO of Superhuman.

------
ninkendo
Nearly all of Apple’s first party software works this way.

Notes, Reminders, Pages, Numbers, Keynote, Voice memos, etc

All using iCloud APIs to synchronize what is essentially local-first software.

You could even count Mail, Contacts, and Calendar, although they rely on more
established protocols to sync.

~~~
davecap1
Apple Health and Activity also seem to work this way, although they also sync
to iCloud.

------
TAForObvReasons
Today's SaaS world is largely economically opposed to the idea of data
ownership. It's a lot easier to make money by renting people access to their
data.

The problem is not inherently technical. The solution must address the fact
that the software businesses favor cloud solutions and other systems that make
it difficult to stop spending money

~~~
quickthrower2
Yes. I always think data freedom is more important than software freedom. For
example it matters less that MS Word is not free as in freedom when you can
open the file in something that is.

~~~
TeMPOraL
And cloud SaaS are enemies of both! After all, in order to have data freedom,
you need to have _something_ to open in that other program, and cloud
solutions do their best to not give you proper open/save file (and even if
there's an export function, and even if everything is actually included in the
export, it often isn't followed by import function that could read that
export).

For this reason, I avoid using cloud SaaS for anything where can avoid it.

~~~
quickthrower2
Absolutely. There are some exceptions though. Github is the obvious one.
Dropbox by nature has your data constantly exported. Google for its sins has
Google takeout.

~~~
TeMPOraL
Yes. And the first two I use, treating them mostly as dumb pieces of
infrastructure. Arguably, the functionality they provide have a crucial ops
component that I'm all for paying for someone else to handle it for me. But
neither Github nor Dropbox locks me into anything.

Google - yes, web e-mail obviously is similar to the above; as for their
office suite, I recently found a good excuse to justify shelling out for a
proper Microsoft Office subscription (though I don't like that it's a
subscription), and I stick to using the faster, locally-available, file-using,
much more powerful (if still proprietary) software.

------
brynb
I've been working for a few months on a database called Redwood that's
intended to make it easier to build this kind of software. Having spent much
of the past couple of years working with libp2p, IPFS, Dat, and similar
technologies, I was curious to see what would result if I started from the
ground up.

[https://github.com/brynbellomy/redwood](https://github.com/brynbellomy/redwood)

So far, the model seems promising. It's fully peer-to-peer, and supports
decentralized identity, configurable conflict resolution, read/write access,
asset storage, and currently is running across 3 different transports:

\- HTTP (augmented with extensions proposed by the Braid project [1][2])

\- Libp2p

\- WebRTC

I've included two simple demos, a collaborative document editor (well, it's
just a textarea at the moment), and a chat app. Would appreciate any feedback
or participation that folks are willing to give.

[1] [https://github.com/braid-work/braid-spec](https://github.com/braid-
work/braid-spec)

[2] [https://groups.google.com/forum/#!forum/braid-
http](https://groups.google.com/forum/#!forum/braid-http)

------
krzepah
Hi everyone, I've been working on this subject for a few months already ;

Thank you OP, your work is wonderful to read and even though I've spent a few
months on the idea already I haven't thought of reusing Dropbox or similar. I
think exciting things are about to come :)

I'd like to submit Working Group proposal to the IETF.

Why would we need an RPC for Independent Apps?

Independent Apps are surfacing as being a solution to the lack of control of
our own data. oAuth Framework has allowed a more secure web, but even if it
makes a difference between an identity provider and a resource host, it does
confuse the resource and the service hosts.

Independent Apps should NOT be claimed by a lone company, let's make it
something that the web owns.

How would it be structured?

I personally believe there should be multiple subjects treated by the IWA
Framework, as one being the qualities of independent apps, and second being
how data is accessed. Both of these are currently Topics of Interest for the
IETF : [https://ietf.org/topics/](https://ietf.org/topics/) \- However the way
this Working Group would proceed should be discussed and decided by it's
members.

Why not submit a single person draft?

I could propose a draft but it wouldn't have the same meaning as if it would
be drafted by a Working Group. As individuals, we are motivated by our own
agenda and the quality of said draft wouldn't be the same. I'm volunteering,
but I'd like to allow other persons to join in as well.

You can join your mails here
[https://forms.gle/igNdd6rH4MnPK8rb8](https://forms.gle/igNdd6rH4MnPK8rb8) ,
at December 6 I will send the Working Group proposal to the IETF with gathered
persons, if accepted I believe it should remain open to anybody to join.

~~~
brynb
You should check out the Braid project. We're already working on IETF drafts
for a protocol of this nature within the HTTP working group.

\- [https://github.com/braid-work/braid-spec](https://github.com/braid-
work/braid-spec)

\- [https://groups.google.com/forum/#!forum/braid-
http](https://groups.google.com/forum/#!forum/braid-http)

------
thawaway1837
Isn’t Office 365 the platonic ideal of a local first software (suite) by this
definition?

High quality desktop apps, data saved in discrete documented file formats,
optional ability to save in the cloud, the presence of collaborative editing,
privacy is protected if you’re using it locally only, etc.

------
mwilcox
any marginally successful "local-first" app is going to go and raise $10m in
vc, switch to software as a service, and add an enterprise mode that requires
user permissions and data access to be managed on the server

~~~
LeftHandPath
Sure, any one company probably will - but there’s a whole market.

As soon as that one company abandons the local-first model, a gap opens, which
will (usually, eventually) be filled by a new company offering local-first
until that new company does the same.

As long as the companies don’t band together and agree to end it, there should
be a company offering that model somewhere somehow.

~~~
mwilcox
Even in the situation I described, the original company would leave the free /
client side version on the site as free marketing. This is the standard today
for enterprise-monetised open source software.

Solving this problem isn't about being local-first. It's about being local-
last. You have to be able to make more money by selling a software license
than you make by selling equity and chasing user acquisition and retention.

Then we'll see people waking up to the fact that all this proprietary data is
a liability and subscriptions are golden handcuffs and people will finally get
back to making real software again

------
throwGuardian
P2P has miles to go before challenging the reliability, convenience and
performance of the cloud.

That said, one area undervalued is Partially homomorphic cryptosystems[1],
where the cloud never ever gets to see unencrypted user data.

I hope the future is fast-local compute on cached data, with the cloud holding
a much larger, encrypted but permissioned data store, offering utility
functions like search over encrypted data

[1]:
[https://en.m.wikipedia.org/wiki/Homomorphic_encryption](https://en.m.wikipedia.org/wiki/Homomorphic_encryption)

------
chuhnk
Interesting to see this. We've taken a similar view for the initial phase of
the micro network. Locality is going to matter more and more as we move into
the future. Although Cloud still has its place and we don't ignore that
either.

[https://github.com/micro/micro](https://github.com/micro/micro)
[https://micro.mu/docs/network.html](https://micro.mu/docs/network.html)

------
tannhaeuser
You don't have to invent entire new paradigms such as CRDT for this. Unix is
all about site autonomy, no-BS tooling, simplicity, and portability. So for
your next project, consider Unix/Linux as deployment target during
development, and only then deploy it to a cloud-hosted Unix cluster, with a
local-first but cloud-hosted DB such as PostgreSQL and standardized middleware
such as AMQP/RabbitMQ/qpid rather than provider-specific solutions, or at
least use de-facto standard protocols such as s3 and MongoDB (if needed) and
supported by multiple clouds. Many people are prematurely committing to k8s
and "microservices" but in my experience, even though k8s as such isn't
intended as a lock-in strategy, it has the effect of absorbing so much energy
in projects (with devs more than happy to spend their time setting up auth,
load balancing, and automating things rather than on business functionality),
and then still ending up with a non-portable, incomprehensible mess of configs
and deploy scripts that it just isn't worth it.

~~~
vbsteven
My view on this is a bit different. I see Kubernetes as the abstraction layer
on top of the cloud providers. In the last few years I have set up multiple
k8s clusters for clients who specifically do not want to be locked in to a
certain cloud provider. Once the software is running on top of k8s it is easy
to switch cloud providers without changing the software.

Switching to another cloud provider this way is trivial and usually only
involves changing the Terraform configuration to setup a k8s cluster on
another cloud. All k8s-specific config/deploy files can be reused on the new
cluster.

This of course only works if (as you suggest) you stay away from cloud-
specific services (SQS, aurora, ECS, S3) and run everything in-cluster, or use
managed services that are available on multiple providers (Postgresql via RDS,
or Digital Ocean managed Postgres, Cloud SQL on GCP)

~~~
tannhaeuser
> _Switching to another cloud provider this way is trivial_

Based on my limited experience, I highly doubt this. Have you actually
deployed cross-cloud k8s setups, or is this merely a theoretical statement on
your part? Deploying to another cloud provider brings a whole new universe of
failure modes and auth quirks, let alone migration and switch-over woos.

~~~
vbsteven
I’ve done a couple cluster switches from k8s on AWS to other providers like
Digital Ocean and GCP. As far as I recall we had no issues and one of those
was done in about an hour were most of the time was spent waiting for
pg_dump/restore.

Note that most of these were not production clusters so switch-over was just
data restore and DNS changes.

I build clusters from the start to not use cloud-specifics where possible and
all cloud-specific configuration is on the cluster edges in terraform which
you have to rewrite anyway when switching clouds.

Auth things like IAM permissions are not an issue if everything is “in
cluster” and auth/permissions are checked there.

Most of these deployments consist of several application servers, PG
databases, redis, rabbitmq etc

------
dijksterhuis
Just wanted to point out that iTunes has had a local focused set up since
inception, using xml format for a library’s database.

That seems to still exist with the introduction of Apple Music. So all library
data (play counts, skips, file locations etc) are stored locally, but
streaming files are hosted remotely.

Although whether this was by accident or design I have no idea.

------
marknadal
P2P & CRDTs are definitely production ready already.

HackerNoon & Internet Archive are using (mine)
[https://github.com/amark/gun](https://github.com/amark/gun) already.

Local-first is very much the mantra of the whole dWeb community. I'm liking
this naming "local-first" as an evolution to "offline-first".

Ink & Switch had a good article on this:

[https://www.inkandswitch.com/local-
first.html](https://www.inkandswitch.com/local-first.html)

Also, for doing End-to-End Encryption, we've built some really good tooling
around this as well: [https://gun.eco/docs/SEA](https://gun.eco/docs/SEA) ,
wraps WebCrypto and works across Browsers, NodeJS, and React Native, so you
can do some really cool cross-environment/platform apps now.

------
quickthrower2
My side project is a local first (local storage on web) JAMStack. For extra
goodness it’s mobile first too.

I really love making apps this way for some reason. I think it’s the focus on
just the UI and not worrying about the back end until later.

For this particular app I’d consider “smartwatch first” to have been better as
its for fitness!

~~~
randomsearch
This is a really good way to prototype and find the minimum feature set for a
SaaS startup.

------
ranjithdsm
The data ownership is very important for business world. The reason why we
built our products for IoT ( www.bevywise.com ) as a more install able version
is the data privacy and ownership.

We really see this largely in manufacturing industry. If not local, we should
provide private servers and data security.

------
longnow
I've been following a local-first methodology without realising it for an app
that I've been developing. It's a workout-tracking app called harder better
faster fitter. It's designed for mobile use in the gym.

[https://harderbetterfasterfitter.com/](https://harderbetterfasterfitter.com/)

At the moment the app is a local only service and there aren't any backups.
Next year I plan to add a backend. I'll be keeping some of the ideas in this
article in mind. Currently I'm using the browser's local storage api to store
data locally. It mostly works, but will be bolstered significantly with a
cloud backup.

------
thirdsurf
I learned of Plan-Systems.org, they’re working towards something like this.
Their company is non-profit, their collaboration tools are open source and
protocol and the service is built on Unity and Unreal, which makes it cross-
platform.

------
api
So happy to see this! We created ZeroTier for the world this is talking about.
:)

[https://www.zerotier.com/](https://www.zerotier.com/)

/shameless but on-topic plug

------
lilactown
I recently started using Actual[0], a local-first budgeting app. It's
encrypted (soon to be end-to-end), local-first and can be synced across my
devices. Would recommend, and I was inspired to use it after following its
author, James Long[1] on twitter for awhile.

[0] [https://actualbudget.com/](https://actualbudget.com/)

[1][https://twitter.com/jlongster?lang=en](https://twitter.com/jlongster?lang=en)

------
oblib
I've been building apps that meet this criteria for several years now. It's
nice to see the concept getting some attention here.

The only thing I'll point out is that CouchDB getting rated "partially meets
the idea" seems pretty weak to me. They reference v2.1 but the latest version
is 2.3.1 and here's link to the docs on how conflict resolution is dealt with:

[https://docs.couchdb.org/en/latest/replication/conflicts.htm...](https://docs.couchdb.org/en/latest/replication/conflicts.html)

If finer grained control is needed it would be up to the developer to
implement it, and it really shouldn't be difficult to do that.

In my case, I use PouchDB to perform a "live sync" with all connected users so
they all get the latest updates to a document. If a conflict arises it's easy
for any one of the users to fix and push it to everyone connected.

------
pjkundert
The Holo / Holochain project was founded with this principle as a primary
goal:

[https://holo.host](https://holo.host)

We’ll be implementing CRDTs soon, but the concept of local control of all
data, authenticated and encrypted communications, etc. is implemented.

One fundamental difference between apps that support this and those that
cannot: agent-centric vs. data-centric design.

Strangely, many “distributed” applications (eg. Bitcoin) didn’t make the
“leap” to agent-centricity, and thus missed out on some key enabling
optimizations.

As a result — they are forced to implement “global consensus” (expensively),
when they didn’t need to, to achieve their goals: a valid global ledger, in
Bitcoin’s case.

It turns out that, to implement things like cryptocurrencies, you don’t need
everyone, everywhere to agree on a “total order” for every transaction in the
entire network!

Agent-centricity, FTW!

------
mgreenleaf
I love this idea. Especially the end-to-end encryption for data that passes
through a server to enable the ease of cloud computing without relinquishing
data ownership.

It also depends on _who_ owns the data. In an enterprise environment the
company usually has a vital interest in the data and on-premise deployments
are a good way of retaining cloud computing without giving up data ownership.
I'm surprised that more SAAS products don't offer on-premise given the privacy
and ownership benefits. The tricky part there is making software that is easy
to deploy and maintain, which might be the reason that it isn't done more
often.

A product like Grammarly that allowed on-premise deployment would side-step a
lot of the issues with sending all that data to a third party. I can't imagine
a law firm ever being able to (legally) sign up for that.

~~~
tarsinge
Maybe on-premise installation happen but are not advertised? I have
experienced one case in a previous company of a deal large enough to justify
one for a client with sensitive data. No need to say operations were not
happy, but I don’t have much informations to judge if it was a good deal for
the company in the end.

------
StavrosK
So I got fed up with current image hosting solutions the other day, because I
realized free image hosting is unsustainable and Imgur has turned into a
social network, which is the opposite of what I want.

So, I figured I'd create my own paid one, and am working on
[https://imgz.org](https://imgz.org). However, I want to add a free tier for
people who are willing to host their own images, and was thinking of writing a
daemon that would run on the user's computer and store all their images on a
directory there. It would have to be mostly-on, but not always-on, since I'm
going to be using a caching CDN.

Is this a good idea? I don't know how many people would know how/want to run
this, but it feels empowering from a data ownership perspective. What does
everyone here think?

~~~
mceachen
I've been working on PhotoStructure (not so much to be an imgur replacement,
but as a way to automatically organize and share my large (many tb) and
disorganized (due to failed photo apps and cos) pile of photos and videos. I'm
releasing to a new wave of beta users soon if you want to sign up and try it
out. It's a self-hosted solution with a web UI.
[https://blog.photostructure.com/introducing-
photostructure/](https://blog.photostructure.com/introducing-photostructure/)

As one entrepreneur/engineer to another: Don't underestimate the legal and
logistical effort you'll incur from a caching CDN. People post pirated,
abusive, and generally bad things, and if it's on your server, it's (becoming
moreso) your responsibility. DMCA and takedowns will consume noon-trivial
time, and makes simple corporate insurance decidedly not simple (or cheap).
It's typical for media hosting companies to hire _teams_ to handle these
issues. I was shocked while working at CNET (way back when) when I found out
most of a floor (in a large building) was for webshots' trust and safety team.

~~~
StavrosK
> It's a self-hosted solution with a web UI

That sounds great, thanks! Is there an easy way for me to distinguish
photographs from my photography work from snapshots I took with my phone?

By the way, your "Get early access" button does nothing on Firefox beta with
uBlock/Privacy Badger.

> People post pirated, abusive, and generally bad things

Oh ugh :( I was hoping this would be curtailed by the fact that this service
is paid-only, although I now realize I might have to rethink my "accepting
cryptocurrency" idea. Thanks for the heads up!

~~~
mceachen
> Is there an easy way for me to distinguish photographs from my photography
> work from snapshots I took with my phone?

Yeah! You can browse by camera (and by lens).

Thanks for the heads-up on the get early access button issue! The link just
scrolls you down to the bottom of the page where the login form is. I use FF
with privacy badger and ublock (and a pihole) on linux and android, and both
of those work. What OS are you using?

~~~
StavrosK
> Yeah! You can browse by camera (and by lens).

That's not entirely helpful because I have multiple cameras... Is there
something like a smart category where I can specify multiple cameras, or
directories, or something like that?

I'm running 71.0b5 (64-bit) Ubuntu, by the way.

------
garfieldnate
This. I want this. It happens to me quite often that I feel demotivated or
uneasy about using some software because I know I'm producing valuable data
specific to me that I could use in the future, but the product stipulates that
I don't get to keep the data. iOS apps are often the worst at this. There's no
filesystem, so all data has to be kept with an app. I loaded all of the books
I wanted to read into iBooks, and then it turned out that the iOS backup skips
books that you didn't buy from Apple. Bye bye book collection T_T.

Some of this could also be alleviated by Tim Berners-Lee's pod idea:
[https://solid.inrupt.com/](https://solid.inrupt.com/). But local-first is
better. I just want files on my machine.

------
josefx
I wrote documentation for configuration files in xsd with the idea that I
could use xslt to display them in a browser on any system and use the same
files for validation. This worked a year ago.

Now browsers consider local files that access other local files suspect and
will refuse to load anything unless beaten. So I now use a python script to
run a simple local http server to view my local files from a single "origin".
However http itself is already considered suspect and many claim it should be
deprecated for https.

In the future I will have to provide a lets encrypt signed https server with
valid domain so anyone can view those files on their browser without having to
mess with about:config settings or their own certificate cache. The cloud is
the future, do not dare to build something that runs locally.

~~~
bouk
http over localhost is considered secure by browsers

~~~
josefx
Is there a documented guarantee of that somewhere? Because I am half tempted
to make a browser based UI for a few tools, but would rather use Java if there
is even a tiny risk that I get the rug pulled from under my feet again.

------
theamk
It’s strange that Evernote is omitted from the list - it is a great example of
local-first app.

Their recent-ish history, when the their free tire become limited to only
syncing a few devices, illustrates that even if software is fully local, and
supports open formats, having the functional cloud matters, a lot.

~~~
pqs
I love Evernote and I use it daily, but Evernote lacks end-to-end encryption,
which is a pity. I store lots of information in that app and I would be more
reassured if I knew that it is only me who can read the data. I would even pay
more in order to have that kind of encryption. I think that no current feature
of Evernote would be affected by encryption as text recognition in images can
be done at the client level.

------
jefurii
I'm surprised nobody has mentioned using git-annex (and Git of course) to
manage data (full disclosure: I develop an archival management app which keeps
its data in git/git-annex repositories). Of the seven key properties, git-
annex gives you 1,2,3,4 and supports 5 and 7; 6 depends on how you store
things. Git-annex supports identity in the sense that each clone of the
repository has a UUID. You can choose to have a central hub if you like but
you don't have to (surprise! you don't really need GitHub!). It comes with
caveats of course: Binaries can be synced but only textual data is actually
versioned, and once you put something in it will always be there unless you
use something like git-filter-branch.

------
Jareddd
Awesome write-up! This brings me back to early 2000's when we we typically
owned most of our graphic design software. Yes it was expensive but, there was
barley any "cloud" features added and if you wanted a new version you simply
had to download the update or simply keep the software you were currently
using.

My team and I have taken the initiative to offer a e mail design tool that is
considered first-class software to the OS
([https://bairmail.com](https://bairmail.com)). The last thing I would say is
developing desktop apps vs a web app is considerably harder thus most
companies are aware they are saving by controlling software updates,
versioning etc.

------
Scirra_Tom
Our software Construct 3
([https://editor.construct.net/](https://editor.construct.net/)) I think meets
most of these points. It runs offline, and we never have access to the users
project files. You can save/load locally, and it runs in the browser. Game
project files are zips with JSON + raw asset files. No syncing with the server
needed so it is fast, and a design mistake that is severely hampering some of
our competitors!

I'm not entirely sure how supporting collaboration in real time belongs on
this list. Seems like a nice to have that isn't really related to the rest of
the list.

------
at-fates-hands
Is this something like Blockstack could help with?

[https://blockstack.org/](https://blockstack.org/)

I'm having a hard time understanding the differences between "Local-first
technology" and something like Blockstack. I'm not saying BS completely solves
the issues pointed out in the blog post, but it seems to me its pretty close.

What do you think?

here is a list of the current apps available:
[https://app.co/blockstack](https://app.co/blockstack)

------
johnmarcus
I've been working on SPA's that launch from keybase.io public folders and can
talk to my local KBFS storage, which is encrypted locally and then and
distributed to the cloud. This way I can access my own data anywhere I have
keybase installed, using apps I don't need a server to host. It's still all
just prototype work for myself, but im excited about owning my own data while
still having the safety of cloud distribution combined with the security of
local encryption.

------
jsfenfen
The paper says pinterest meets the "collaboration" ideal but github doesn't.
I'm sympathetic to the idea that _nothing_ meets the ideal, but c'mon.

------
wazoox
I'm using NextCloud for that. Nextcloud can be basically used in local-first
mode (mimicking dropbox).

My phone automatically uploads all pictures to the NextCloud. Then there are
apps. For instance I use Nextcloud with the Music app to stream my own mp3s
from my Nextcloud to my phone running Ampache.

There are also collaborative editing tools, and various options to edit all
sort of documents in a web UI, and always the local editing fallback (or the
opposite way, as you see fit).

------
uber-geek
A more recent issue, this one with youtube banning google accounts for posting
too many emoji's during a live stream,what if all of your business data was on
your google account? You would be done unless you could get unbanned. There is
something to be said for using local apps. The engineers I work with are often
out of range of any network, so having apps that run on their laptops is
crucial.

------
davidkuennen
Very cool. Let me add to all of these great examples in the comments. I
published this app with focusing on exactly these points just two days ago.

[https://stockevents.app/](https://stockevents.app/)

All tracked stocks stay within the app. You only pull information from the
servers and store that information locally for offline use.

------
hwwang2005
I'm creator of BLOON ([https://www.bloon.io](https://www.bloon.io)). If BLOON
had been put in the Table 1 in the paper, the values could be:

O O - - O - O

I think BLOON is closer than Git+GitHub.

And great article & paper! They do give us many inspiration about improving
BLOON. Thank you!

------
sans-serif
iOS apps like Bear achieve this by building on iCloud storage

------
miki123211
Resilio sync is a perfect example of such app.

It's basically a P2P based Dropbox with no accounts, full end-to-end
encryption and no folder size limits.

It's not open source, but it can work without a central server if you need it
to. It's also amazingly simple to set up, much simpler than sync thing.

~~~
mackrevinack
i used resilio for years and thought the same about syncthing until i switched
last year and now I actually prefer how syncthing is setup.

if you pay for resilio there is an option to add all your folders in one go
but on some computers I don't want to add all of them anyway so that's not
much use to me. with the free version you have to manually add folders one by
one but to do that you need the key which means you need to copy them to a
text file and add them on another computer.

with syncthing, it will detect other syncthing devices on your home network so
you just have to add the ones you want then accept the request from another
device.

once that is done you select which folders the device has access to and then a
notification will show up on said device asking you to connect. so basically
no fiddling with keys or having to store them somewhere secure

(this is all presuming i was using resilio correctly, maybe there was an
easier way I was not aware of)

------
toolslive
My company built this more than 10 years ago, and nobody was interested.
Here's a press release from 2007:

[https://www.macworld.com/article/1058169/mcnucle.html](https://www.macworld.com/article/1058169/mcnucle.html)

~~~
fastball
Might have something to do with the fact that McNucle is a freaking terrible
name.

~~~
toolslive
no, most end users did not understand (or cared about) the data ownership
problem back then. Times have changed.

~~~
procombo
That name though...

~~~
mackrevinack
lol

------
gosubpl
Things are happening in that space already! I work at Actyx and we have a
production ready stack for local-first real serverless (peer-to-peer)
applications. Please take a look at [https://actyx.com/](https://actyx.com/) .

------
spieglt
If you want an easy way to make a file (or .zip of files) more private before
storing it in the cloud for backup/availability purposes, please check out
Cloaker: github.com/spieglt/cloaker. It's simple, password-based, drag-and-
drop file encryption.

------
CPAhem
Those 7 points remind me of the Syncdocs
[https://syncdocs.com](https://syncdocs.com) app which ticked most of them.
Using an encryption app and Dropbox will let you do the same thing, too.

------
gioscarab
I'm applying the local-first methodology in PJON, a new networking open
standard : [https://github.com/gioblu/PJON](https://github.com/gioblu/PJON)

------
interactivecode
There is a longer version of this article here:
[https://www.inkandswitch.com/local-
first.html](https://www.inkandswitch.com/local-first.html)

------
ph0rque
That is my vision for [https://edible.estate](https://edible.estate), version
2 (version 1.0 should be done this year).

------
alleycat5000
Reminds me a little of the Upspin project:

[https://upspin.io/](https://upspin.io/)

At least some of the motivations are the same.

------
FpUser
Article mentions the following tweet:

 _What was the last app you used on your laptop that wasn 't:

Terminal

VS Code

Web Browser

OS settings _

That guy must be living in a very limited / imaginary world world. I use
boatload of local software

~~~
habnds
Which are your favorites?

~~~
FpUser
I do not have favorites. I use various tools for various activities.
Mechanical design, electronics, software development, video, photo, data
analysis etc. etc. Way too much to list all. Some apps are being used very
rarely but I still need those when the time comes

------
chaz6
IPv6 will make it easier to access your data anywhere. I am putting this here
because so far nobody has mentioned it.

------
rburhum
why would the section marked “Git+Github” have “collaboration” marked as ‘no’?
Like that is what it actually does...

------
saurabhnanda
Not related to the actual content, but may I please request you to increase
the contrast on your website. Grey on grey is very hard to read.

~~~
josteink
Firefox reader mode. Instantly fixes any crappy design on any site. Learn to
love it :)

------
amitport
Hmm... homomorphic encryption.

------
touchpadder
Abolition of ownership is a communist idea. Local first, cloud for sync and
backup.

------
rado
Common sense.

~~~
ptx
You might think so from only reading the title, but what the article/paper
actually proposes is using CRDTs to create a synthesis of the advantages (for
the user) of both cloud applications and traditional local applications.

