
Offline First – A Better HTML5 User Experience - openmaze
http://www.joelambert.co.uk/article/offline-first-a-better-html5-user-experience/
======
mbrock
The most useful pattern I know of for offline web apps is the command queue.

Basically, the rendered state of the client is the acknowledged server state
plus the client-side command queue.

User actions don't make a server request and then update the UI. Instead they
directly append to the local command queue, which updates the UI state, and
right away the client begins communicating with the server to make the local
change real.

While the client's command queue is nonempty, the UI shows a spinner or
equivalent. If commands cannot be realized because of network failure, the UI
remains functional but with a clear warning that changes are waiting to be
synchronized.

(The API for connectivity status are useful for making sure that the command
queue resumes syncing when the user's internet comes back.)

~~~
Fiahil
You still have to handle cases where the server state has been updated
(possibly via another medium, event, ...) and when the user's internet comes
back it's not a matter of pushing clients' commands anymore. Instead you have
to merge changes (either backend or front end side) before fetching the new
state. For example, I try to purchase an item on my desktop, but can't because
I lost connectivity. So, I proceed to buy it on my mobile. When the net comes
back, did I mean to buy this item two times or just one ? In this case, it's
easy to find a workable solution (cancel the second order command), but you
get the idea: It's not trivial.

CQRS and ES, are wonderful tools in such situations. Past that point, your web
app is far away from the little CRUD mashup it was at the beginning.

~~~
pkaeding
JSON-Patch [1] can be helpful here. You can maintain a list of changes that
need to be applied, such that they only touch the parts of the document that
need to change (other changes to other parts can be interleaved). If
necessary, you can include tests to assert that some value in the document is
what you expect it should be, and the patch can be rejected if that test
fails.

[1]: [http://jsonpatch.com/](http://jsonpatch.com/)

~~~
coldtea
That's if what's to be changed is a document (which is a big if).

Things get ugly quickly for any more complicated scenario.

~~~
pkaeding
Yeah, that's true. JSON-patch won't help with all situations. It is still an
under-utilized tool, IMO.

------
c-smile
Wearing my architect cap...

There are four types of applications in that respect:

1\. always disconnected (e.g. calculator)

2\. occasionally connected (e.g. email client)

3\. occasionally disconnected (e.g. messaging client)

4\. always connected (e.g. trading terminal)

First architectural task I am doing when have given an application idea is its
classification using these models.

Each particular case requires its own storage/caching approach. "Offline
first" is too broad.

~~~
krick
I'm not sure I get what you mean by "always connected". Because speaking about
how stuff _should_ be written, I would expect developer never goes farther in
his assumptions than #3. That is, I would hope my trading terminal still works
just fine after the whole office goes offline for 10 seconds (or more, doesn't
really matter). Actually, I would care quite a bit more about my trading
terminal handling such cases, than, say, messaging client.

~~~
c-smile
At the age of electronic trading that 10 seconds off-time of your trading
terminal makes it almost useless.

~~~
tim333
There are different types of trading. The last few I did involved phoning the
bloke at iDealing so he can call the market makers and get back to me a few
hours later.

------
joelambert
Wow, cool to see this resurface again! I wrote this ~4 years ago and the web
has come on along way in that time. New features such as service workers are
definitely making offline a lot easier.

On the subject of progressive enhancement, I'm a huge advocate and believe it
is in general the way todo content sites. As has been pointed out above
though, some use cases do require a different approach. For context this was
post was written after working on a number of HTML5 apps that were wrapped in
Cordova/PhoneGap.

------
substack
Offline is not a mere feature you can bolt on to existing architectures
because you are really building a distributed system. Architectures that work
well for distributed systems and especially p2p architectures thrive in this
environment. Elsewhere in these comments people are discussing command queues,
but this idea presumes that servers are somewhat reliable and that all
operations need to go through a centralized point of control and failure.
Instead, you can take this idea further and implement a kappa architecture
(sometimes also called "event sourcing") where you maintain a log locally on
every client which is the source of truth, not a server. When a network
connection is available, you can replicate the log to a server or directly to
other clients. You can build indexes (materialized views) that sit on top of
this log to answer queries more quickly than reading the entire log out every
time. You can also blow away these indexes and rebuild them from the log
whenever your requirements change (migrations).

Unfortunately the web is missing a few pieces that would make it a very good
platform for fully p2p, distributed apps. Service workers are a good start,
but they have a 24-hour upper cap on max-age of the service worker itself, so
users can't trust on first use (TOFU) and then be more secure against kinds of
active targeting. The suborigin specification and iframe sandboxes for
distributing apps offline and p2p would be much more useful for offline
sandboxes if they didn't require that a server send an http header. These will
become much more important as the web bluetooth API matures, which can allow
distributing user data and application updates in a totally offline
environment.

Even without being fully offline, it's very odd that when an automatic update
to android or windows comes down the pipe, people in remote areas download the
exact same bytes from servers in America over and over again, all over a thin
pipe. They could fetch that data from each other and save a lot of money on
bandwidth and data caps.

------
mark242
tldr: Someone wrote a bunch of paragraphs and LOC around
JSON.parse(window.localStorage.getItem(key)).

If you're really interested in learning about creating a workable offline web
app, Google has some great documentation.
[https://developers.google.com/web/fundamentals/instant-
and-o...](https://developers.google.com/web/fundamentals/instant-and-
offline/offline-ux)

Last, the Safari team needs to seriously get to work on Service Workers. We
will see web apps grow by leaps and bounds once the service worker spec is
opened up to iPhone users.

~~~
peller
> We will see web apps grow by leaps and bounds once the service worker spec
> is opened up to iPhone users.

Sounds exactly like what Apple _doesn 't_ want; no 30% revenue cut, no walled
garden of control, ...

------
jedberg
These are basically the same rules for any distributed application, like
something using microservices. An app is just the edge node of a distributed
system.

In any distributed system, the biggest cost is moving data between nodes, and
therefore the biggest failure case is when data is moving slowly or not at
all. It's a case you should always be prepared for.

If you write your app in a way that assumes the network is bad, which you
should always do, whether it's an app or two microservices, then you'll have a
more robust system.

~~~
intrasight
> These are basically the same rules for any distributed application

And we've been doing those for like 50 years now. So why is this still so
hard? Because new developer were, for all intents and purposes, born
yesterday.

------
ebbv
I get where this movement is coming from but I just fundamentally disagree
with it. Making offline first web apps may make sense for certain applications
where you expect you users may need to use it offline, but it doesn't make
sense for all apps and it can require a fundamentally different way of writing
your application which is a waste of time and effort if it's not a likely use
case for your users.

~~~
jedberg
All mobile apps will be offline at some point. Cell coverage doesn't cover
100% of the planet yet.

The question is, how good of an experience will you be able to deliver when
your client is inevitably in one of these places?

~~~
ebbv
Of course but if you're writing a web app who's only purpose, for example, is
to talk to support staff for your company then having an offline mode is not
really useful. If you're offline you can't talk to the staff, so all the app
needs to do is fail gracefully. Designing it "offline first" would be silly.

I actually believe that >50% of web apps probably fall into this category,
where they really cannot function properly offline because the online-ness s
core to their functionality. That's why they are web apps in the first place.

Now if you are designing a web app that is a web version of a more traditional
native app like Google Docs or something, then sure offline first makes sense.
But I don't think that's the majority of web apps.

~~~
mbrock
Offline first basically means that if you send a message while your network is
temporarily down, it's just queued and resent immediately once the link is up
again, like your email outbox.

Since network links are always flaky, it just makes sense to do it this way.
Since they are also always relatively slow, it makes sense to cache data
locally in order to give a faster experience.

Not doing things offline-first in an app basically means that you are
introducing synchronous requests everywhere: reading a support reply from
yesterday is a synchronous request that fails "gracefully" if your 3G happens
to be down, etc.

Telegram's web app is pretty nice. The app code is cached offline with a
service worker and updated whenever possible, so it loads _instantly_. The
most recent messages from your contacts are saved in the client as well. I
appreciate all that stuff as a user, and the more stuff works offline the
better, because it also means it's faster and more reliable.

For more info:

[http://alistapart.com/article/offline-
first](http://alistapart.com/article/offline-first)

[https://developer.chrome.com/apps/offline_apps](https://developer.chrome.com/apps/offline_apps)

------
fareesh
Anyone using PouchDB + CouchDB in production to solve this type of problem? It
is my go-to solution but I'm unaware of problems at scale.

------
blazespin
Maybe add 2012 to title on HN. As that's from when this post is.

~~~
wslh
I think this is an exception, until we don't have a good offline experience
this article will be current.

~~~
AgentME
It is missing anything about ServiceWorkers.

------
tannhaeuser
Isn't it simpler though to just use static HTML + optional Javascript (like we
used to in the 2000's eg. progressive enhancement years)? I mean why use M-V-
whatever for content-driven sites at all?

~~~
CaptSpify
Maybe I'm mis-understanding but it sounds like your talking about something
entirely different?

Some sites just wouldn't work well with HTML+optional JS. Google maps (IMHO)
would not work well that way, so using an offline-oriented model would make
better sense.

That being said, if your a news site, or a blog, yeah, a simple static page is
probably a better solution.

~~~
tannhaeuser
Of course true webapps can benefit from an MV* approach (like enterprise-type
LOB apps). But gmaps IMHO isn't a good example, as it isn't MV*; rather, it
fetches prerendered bitmap or vector graphics from the server.

------
tyingq
I would throw in some guidance that says "don't trust anything the client side
sends to you". Developers used to server side MVC could expose themselves to
client-side manipulation that open up some security issues.

I remember some early ecom cart implementations where you could "name your own
price" as an unintended feature.

------
kqr
I sense a trend of trying to cram everything good about native apps onto the
web. Do people who do this stop to think whether the web is actually the
correct platform for their app?

~~~
ciconia
If you need to build cross-platform apps - and that is mostly the case
nowadays - the web is actually not such a bad solution. I mean, what is the
alternative?

~~~
GrinningFool
Gtk

Qt

The list is actually quite extensive:

[https://en.wikipedia.org/wiki/List_of_platform-
independent_G...](https://en.wikipedia.org/wiki/List_of_platform-
independent_GUI_libraries)

\--

For mobile, clean separations also helps. It is definitely possible - and less
complex than you'd expect - to have core functionality in a shared library,
wired up to platform-specific native GUI toolkits.

But your question illustrates the problem. The pervasive presence of toolkits
that add layer upon layer to create "cross platform" are the new norm. People
are literally losing awareness that other options exist - much to the
detriment of end users.

We have built an entire industry around such tooling - and long ago stopped
questioning what value it brings.

~~~
kaoD
I'm mainly an app developer that transitioned (a long time ago) from native to
web apps, tired of code duplication. I'm well aware of other options.

The web comes with its warts, but I've yet to see an app platform as ergonomic
and comfortable for the developer as the web. For 99% of my use cases,
anything else is overkill and too much of a hassle. It's not the web's fault
that it's a better app platform than actual app platforms.

~~~
RantyDave
Apple's stuff is actually really nice. Sadly Apple specific, obviously :(

------
cmurf
It continues to baffle me how much on disk cache is used by web browsers, but
won't be used to load a page I know is in the cache if I'm offline.

------
DecoPerson
I'm surprised no one has mentioned Meteor[0].

[0] [https://www.meteor.com/](https://www.meteor.com/)

~~~
flippyhead
Me too. We picked Meteor specifically because it's approach incorporates ideas
that make doing offline a lot easier.

------
afarrell
Doesn't Chome on iOS prevent this because it reloads whenever you re-open a
page, even if you don't have an internet connection?

~~~
RandomInteger4
I think there is a flag under chrome://flags where you can turn off automatic
page reloading if the system is disconnected from the internet.

That being said, doesn't appcache provide the offline page if the browser
can't connect?

------
mozumder
That's nice and all, but no one is going to have all the images/videos/audio
available offline for their media sharing social network app.

Web servers exist so you don't have to have all that data locally.

------
stuaxo
I wish more native apps were offline first.

Take the tripadvisor app; when you go to the website you are nagged to go to
the app, if you click you end up in the google play store and it loses where
you were.

And of course it only works online; utterly pointless.

------
edem
Funnily this site does not wor k well with my crappy mobile connection. It
loads 3 paragraphs and strangely the 4th paragraph is cut in half. I waited a
minute for it to load.

------
opvasger
I feel like commenting on the callbacks and variable-bound-contexts, and
saying something about the glorious wonder of modern JavaScripts... better
not, tho :)

------
dimino
How do you write unit tests for something like this? Are there frameworks that
make this kind of model easier to deal with?

------
cryptarch
How about offering a HTML experience in the first place?

I'm really tired of "HTML" websites that are actually a heap of Javascript
writing to a virtual DOM, with only a few references to scripts, most of it
being for surveillance (e.g. ad trackers, surveillance for profit with the
side effect of it being available to intelligence services via subpoenas, gag
orders and secret courts).

------
BorisMelnik
very interesting - I'd love to see a feature like "queue / share later" (maybe
Buffer etc) to share when you get back online again.

~~~
robwormald
You might find
[https://developers.google.com/web/updates/2015/12/background...](https://developers.google.com/web/updates/2015/12/background-
sync) interesting?

------
foxhop
Offline First versus Resilient Web

------
ebbv
Dude, I get it. Stop being so condescending. Maybe you're trying to be helpful
by including those links but it comes off as incredibly insulting and
pedantic.

I was talking about a real-time communication application.

You can make the argument that the application should try to cope with a
sporadic connection, but in the real world this is inefficient for both the
user and the staff if the connection is going in and out and they are trying
to hold a conversation it might be better to show the user a message that says
hey your connection is down try later when you have a better connection or try
an asynchronous support method like submitting a ticket.

Regardless that was just an example off the top of my head. My point still
stands; web applications are web applications for a reason, and "offline
first" doesn't make sense for most of them.

~~~
mbrock
I disagree and I explained why. Telling me what the "real world" is like is
the same condescension you complain about.

If a couple of links to articles is "incredibly insulting" to you then you
shouldn't discuss stuff on the internet.

~~~
ebbv
Your example indicates you don't get it. You mention responses "from
yesterday". That's not going to fly in an application that's real time that
requires staff to keep the conversation open. You're describing a ticket
system, I'm describing a live chat system.

I said "in the real world" as opposed to abstract talking about an
application; in the real world a staff member would be stuck waiting for the
response from this user where you're trying to keep the conversation going
despite their sporadic connection. I guess I could have said "in meatspace" or
something.

Links to explanations of the topic being discussed is _of course_ incredibly
insulting. Come on.

Anyway, whatever man. You are not pleasant to talk to so I'm done.

~~~
mbrock
In the real world, network links go down all the time, even if just for three
seconds.

As for etiquette... I politely explained how I see things different from you
and provided a couple of links for reference. You immediately called me
"condescending", "incredibly insulting", and "pedantic".

If you want a pleasant conversation, that's not the way to do it...

------
CorvusCrypto
I see the author's vision, but I have to say that I still favor progressive
enhancement over this. Using javascript only assumes a lot about the user even
in this modern age of web development.

Also as a nitpick, I would say don't make your data objects SINGLETONS, make
them SINGLE INSTANCE.

EDIT: someone deleted their comment but brought up a good point that you can
have progressive enhancement with this. Yes, however the author made it clear
he favors the JS or nothing approach. And there's even this snippet:

>An offline first approach would be to move the entire MVC stack into client
side code (aka our app) and to turn our server side component into a data only
JSON API.

~~~
mark242
Here's your user assumption: 99.8% of browsers have Javascript turned on.
Creating a fallback version of your app that works without Javascript is
basically like developing for IE 5 -- you are completely wasting your time.

Here's another one: mobile browsers make up a majority of web traffic now, and
Chrome and Safari are pretty well split in terms of share. Sending HTML tags
over the wire is a waste of your users time. The idea of "progressive
enhancement" should be thought of more as, "what can we display quickly on our
user's screen while we're downloading and parsing the Javascript to make the
application work". Note that this doesn't mean loading screens, just an
initial state which shows the user progress in building up the app from that
initial http connection. Regardless of the negativity around AMP, Google has
made us think about what exactly we need to do in order to provide a good
experience quickly and a feature-rich experience within a couple of seconds.

~~~
CorvusCrypto
Progressive Enhancement was laid out clearly actually so that basic content
should be accessible to all web browsers and basic functionality should be
accessible to all web browsers.

As for google, they also recommend using progressive enhancement and they even
removed their ajax SE scheme posting about this wherein they even give a
recommended trick for compatibility testing.

[https://webmasters.googleblog.com/2015/10/deprecating-our-
aj...](https://webmasters.googleblog.com/2015/10/deprecating-our-ajax-
crawling-scheme.html)

------
yellowapple
I disagree rather strongly with the implicit advice here: that HTML5
"applications" ought to rely on client-side processing first and foremost.
This usually translates to "shove more Javascript down the user's throat",
which is the opposite of "a better HTML5 user experience" IMO. If I trust you
enough to want to run arbitrary Turing-complete code of your making, I'd be
more inclined to download a native implementation for my platform; otherwise,
I generally have better things for my CPU to do than burn cycles on things
that should be done on your own servers.

I do agree, however, that the HTML "app" ought to be treated equivalently to
native implementations -- that is, it should communicate over the same API (or
perhaps an internal equivalent) as native apps. I like to think of this as
"middle-end" web development, with the HTML-generating server working to
mediate the browser's expectations (HTML and CSS and _maybe_ some Javascript
if it's really needed) with the actual business logic provided by the API.

