
Why all my iOS Apps are on hold - or iCloud sucks Part II - mikecane
http://createlivelove.com/246
======
doe88
The term iCloud regroups several technologies. As far as I know the simplest
one a basic key-value store works very well, I use it without any problems.
However, last year I tried to integrate CoreData syncing through iCloud in one
of my apps and I couldn't make it work reliabily, I spent a lot of times I
tried many things, but usually there always was some irrecoverable corruptions
that ended-up preventing any syncing. I never shipped this code, I was too
afraid, it was too brittle. I haven't retried since. I heard the situation has
improved but I don't know for sure, it was such a nightmare that I simply
avoid to spend any more time on this api.

Edit: I found this quite recent post with similar issues
<http://www.jumsoft.com/2013/01/response-to-sync-issues/>

------
gfodor
it would be helpful if this article included some actual details about what
doesn't work for people considering adding iCloud to their apps.

~~~
potatolicious
iCloud in general "just works". It was pretty terrible from the get-go, from
reliability to really egregious bugs in the OS, but at this point is pretty
stable.

The real problem here is the combination of iCloud and Core Data (for non-
Apple folks, the built-in persistence/object model framework).

In a somewhat simplified way, Core Data is a data store most commonly backed
by SQLite. It's used everywhere from your mail app to the Facebook app. It's
handy and convenient when it isn't trying to kill you - it's one of the more
complex frameworks that Apple ships, and the documentation is _extremely_ poor
for it. It also doesn't help that there is a _lot_ of quirky behavior that's
more or less tribal knowledge to long-time Apple devs and WWDC attendees, and
is has enough concurrency gotchas to fill a container ship.

So, Core Data is not the simplest piece of code to work with from the get-go,
and then you pile iCloud on top.

iCloud is conceptually pretty simple - there's a directory on your file system
that is replicated to Apple's servers, and back down to each one of your
devices. When files change the OS sends notifications to your app informing
you of such. If you're using iCloud as a distributed file system (a la
Dropbox), it's easy to integrate.

The trouble with this model is that Core Data stores can be in fact quite
large. If you have a 30MB database of, say, recipes on your disk, you really
don't want to resend the whole thing every time your user alters a row. Apple
special-cases Core Data stores in iCloud by sending only deltas, but the
implementation is utterly broken:

\- There is no baseline state(s). The database is literally the playback of
every delta, with no canonical snapshots. Ever.

\- iCloud is really bad at sequencing the deltas, especially if (somewhat)
concurrent changes are happening. Core Data is also _terrifyingly bad_ at
conflict resolution when deltas from different devices don't line up.

\- The above points means that it's _trivially easy_ to break your entire
database, as your deltas no longer play back to a sane database file. When
this occurs there is no way to roll back your database, since there is no
concept of a snapshot or last known good state.

\- This is compounded by some really horrible API design on Apple's part. When
your iCloud database becomes corrupt _it fails silently_ , with absolutely
_no_ API that allows one to determine said broken state. Your app literally
stops working with none of the OS making a single peep, and no way for you to
proactively check your database's state. iCloud seems to print some debug
things into your console indicating this failure, but neither your app nor
your users have access to it. This manifests in your app (and in tech support
requests) as "iCloud just stopped syncing" - your device stops transmitting
deltas and simply works off of the last locally-cached good state, all without
telling the dev or the user.

\- This is further compounded by the fact that the entirety of Core
Data+iCloud was rushed out the door. Documentation was non-existent even up to
the day the new version of iOS went out to the public. There was literally
only one source of information: a thread on the Apple dev forums that Apple
engineers would sometimes pop into.

\- This is further compounded by the fact that even the Apple engineers
themselves seemed to have no idea how to implement an app with iCloud + Core
Data. The example code (provided deep, deep within aforementioned thread) made
90 degree turns seemingly every few days. The same sample would have
_dramatically_ changing implementations from week to week. The Apple engineers
in the thread seemed as mystified as the people trying to use the framework.

\- The poor documentation and poor state of sample code, as well as what is
IMO a fundamentally broken architecture means that iCloud + Core Data remains
a complete nightmare to this day. I think the poor architecture is really the
core of this here - while Apple could have provided better docs and better
support, I believe the fundamental architecture choices made here means Core
Data + iCloud would never have been workable.

\- The icing on top of the cake was an extremely egregious bug that existed at
launch (unsure now, you couldn't pay me enough money to work with that API
ever again). "Okay, so our deltas are completely fucked" you say, "Why don't
we just delete the app's entire iCloud bucket and start over?"

A jolly good idea until you realize that a bug in iOS/iCloud means that _when
you delete an app's iCloud storage "bin", it cannot be recreated by that app,
ever again_.

~~~
gfodor
wow. thanks for writing that up. if it's any consolation, you likely just
saved me untold amounts of pain as i am wrangling with what approach to take
for saving a small but not trivial amount of structured data in my app. this
pretty much rules out core data!

~~~
vor_
Don't rule out Core Data! It's very nice (especially with the new concurrency
APIs); it hasn't yet been brought totally up to speed with iCloud, but it's
been improving every year. Remember that OS X is on annual releases now, so
there will be more improvements this year and so on.

------
stcredzero
I was just about to try and convert my app to use iCloud, and incorporate the
features in a 2nd one. After going to the 2011 conference, I followed the
message and took the extra effort to put the 1st app's information in Core
Data, and then only afterwards heard how bad iCloud was. Now I'm thinking of
moving my data back to a document-based architecture. Now there's grousing
that even that sucks.

If iCloud was hitting on all cylinders, then Apple could kill MS Surface by
having seamless sync between laptops and tablets. Instead of a chunky
tablet/laptop that's mediocre at being both, you can have a fabulous tablet
and a great ultra-lightweight laptop and not have to worry about syncing. Need
to pass around something to show a coworker? It could just work.

This is a sign that Apple is either not hitting on all cylinders, or bit off
more than it could chew. (To be fair, distributed data is, and always has
been, quite a hard problem.) Apple could be moving in for the kill. Instead,
they're recovering from stumbles while their opponent has left themselves wide
open.

~~~
taligent
Syncing does generally just work between laptop and tablet.

I add a Safari bookmark on one and it appears instantly on the other. Likewise
for contacts, calendars, apps, music and any iCloud enabled app e.g. Coda.

Then just add Dropbox and Evernote and you're laughing.

~~~
stcredzero
Then why is there so much grousing in the App Developer community?

~~~
interpol_p
It's the combination of Core Data + iCloud that is the biggest problem.

Using the document model (as long as it's not a Core Data document) with
iCloud is fine, as is the Key-Value store.

Using Core Data on its own is also fine, it's a great framework and we use it
a lot.

~~~
stcredzero
Yeah, I have no issues with Core Data. It's also not so complex, IMO. I'm
going to have to do a migration to iCloud document for existing users with my
next release, though.

------
cloudmike
To all those suffering from Core Data + iCloud syncing: we built Simperium (YC
S10) to solve this problem. The iOS/OSX and JavaScript libraries are open
source, with the backend to follow (or you can use the hosted service). The
site is at <http://simperium.com> and the repos are at
<http://github.com/Simperium>

We were recently acquired by Automattic, and we're now putting more resources
into the project (for example to make it work on Android as well). If cost for
the hosted service is a concern for you, you can get in touch and we'll work
something out (info@simperium.com).

------
robterrell
Instead of waiting two years, why not implement your own sync service? I know
that sounds like snark, but using something like CouchDB and TouchDB can give
your users an experience similar to the one promised by iCloud.

~~~
DeepDuh
I'm very interested in those two solutions. Does anyone here have some
experience about CouchDB/TouchDB on iOS/Android? Specifically: Does it
replicate reliably to a CouchDB or Couchbase server? How are conflicts
handled? Are there wrappers already available that map to ObjC/Java native
data structures, similar to couchdbkit on python?

~~~
greendestiny
Hey yeah I've used TouchDB on iOS quite a bit. It syncs reliably to Couchdb -
I actually haven't tried couchbase. Conflicts are handled in the default
couchdb way I think - I haven't really touched on this much. TouchDB syncs to
native structures that are derived from CouchModel.

~~~
DeepDuh
Great, thanks for the helpful info!

Oh, one more thing: Did you ever load multiple TouchDBs? I'm thinking the
overhead per DB (not per server) should be pretty small, no?

Background: I'd like to build an application where I separate entities into
multiple couchDBs, partly since it saves me view definition code, partly
because it gives me more flexibility for per-document access permissions.

~~~
greendestiny
Yes I have a framework that splits itself into 3 databases and has quite a lot
of attachments. There isn't a big overhead as far as I can see. Does couchDB
do what you need with per-document access permissions? It seems to be the
biggest downside to couchdb is having to do so many workarounds for per-
document access permissions.

~~~
DeepDuh
That's also my conclusion. However I'm not aware of a DBMS that

\- is document based

\- can easily replicate to laptops and embedded devices

\- has all the security I want, e.g. encryption, per document security

\- has the bindings I want and/or a new binding can easily be implemented ad-
hoc

Given the advantages and disadvantages I'm leaning towards CouchDB and rolling
my own security wrapper. IMO that's still better than going with an RDBMS and
then implementing replication[1]. How do you see this?

[1]edit: and also giving up the document based approach, thus not having
arbitrary mapping from the get go.

~~~
greendestiny
I love couchdb for its replication, not only did I use it for a previous
company I worked for and still contract with, I'm using it for my 'big idea'
project. Once you have data replication you do need some kind of different
approach to user/document security - after all they can take the data they
have and do anything with it - so you need to treat replication as data input
and validate there, like couchDB does.

I think the real downsides are scaling and performance but there are options
at least. Given that TouchDB is essentially couchdb mapped onto sqlite I do
wonder if there are going to be good options to map couchdb replication onto
postgres in future.

~~~
DeepDuh
[edit: sorry, I replied here, because it wouldn't let me reply to your other
answer first and I thought it was because we've reached some kind of maximum
depth]

Thanks, I see. Ok, if it's just string or binary based it will work from a
correctness standpoint. I'd expect big performance problems with ad-hoc views
(if they even exist here), but these shouldn't be used in production anyways.
Also, I'd expect replication to take quite a bit longer because it needs to
map every document with changes, so there's going to be a tradeoff between
initialization time and replication time between the two approaches.

I'm thinking it's going to be beneficial for larger databases to have a
TouchDB 'server' somewhere, which constantly replicates with CouchDB and sends
the devices a complete snapshot sqlite file for the initial setup. After that,
they can talk to CouchDB directly. Otherwise it could take like half an hour
to initialize a 50MB DB on a device, which is not really feasible.

~~~
greendestiny
In fact for a while you could only have adhoc views, but my knowledge is 6
months out of date.

Yes definitely do that. I bundle the sqlite in the app store updates and then
mount that as an external db and pull from itself - its a bit awkward but it
worked. I had to write the external db stuff, but it was pretty easy.

------
adjwilli
Yeah with my apps I haven't noticed many iCloud related problems. I generally
however just store simple keys in iCloud then pull the real data from my
server based on those. It's not as robust as I'd like it but works.

------
firloop
A quick Google search provides this article:
<http://createlivelove.com/238/why-i-hate-icloud/> that explains his hatred,
but it's not terribly detailed either.

~~~
jonknee
It's also linked in the OP...

------
seivan
I write my own backends. I feel safer that way. Already got a framework for
communicating with a resource based API properly with its local model (Always
Core Data unless not trivial like a map).

------
archagon
I've never worked with iCloud, so I'm curious: why do devs say that Dropbox is
so much better than iCloud? From what I've seen, both work similarly in that
they abstract out the cloud as part of the file system. Wouldn't you still
have to resolve conflicts and do all that other dirty stuff if you use
Dropbox?

Edit: potatolicious has a pretty great explanation. It looks like it's mostly
CoreData issues. But for those cases, Dropbox wouldn't do any better, right?

~~~
rdouble
Dropbox isn't a great substitute for iCloud when it comes to CoreData because
it only does file syncing.

------
cpr
I've heard these rumblings for quite a while--haven't done anything with
iCloud support myself--, and, yes, we need some details of what's not working.

Or maybe the article's implying no one really knows?

------
bitcartel
I posted this exact same link a few hours ago, and it was accepted as a new
submission. So what's going on? Can moderators reassign story submissions?

~~~
nkurz
All of your submissions for the last week have been autokilled. Something must
have triggered a spam detector. Or maybe the Youtube submission (brilliant
video) did it. Your comments are still visible, though. Writing polite email
to PG is probably your best recourse.

------
andrewljohnson
Sorry, but it sounds more like user error.

I won't argue that CoreData has its pitfalls, but at the end of the day, many
of us both write our own syncing solutions, and successfully use iCloud.
Neither of these things is actually all that hard, and you certainly don't
need to use iCloud to have your apps featured.

~~~
igravious
Uh, see this link up yonder: <http://news.ycombinator.com/item?id=5185497>

~~~
taligent
The OP said CoreData has pitfalls and they are 100% correct i.e. the link you
provided.

The OP said you can roll your solution using iCloud and it will work and they
are 100% correct.

The OP said you don't need iCloud to be featured and they are 100% correct.

Which point is wrong exactly ?

