I've worked with iCloud in conjunction with Core Data. It's as broken as the article describes, and in fact has gotten worse over time.
No one really fully knows WTF is going with iCloud - documentation is basically nonexistent, support unavailable, implementation broken in trivial ways, and error messages inscrutable. I was at an Mac/iOS conference recently where the Core Data iCloud talk morphed from a standard talk into a "here's the mic, anybody have any idea what's going on with this thing" affair. Helpful, but indicative of the state of things.
IMO, from having studied this for many, many hours over many moons, is that what Apple is trying to do is fundamentally hard, if not entirely unfeasible. They're trying to replace a smart server with one that's dumb as a doorstop. More specifically, they're trying to emulate a CRUD web service with a file sync engine.
Conflict resolution is left up to individual clients, since the server doesn't do any "thinking". Likewise, there is no canonical, authoritative state of the store, since the server doesn't "think", only the clients do. Apple was hoping that by shoving a bunch of diffs of your database onto the server, that clients can reliably reconstruct a sane database by playing them back - except that multiple clients are updating the diffs simultaneously and there is no server-side conflict resolution.
Oh, and if stuff fails, there are no regular snapshot states to fall back to, because iCloud is a file store, not a database engine. Your whole store is now corrupt. Enjoy.
Oh, and there's no way to nuke the database and start over - there are metadata files that are undocumented or that we don't even have sandbox access to, that interfere with completely destroying the database.
At this point you might think "Fine! I'll go to my Settings -> iCloud -> Manage Storage menu and delete everything associated with that app and start it off on a clean slate!"
And then you find out that, even today, in iOS 6.1.3 land, there is a bug from iOS 5.0 where if you delete an app's iCloud bucket it can never be created by that app, on that account, ever again. I wish I were exaggerating. In perpetuity. Presumably some state gets fucked and now this app is forever more unable to store anything on iCloud.
To make even the most trivial experiment on the iCloud API involves creating a brand new app.
iCloud (in so far as it relates to Core Data and document storage) isn't just buggy, isn't just poorly implemented - it's non-functional.
Good lord, that sounds like a complete mess. I haven't been anywhere near it as a developer so I can't really speak on how bad it is from that point of view.
As a user though I have absolutely no idea what is going in with Apple syncing. My wife has an iphone / mac syncing explosion that I simply can't fix. I just don't have any idea where the data is or why everything is duplicated in her iphoto collection. Or what's going to happen in when you remove something from one place.
Most of all I cannot for the life of me explain to my wife why it is that when she lost her phone and restored a previous backup to a new one it DELETED ALL HER RECENT PHOTOS from off her mac without warning. Gone, forever. I watched it happen before my very eyes and it's made me want to never buy another apple product again.
"Good lord, that sounds like a complete mess. I haven't been anywhere near it as a developer so I can't really speak on how bad it is from that point of view."
Let me tell you it's not exactly happy land for us consumers either. Apple's "cloud" is a mess. And don't even get me going about the "new" and "improved" iTunes. Frankly I pity developers, cause Apple's mess is giving them a bad name.
I can't think of 1 icloud enabled app (on Windows or Mac) that I've used that worked well. The apps themselves are fine, it's the cloud integration that fails. I mean just look at Apple's own forums.
After 10 years as a Mac user, I'm actually considering moving back to Windows. I'm just over it. It's like every new OS (or app) something goes missing. I'm tired of spending hours searching for replacements for features Apple has removed. And I'm tired of being force-fed services and apps (that Apple's PR states is "gonna change my life") still in beta.
I switched from linux (after 9years) to an iMac and iCloud was one of the first things I couldn't believe how poorly it works.
Ubuntu One sets a standards it seems. Right click your desired folder click ubuntu one and it gets synced. with iCloud (and box, dropbox) I have to duplicate the folders, weird.
I spare you the finder, sftp-integration and the keyboard-layout bitching.
Were you using timemachine, and if so did the photographs get removed from there as well? No matter what, that sounds like an incredibly poor user experience.
> "The only downside of not using shiny but questionable new features is that your app won't get featured in the App Store."
This is a pretty huge downside. The App Store is a huge mess - there is effectively no web frontend for it, it's tied deeply into a desktop application, and it's obtuse. This makes marketing nigh impossible and results in Apple having an enormous, and disproportionate, ability to pick winners and losers.
>The App Store is a huge mess - there is effectively no web frontend for it, it's tied deeply into a desktop application, and it's obtuse.
I was a little shocked when the switched to the 'card UI.' On a mobile device where there is very little real estate, they completely nerfed the amount of app information on the discover screens. Added so much physical effort and time it takes for the user to peruse a like number of apps compared to the former UI.
Core Data is barely functional locally, requiring an enormous amount of invasive runtime hooks to create the appearance of a "transparent" (it's anything but) object store on top of Objective-C objects.
The fact that anyone, anywhere (including Apple), actually expected Core Data to work as a distributed object store is totally beyond my comprehension.
> I have no experience with iCloud, but I'm pleased with Core Data for managing a persistent object graph. It automatically handles version migrations, undo, and faulting, and some interfaces can be populating with almost no code via bindings. I'm happy with it. Still, I'll be very interested to watch the Core Data presentations at WWDC.
Really? Not had any problems with Core Data on top of the default SQLite store (not iCloud, mind you). The interface is very reasonable, since it doesn't pretend to be transparent, and in my experience it works as documented. Care to go into detail?
I work on a fairly complex Core Data implementation in a heavily multi-threaded app.
It requires a lot of discipline, and we rely on a lot of learning-through-pain since the documentation beyond the most basic implementation is mostly non-existent.
Core Data is pretty simple if you do all of your work on the main thread, but then your'e also blocking your main execution loop for operations that can take a long time. A large save will easily reach into >200ms land on an iOS device, and your app will feel like crap.
Thread safety in NSManagedObjectContexts is somewhat documented, but has a lot of quirkiness to it. There is some general knowledge around what is and isn't thread-safe, but almost nothing surrounding how to build a multithreaded app and not be murdered by Core Data. Much of this is trial and error and observing what open source projects are doing.
Once you settle into a decent threading scheme that won't kill you (probably with multiple NSManagedObjectContexts) you have to worry about communicating changes across multiple contexts - then you delve into parent-child contexts, where the document gets even more bare. Then you get into propagation delays when crossing context boundaries.
Keeping your code clean through all of this is also a challenge. There are also runtime issues - our implementation in Core Data experiences unbounded growth in memory usage on iOS5, but not in iOS6, forcing us to manually mitigate memory use (which Core Data is supposed to take care of), creating even more thread-safety concerns.
All in all, it's a pain, but it's also the only realistic object store/graph on the platform.
> A large save will easily reach into >200ms land on an iOS device, and your app will feel like crap.
Can you give an order of magnitude at this would happen? I usually only have 50-200 entities in my database in total, saving maybe half of them at the same time. Yet libraries like RestKit insist on a two-context setup to avoid the saving delay you mention. I'm not sure if I should fight the libraries and go back to a single-threaded setup (that I understand) or just go with the parent/child context flow.
Fetch and save operations get pretty slow with even just 10,000 entities (if you touch all those entities). It gets a little better if you pre-fetch all the entities rather than one at a time, but the save is still very slow. Breaking that out into a backround thread is feasible, but as the previous commenter mentions, then you risk momentary divergence between the UI and the backend.
This is fetching every entity by its indexed key. My understanding of saving in a background queue is that during that 1-2 seconds, the ui will still be using the pre-modification data set, so very briefly the ui may appear incorrect, no? Not a huge deal, but if the ui will be wrong, then for our app that would still force us to pause the ui until the save finishes.
I just hope people can differentiate between Core Data and iCloud from both your comment and the article.Core Data isn't broken. It might be somewhat magical, but it's not broken.
2) Core Data requires making your model mutable, and spreading dependencies on Core Data's abnormal objects throughout your entire code base.
3) Since the abnormal objects are spread throughout your code, and they're tied to non-thread-safe NSManagedObjectContexts, concurrency becomes rather difficult. You have to deal with multiple managed object contexts becoming out-of-sync across threads. Most people just give up and perform expensive operations like -save on the main thread.
4) Core Data merges the concepts of your on-disk model with your in-memory model, despite the fact that the two problem domains have very different requirements in terms of API design, data longevity, relational design, etc. This leads to code that is neither a clean in-memory model, nor a clean on-disk model.
5) Since Core Data objects are abnormal objects, one must produce a considerable amount of boilerplate just to define new model objects. What time you save upfront using a GUI to design your original model, you spend 10-fold in maintaining the code to support it, and in being handicapped by the contraints Core Data places on your code.
6) NSPredicate is awful. It's a brain-dead query language that often make it impossible to cleanly normalize your data store. Aggregate operations are expensive and often require O(n) cost, the query language is poorly documented and poorly specified.
Core Data is a very poorly fit abstraction, but it probably works well enough for simple use-cases. Once you move into concurrency, complex/large models, aggregate operations/projections across your data, or have to deal with it failing, it falls over.
After using it myself for a large project and struggling with all of these issues (and more), and watching another team do the same on their own project, I'd never make use of it again. SQL (via sqlite) is a far better abstraction for managing on-disk data, especially when doing so in a highly concurrent application where one can rely on SQLite's transaction support.
2) I don't get this. Your model is tied to a context manager, even if they are mutable, you don't need to merge context throughout your base unless you want to.
3) They are not thread safe, the same way UIView isn't thread safe. Why would multiple managed context become out of sync unless you allow it? It's not difficult to deal with. Create a context in the queue you are in. Merge it when you're done.
4) You can use Core Data without persistent store and make it fully in memory and everything will work fine. You can even separate your memory context with multiple context managers. Separating your creates and your deletes and etc.
5) Fud. Write a wrapper, use existing wrappers.
6) Agree.
I am actually kinda curious what problems you've gotten into recently. I've seen Core Data be improved ever since I started using it in 4. I wish it had Cocoa bindings on iOS... but ah well.
I've never had any concerns here.
I don't use it for complex stuff. Ironically I didn't use it for a graph though I was told it would be a perfect fit, i wasn't really sure how to traverse without pulling everything into memory. So ended up just using NSObjects and keyedarchiver. But I would be curious to see if someone more familiar with Core Data would have solved it with CD.
I touched on this in a peer comment, but it ties your hands in terms of doing rather normal things that would make for a better model API.
> 2) I don't get this. Your model is tied to a context manager, even if they are mutable, you don't need to merge context throughout your base unless you want to.
Using an immutable model makes concurrency far, far easier (among other things).
> 3) They are not thread safe, the same way UIView isn't thread safe. Why would multiple managed context become out of sync unless you allow it? It's not difficult to deal with. Create a context in the queue you are in. Merge it when you're done.
Like I said in reply to your other comment -- The difference is that the 'model' is what binds all of your code together. The constraints of Core Data therefor introduce tightly bound constraints on all your code.
> 4) You can use Core Data without persistent store and make it fully in memory and everything will work fine. You can even separate your memory context with multiple context managers. Separating your creates and your deletes and etc.
I think you're confusing what I mean by in-memory model and on-disk model.
The way one defines and represents on-disk storage is very different than in-memory storage: the data is persistent, it must remain accessible over time, the model must be versioned, and changes to the actual model definition must be applied as migrations between versions.
In-memory models, however, live only for the lifetime of a particular run of that application, may be modified freely, require no versioning, and require no migrations other than code refactoring.
By tightly binding these things together, Core Data creates the worst of both worlds: all the complexity and downsides of on-disk models with almost none of the flexibility and simplicity of in-memory models.
> 5) Fud. Write a wrapper, use existing wrappers.
That involves quite a bit more work than defining a simple class. Tools such as mogenerator are big, complicated, and a hassle. The point of Core Data is to save effort, but it generates more work and complexity.
This requires performing all mutation on dedicated code paths. If you require non-asynchronous completion, you have to use the 'wait' variant, but this introduces deadlock risk between threads.
This in turn litters your code with Core Data isms, and means that one can not simply make use of their model as a standard in-memory model.
Compare to SQLite, where one can simply open and commit a transaction synchronously on any thread.
> Litters the code with Core Data ism? The same your code is littered with UIVism, NSFoundationsm and etc?
The difference is that the 'model' is what binds all of your code together. The constraints of Core Data therefor introduce tightly bound constraints on all your code.
> Please stop using the word thread and use the word queue :)
Why? Queues are M:N mapped to threads, so they are still threads. Additionally, not all work is always done on queues, especially when working with legacy APIs.
That's a pretty carefully thought out list of grievances.
What do you mean by 1), though? The link shows a list of methods you can't override on NSManagedObject, but they're things like -class and -isEqual:, which seems pretty reasonable for a system that needs to inspect and compare its objects.
Core Data does solve a few tough problems well. Faulting, uniquing, and cross-context change merging are probably the top three. Yes, it can be much simpler to just query SQLite, but I've never seen somebody do that and abstract away updating stale copies of objects.
> What do you mean by 1), though? The link shows a list of methods you can't override on NSManagedObject, but they're things like -class and -isEqual:, which seems pretty reasonable for a system that needs to inspect and compare its objects.
Well, keep scrolling :~)
Especially 'Custom Instance Variables' and 'Custom Accessor Methods' and 'Validation Methods'. I don't think it's controversial that defining derived instance variables, accessor methods, or other derived state on model classes is an unusual or unwarranted desire, yet doing so with Core Data requires considerably more effort and complexity than a simple model object.
I've subclassed NSManagedObject and added instance variables and custom accessors literally every time I've written Core Data code. It's usually not a problem -- yes, you do have to do a couple things to preserve the underlying behavior.
I've found the 'couple things' to introduce enough overhead, that when coupled with the other downsides, Core Data hasn't saved me time, reduced LoC, provided any performance advantages, or improved code quality.
Well, they are; they inherit NSManagedObject, which descends from NSObject. It's true that model objects need to follow a bunch of conventions, but that's by design.
> Core Data merges the concepts of your on-disk model with your in-memory model
That's a feature. I rather like how it forces you into rethinking your design.
From your other points it sounds (mind you, sounds, I am not judging) like your app was the wrong fit for Core Data. It's not a replacement for a full-fledged SQL RDBMS and was never intended to be. Core Data works swell for its intended scope.
I don't know anything about Core Data so pardon my ignorance. How has Apple marketed Core Data? I'm asking because the issues you've mentioned are solved well by SQLite, but perhaps that's because Core Data wasn't designed to be used in anything but the simplest use cases? That tends to happen with solutions that are magical.
Core Data does look pretty awful if you compare it to an arbitrarily awesome imaginary library.
All you're doing in this post is phrasing problems that are fundamental to data storage in core-data specific verbage, and calling them a grievance against Core Data. But they're not. There's no other magical solution that somebody could write that does it any better.
For example:
> Since the abnormal objects are spread throughout your code, and they're tied to non-thread-safe NSManagedObjectContexts, concurrency becomes rather difficult.
Concurrency is difficult independently of how you go about it!. The fact that concurrency continues to be hard is not Core Data's problem. The question is, is it easier or harder to use Core Data's primitives to achieve concurrency than it would be doing everything from scratch? I think it's easier. But Core Data against magic is not a fair comparison; a fair fight is between [hard problem with off-the shelf tool] against [hard problem without any tools].
Or this one:
> Since Core Data objects are abnormal objects, one must produce a considerable amount of boilerplate just to define new model objects.
Again, the choice is not between [boilerplate] and [no boilerplate]. As you've correctly pointed out, the mapping between on-disk and in-memory is a hard problem, and the solution will involve writing code that bridges the two worlds. The question is, is [boilerplate that's defined over 10 years of Cocoa convention] superior to [boilerplate that I invented from scratch to solve my problem]? I'm going to claim, 90+% of the time, that you want door A. Perhaps you really do want door B occasionally. But there is no door C with a boilerplate-free existence.
> After using it myself for a large project and struggling with all of these issues (and more), and watching another team do the same on their own project, I'd never make use of it again.
And as a person who has worked on probably over two dozen Core Data applications, I would not touch any codebase that had this philosophy with a ten-foot pole. Concurrency, boilerplate, memory/disk impedance mismatch are the same problems wherever you go. It is perfectly fine to say "I don't like using CD syntax to solve these problems" as a personal preference. But it is another thing entirely to say that CoreData is "broken" because one is either unable or unwilling to solve these same problems we solve every other day of the week in a slightly different syntax than one is used to, particularly when that syntax is generally accepted as a platform standard.
To use an analogy, it would be very poor form to try a Ruby project or two and conclude that ActiveRecord is terrible and that it "falls over" once you step outside "simple use-cases" and that you would never do another Ruby project unless you were emitting postgres queries by hand. There are not a lot of Ruby developers who would sign up to work on that project.
IMO, from having studied this for many, many hours over many moons, is that what Apple is trying to do is fundamentally hard, if not entirely unfeasible. They're trying to replace a smart server with one that's dumb as a doorstop. More specifically, they're trying to emulate a CRUD web service with a file sync engine.
Having dug through iCloud's underlying structure, I can tell you what they're doing is very close to what Microsoft's Sync Framework does - right down to the naming of some of the columns in the underlying database.
In fact, if I didn't know better, I would think it was a peer implementation of the Sync Framework (which considering the iCloud server is hosted on Microsoft Azure which has native support for Sync Framework built into hosted SQL Server, isn't too far fetched).
The difference is that Microsoft's Sync Framework seems to work.
> IMO, from having studied this for many, many hours over many moons, is that what Apple is trying to do is fundamentally hard, if not entirely unfeasible.
"Distributed Databases are Hard" has been a mantra for many decades, probably getting close to a half century. We have known, basically forever, that this is hard on a fundamental level. (Google "Two Generals" problem.)
> Apple was hoping that by shoving a bunch of diffs of your database onto the server, that clients can reliably reconstruct a sane database by playing them back
This is called a variety of things. One of the terms is "transactional replication." It can get very very sticky when there is more than one party originating write transactions, though.
That's funny, I had no clue how it's supposed to work as a user either. Or at least not in the 10 minutes I was willing to give it. The Steve Jobs' Dropbox story is even more hilarious in retrospect. Hilarious at the time too, because Jobs had no intention of providing his product(iCloud) to the philistines who use Windows.
I just change the topic to a different backing store when the client wants "cloud persistance" and says "iCloud" without understanding the pain they're asking for.
From what I've heard iCloud is primarily Java (and some C++)
Not that I'm blaming the technology specifically, but my impression is that Apple is being naive.
Of course you can build with Java, but if you knew better and have the resources Apple has, you would be aiming somewhere else. Google can do this because the infrastructure is there.
Not in the case of Apple
And as this shows, the concept itself of how the service works is broken. This is smelling of 'walled garden fundamentalism' + 'technical ineptitude'
You realize that a plurality of Google's backend is built in Java, right?
I know it's fashionable to hate on Java, for a lot of good reasons, but I think it's naive to dismiss the power of the platform itself.
We may all have (completely valid) complaints about the outdated and verbose syntax of the language, but the actual JVM itself is fantastic. There are more scalable, fault-tolerant, high performance Java systems out there than the rest of the implementations on all other platforms, combined.
"You realize that a plurality of Google's backend is built in Java, right?"
Yes, I know. The problem is not Java directly. Reread what I wrote
Google has the infrastructure and resources to make Java work for them at their size. Because they have whole teams dedicated to build their infrastructure (FS, storage, etc)
Apple has to optimize their resources. They're not going to rebuild what Google did and I'm not seeing them using, I don't know: Cassandra, Hadoop, etc
What I'm saying is that Apple is probably trying to reinvent the wheel. With their resources, they should go with Scala, Clojure or use helper technologies.
Apple has 72,000 employees while Google only has 54,000 employees. Apple is over 30% bigger than Google. I would guess that Apple has lots of infrastructure people too.
Apart from that, in Google most (core) jobs are software related, in Apple not so much (there's hardware, design, logistics, manufacturing specialists - even if the bulk is @ Foxconn and similar contractors).
Even if we consider software jobs only, I believe the bulk @ Apple is working in OSX/iOS, not to mention iWork, Final Cut, etc and not directly in Cloud infrastructure.
Practically speaking, for the problem in question, none of these employee counts matter much. 200 engineers is a large number to put on any project, but a tenth of a percent of their total employee count.
"
Employees
As of September 29, 2012, the Company had approximately 72,800 full-time equivalent employees and an
additional 3,300 full-time equivalent temporary employees and contractors. Approximately 42,400 of the total
full-time equivalent employees worked in the Company’s Retail segment.
"
That page says Apple has 80,000 employees, leaving them about the same size as Google. Google has people on the payroll buidling Google Glasses, The Google Car, Gmail, App Engine, Go, Macs, Docs, Chrome OS, and a lot more projects than we probably know. Remember this:
http://www.google.com/enterprise/search/
It's true that Apple is highly focused, and they probably have half, if not a quarter of the products that Google has.
> They're not going to rebuild what Google did and I'm not seeing them using, I don't know: Cassandra, Hadoop, etc
There is a non-zero amount of irony in your bashing of Java then picking out two projects that apple isn't using, but you apparently think they should but using, which are both written in Java.
I am not bashing Java and I know what these are built on, but apparently no one read what I wrote and stopped reading at 'Java'
I am saying is that probably Apple has a NIH mentality with regards to infrastructure and thus are trying to reinvent things like that. And of course, reinventing these in pure Java takes more time than using Go, for example.
I am one of those developers who don't jump on the bandwagon right away. Experience has taught me to be very cautious. It also comes from doing a ton of work on mission-critical systems where you don't do stuff like install the latest software updates without massive testing on off-line systems.
Yes, sometimes I miss out on really neat developments. iCloud is one of those cases where I am glad for being a slow adopter. My mental red flags went up as soon as iCloud came out. Not one of my apps uses it.
On the end-user front iCloud is also an absolute mess. I probably have a dozen i-devices on the same account (testing, etc.). Crap appears and disappears from devices seemingly at random. The worst of it was what is happening with my wive's iPhone and mined. We share an iPad at home. Now contacts are getting deleted from her phone and mine at random and so are notes. It's at the point where she is not using notes any more because --her words-- they disappear within a hour or two of writing them.
Of course, turning off iCloud is the solution. The problem is that this isn't as simple as throwing the switch. If you turn off iCloud on your device it might actually delete things that it shoved into the iCloud service. And then it deletes the info from your other devices. And it does this in some amazingly weird ways. Without telling you what it is about to delete.
For example, I have never explicitly stored contact information on iCloud. Never. I type new or updated contact information into the phone. That's where it belongs. In fact, I never even use my iPad for this. Yet, some contact information for some strange fucking reason is sent over to iCloud. The first time I turned off iCloud piles of contact data evaporated from my phone. I had to turn the damn thing on again to get it back.
I haven't had the time to research how to fully disable iCloud on ALL of my devices while, at the same time, keeping or merging the state of my data on that particular device. As a developer I managed to avoid iCloud. As a user I was not that fortunate, I succumbed to the new shinny thing on the shelf. Now I want nothing to do with it. Thank you Apple.
The only thing I can think of is to set up contact, calendar etc. sync to Google (via ActiveSync or otherwise). Calendars won't magically transfer over since they're assigned to a particular calendar, on iCloud in your case, but contacts should get pushed to Google. That'll at least give you a backup of your contacts, and setting up the other devices the same way should sync them all up.
Can anyone else chime in? Does this sound reasonable? I've had everything on my iPhone synced up to Google services for about 4 years now and it's never caused any issues.
Frankly, I'm just thinking of writing an app to export everything to a MySQL database on one of my servers and do my own version controlled backup. Not sure Apple would ever approve such an app, but I can certainly write it and use it internally.
A long time ago I learned there is nothing more important than ensuring your data is safe. I learned this the hard way in college when a drive failure evaporated six months of hard coding...and I had zero backups. You don't do something like that twice.
With iCloud Apple has given us a tool that is so badly done it has the potential to evaporate our data. Amazing.
There's a reason Microsoft code runs the IT infrastructures of small, medium and massive corporations. They get it. They don't engage in making petulant adolescent choices for their users. Crap like removing "Save As" from applications and forcing duplication --and even storage-- of files on iCloud from your desktop.
I've been using Apple products since the first Mac. I could not see any way to trust Apple with enterprise data. No way.
That's the downside of keeping your data in a service. This is a conceptual problem due to the lack of agreed upon data protocols. Google Reader has highlighted the issue well, and more services are to follow I'm sure. This is an issue that never existed for files since they by definition conform to a specific format.
RSS and CalDAV are examples of web data formats which do work. I'd like to see more formats like that, or perhaps an ability to fall back to export data as files from web services.
It's interesting how the iCloud branding plays into this. Because Apple includes so many services under the umbrella term iCloud, it can lead to a very different impression of the service between the users and the developers. All the talk about "iCloud is broken" is about one specific part, Core Data syncing. The rest, like online backup and document syncing and calendars, seems to work most of the time. I actually saw some comments elsewhere along the lines of "iCloud works fine for me as a user, so it's probably just bad developers who can't get it to work, not any issue with Apple."
Even more than that, iCloud Core Data syncing appears to be so broken that approximately nobody is able to even ship an app that uses it. Thus, users are never exposed to the brokenness because it's so broken you can't give it to them.
But only most of the time. Last week, my iPhone inexplicably deleted all the pages in my Notes application that I made in February 2013, around 20 of them. Fortunately, they're still there in my synced e-mail account, but there appears to be no way to sync them "back" onto my iPhone.
And then last month, syncing my iPhone with iTunes inexplicably deleted the 40GB of my music from my iPhone. I hadn't done anything differently from the hundreds (?) of other times I've synced. I just had to go one-by-one and re-check all the playlists I sync, and wait a few hours for all the music to be re-copied. But still.
As easy as Apple products are to use, I'm always afraid something is going to go horribly wrong, in a way I never was with PC's, for example -- they were harder to use, and did less, but you could at least be reasonably sure of what was going on behind the scenes.
(Shameless plug) Firebase is an alternative to iCloud for many apps. It's built as a database, rather than a file store, so it's ideal for storing structured data. Conflict resolution is something we've been thinking hard about from day 1, and we think we have a pretty solid story [1].
We launched native iOS support a few weeks ago [2]. You can also access your data from JavaScript or from a REST API. Android and other platforms are in the works. We're a platform-independent service, and since you (the developer) is paying for the storage, we have the right incentives to give you maximum access to your data (unlike iCloud), and a good developer experience.
We've also put a ton of work into making it easy to learn and easy to use. See, for example, our quick tutorial [3].
If you've used Firebase as a substitute for iCloud, we'd appreciate your thoughts in the comments!
Hey, I've been using Firebase for a web client I've been working on for a bit and I like it a lot! Would you consider updating your transactions page with an example of a text change? Such as adding a word in the middle of a paragraph. I'm not quite sure how that would work.
Collaborative text editing generally requires a bit more than a simple transaction to work well (Operational Transforms are generally involved). You can use transactions as a basis for this though. We've got some examples coming out soon to show how to to do this. Stay tuned...
Frustrated with just iCloud? No I'd go further. I'm frustrated with the whole iOS platform. It's like writing an app in the 1990s. Sure it works. But it's painful as hell. So much so that I've decided this latest project on iOS was probably my last unless something big changes. And just for the record I have a few flagship apps in the app store with millions of active users and I've worked quite extensively with the platform. So it wasn't a case of trying it and then giving up in frustration. It's more like watching your peers blow past you because your toolset is from yesterday and their's is from tomorrow.
This sounds unfair. I have been developing for the platform right from the start and I am very happy about the whole toolchain. My only real gripes are with the review process, nothing else. (I have never used iCloud as a developer, so I can’t comment on that.)
I'm not OP, but one key complaint I have is that there isn't much in the way of testing. Unit tests are somewhat well-developed, integration testing is a complete mess.
Much of the iOS community still creates software by the seat-of-your-pants model (i.e., write code, boot it up, does it look like it works? ship it). Engineering rigor is still a concept relatively foreign to iOS, and it shows. Apple has little-to-no built in support for anything that smells like TDD.
When you're talking to your server-side peers who have good test coverage, good regression testing, and more generally just sound engineering processes, you can't help but get bit jealous.
I was a test zealot all my academic life (it's perfect for building compilers). But in my experience as an iOS contractor, most apps are polished Thin Clients for a remote backend. Which incidentally is the only thing that Objective C is really awesome for. As soon as I write any interesting logic, it is actually a bug, because it should be pushed into the backend. After all, it must behave the same on Android and on the web client.
That leaves us with bug classes like: multithreaded Core Data misuse, "UI can be broken by using three fingers exactly like this", popping a navigation controller as a reaction to a network operation while it is currently being pushed (based on network latency), networking code that does not properly handle session extensions while multiple concurrent threads are in progress, labels use the wrong font, code uses UIKit on the main thread, ...
Good luck writing tests for that and being more cost-effective than code reviews & manual testing.
I feel like this has less to do with iOS and more with the challenges of testing client-side code vs server-side code. Even Android projects are quite tricky to test despite Java having a strong heritage when it comes to unit testing.
Part of it just comes down to the way the GUI framework (whether it's Cocoa or Android) cuts across all aspects of the application, making it difficult to isolate individual components to test. Add to that a bunch of asynchronous code (which is almost always necessary when you start dealing with the database/network/filesystem) and you have something quite tricky to automate testing for.
Most of the server-side code I've dealt with has been an order of magnitude easier to test, but I think a lot of that comes down to the nature of their respective domains.
Actually this is still the reality in most corporations out there.
I seldom see TDD, unit testing, continous integration and so on outside the startup world.
Most of the projects I do in the enterprise start agile and slowly evolve to a mix of agile and waterfall. When stress for delivery raises, many entrerprise project managers just kill unit testing related tasks.
I mainly mean my web-developer peers. Their toolsets are moving so rapidly that I see them surpassing us in the next 2-3 years unless we make some huge improvements. Just my opinion.
Web development is quite a different beast. There’s huge amount of life and struggle in the ecosystem, but only a fraction of that pushes the platform forward. I am relieved every time I move from some web development work back to plain, sane Cocoa APIs.
And just for the record I have a few flagship apps in the app store with millions of active users
Source please.
because your toolset is from yesterday and their's is from tomorrow
This is why I don't believe your first point without sources to verify. Eclipse+Android SDK is hardly from tomorrow. Windows 8 possibly. What else does that leave? Nada.
Those are just two that I have built. I have an enterprise platform that I architected from the ground up that companies like pfizer use for it's whole workforce. I have ones written for Disney, Hearst, GE, MaryKay, Hallmark, Sesame Street, etc, etc. The one I'm building now is ready to scale to ~5M users right out of the gate and I believe we will succeed.
Now put your apps up and let me pick through them and tell you why you're wrong because I just don't believe your work is credible.
Hey man, not sure how worried about your info getting out there, but generally when people troll for 'proof' here I ignore it. Essentially it's just one more avenue for people to use against you, especially in regards to linking a name/identity to work and I generally (personally) try to keep them separate. Your choice, just something to think about... also, don't feed the trolls etc ;)
Well, I don't really agree, but I will say that the iOS testing world has a lot of catch-up to do. Ruby developers (as an example) have much, much, much better testing tools. There is no doubt about that. The hoops you have to jump through to create acceptance tests are quite annoying, especially when you can't even test UIAlertViews, or basic things like Airplane Mode...
I don't think that's a fair comparison. Testing GUI code has always been a lot trickier than testing backend code, and iOS apps tend to be GUI-oriented while Ruby projects tend to be backend-oriented.
That's not to say that the iOS testing tools are great, but I've never had a particularly pleasant time testing GUI code on any platform.
True, Ruby doesn't usually get used for GUIs. But take a look at the Javascript and browser testing tools that are available. Is there anything like Sauce Labs for iOS?
Why not jump into RubyMotion and use these testing tools?
And Apple's WWDC session about acceptance tests using Instruments UI Automation specifically handles UIAlertView.
I think the problem is that iOS apps are getting dumber and dumber while their UI bling gets more complicated/asynchronous. The value of tests is too low for The One Widely Accepted Testing Framework to emerge.
HTML5 and all the surrounding technology is very exciting IMHO. They have the best tools and are quicker to adapt new techniques because of they get to work with a net (the fact that they can rollback quickly if needed). I get a little jealous when I see how quickly they can get something up and running that would take me a few days.
I've worked with both and I can say this is just a case of "the grass is greener".
Nothing "exciting" about HTML5 and the Web for people well versed in a mobile framework. The most interesting things are re-inventions of the wheel, only more hacky and less performant.
Plus, since you can use web views in your native mobile app, there's literary nothing (technologically-wise) you can achieve with HTML5 that you cannot achieve with a native framework. The inverse is not true. Anything that requires extensive collaboration with the underlying machine, low latency, ad-hoc connections to interfaces etc is not possible (you cannot write Garageband with HTML for example. And Flipboard or even Clear would have been noticeably slower).
(Except the distribution method of course -- but then again, nowadays people still debate if it's all about native or it's all about the web, so that's like 50-50.)
Yea I tend to agree on a lot of those points. I'm currently working on a native application that does heavy AV compositing and there is no way that HTML5 solution would be as capable.
Mainly the pain points are on the API and the legacy nature of it, and the surrounding crappy tools. I see cool IDEs, testing frameworks, scaffolding/bootstrapping sets and I can only be a little jealous. Sure we have our share, but it's a small slice of where the majority of effort is in the web world.
My hope, crossing fingers, is that one day HTML apps will truly be able to compete across all facets and access native functionality with the same performance as native platforms. Will this ever be the case? I can only hope.
I think the open source toolchain for Cocoa is pretty great. There are hundreds of incredibly useful projects on Github, like RESTKit or CocoaPods or AFNetworking, libraries to ease the pain of Core Data fetches, etc.
ObjC does have a legacy nature, but I think the community is doing a great job of making life easier.
I went to disabled iCloud on one of my Macs and I got this warning:
"If you turn off Documents and Data, all documents stored in iCloud will be deleted from this Mac"
Let's get past the jaw-to-the-floor, how-dare-you, !!!$$%#$## moment. They are morons and don't respect your data.
So, how do you figure out just what is in iCloud? I don't see an easy iCloud viewing option in Finder.
Doing some digging this is what I found:
- Open Finder
- Fire-up "Go To Tolder"
- Enter ~/Library/
- Go into "Mobile Documents"
That's what's on iCloud. You have to manually dig through it all and figure out what's there.
If you delete anything there it will be deleted from all of your sync'ed devices, including your local Mac. It will also end-up in the trash folder.
You could copy it all to a USB drive, memory stick, or, better yet, Dropbox. Then you can completely disable iCloud knowing your data is safe and iCloud will never cause you further grief.
Yeah, that's just a hidden surprise for you to discover next time you sync, as I unfortunately discovered when it deleted my entire schedule from a corporate Exchange Account. So not only does it apply to your iDevice, it also applies to any accounts that you've linked to the device and conform to the latest generation of CYA extensions for the Enterprise.
And that's the problem in a nutshell. If it's consumer-facing it's years in development and polished. If it's for us lowly developers, well don't hold your breath.
My trust of iCloud disappeared when a restore from backup lost all important data (passwords, Google Authenticator keys, etc.) with no indication. The backup was listed as successful, the restore was listed as successful and yet the only data I needed it to restore was lost.
This was also when I learned that Google's process for restoring lost two-factor credentials ultimately ends in an email form which sends to higgins-prod-lost-hard-landing@google.com which been bouncing since 2011. Thank FSM I had some some backup codes printed…
I don't think that's right. Secure things in iOS are in the Keychain, and the Keychain is only backed up if you do an encrypted backup via iTunes. Other backup methods (iCloud and non-encrypted iTunes) just leave it out of the backup.
The scary thing is that you hope your apps can handle the case where they have local data but no authenticated user for it. Some of them just treat no authenticated user the same as a first start and reinit the local data.
The backup, encrypted or otherwise, doesn't contain the actual keychain file itself, but rather a plist export of a subset of the keychain.
Things that are marked as kSecAttrAccessibleAfterFirstUnlockThisDeviceOnly are never included in this backup file. Google's GTMOAuth library uses this value for its keychain records.
I think the main problem is that there's zero UI indication that your backup is incomplete and the Apple store staff aren't instructed to ask/communicate that. It'd be a very different story if using iCloud or iTunes without a password triggered a “Because our programmers are lazy, we're not backing up your sensitive data” message.
When ever this topic comes up, including at iPhone conferences I attend, no one seems to have ever heard about Simperium, which pretty much has as their mission statement to solve this problem and seems to have actually done so quite well (yet people are mentioning, quite often, only tangentially-related companies like Dropbox); I believe they are even a Y Combinator funded company... I would be fascinated by an explanation of "what they could be doing differently so that people would use them".
> "I would be fascinated by an explanation of "what they could be doing differently so that people would use them"."
I've also never heard of them until now. Perhaps they should show up to said conferences, or at least keep tabs on Core Data-related talks at conferences and give the organizations/speakers a heads up.
I've talked to folks from Simperium before and think that they have a good idea but I remain skeptical of their approach . In particular their Core Data syncing strategy.
As I understand it Simperium observes changes to the Core Data models you specify, maps those changes to operation transforms, and distributes the transforms across whichever clients have subscribed to the service. That's great if all your app needs is simple data replication to multiple clients. Unfortunately operational transforms are not necessarily snapshots of database transactions and conflict resolution strategies do not guarantee a valid database.
Saving a Core Data transaction on one client may generate multiple operations which are replayed interleaved with operations from other clients. Conflict resolution is then performed between operations rather than transactions. Simperium can probably take steps to minimize that case but as long as I can construct transactions which cannot be expressed as a single operation this will happen.
Worse a client can apply operations without conflicts only to find that its models no longer pass its own validation rules. For example two clients can each add a child object to a "has one" association. Both "create" operations can be applied without conflict but the parent object no longer passes the application's validation rules.
This doesn't make operational transform based synchronization untenable but it does imply a new set of constraints on how applications should model their data in order to successfully apply operations from other clients. That may not be a popular sales pitch but I'm reluctant to feed structured data through a service that doesn't at least discuss the model and its constraints in some detail. Otherwise I fear such a service is another iCloud like trap which will eventually (though perhaps less frequently) hit catastrophic edge cases from which there is little hope of recovery.
> Saving a Core Data transaction on one client may generate multiple operations which are replayed interleaved with operations from other clients.
Interesting; I've implemented OT myself, and this is not a fundamental failing of the concept: in fact, even fairly simple implementations like ShareJS do not succumb to this fate (the easy way to model it is that you end up with larger macro-operations that are capable of transforming eachother's components, while still having to be played back one macro-op at a time; you also just stop playback while a transaction is in progress). I am totally willing to believe that Simperium's implementation (which I believe thinks about the separate objects as separately transformable documents) may have issues here, though.
> Worse a client can apply operations without conflicts only to find that its models no longer pass its own validation rules.
This is certainly true, but is endemic to the entire idea of offline synchronization and has nothing to do with operational transforms: you simply can't do this without accepting "a new set of constraints on how applications should model their data in order to successfully apply operations from other clients"; this is clear from CAP (as rather than become unavailable when the network is offline/partitioned, we have to lose some consistency).
> That may not be a popular sales pitch but I'm reluctant to feed structured data through a service that doesn't at least discuss the model and its constraints in some detail.
This is totally fair. (I can entirely appreciate it as well, as it reminds me of all of my complaints regarding how seldom you would hear companies like Parse and StackMob attempt to drill in how important security should be while using their services, at best leaving it as an appendix in their documentation. I believe they have at least been getting somewhat better about this since my talk on the subject at last year's 360|iDev.)
There's been quite a few articles from people far smarter than I about it, and most of them agree that whilst doing a server backed database synch service is hard, it shouldn't be this buggy at this stage.
It's like Game Center, Apple doesn't bother with any apps that really use it so there's many, many problems reported on a consistent basis, and they don't seem to use iCloud for anything involving Core Data so they're not dog fooding and having engineers swear at code. No swearing means no one is motivated to go back and try improve it.
I tried to add iCloud features to an app as part of a contract gig. It was disastrous. The paying customer knew we were capable, but began to wonder what was going on with us -- it was frustrating to have our client lose a bit of respect for us when the cause was the brokenness of iCloud.
They, understandably, thought there was no way that something so heavily promoted by Apple could be such a lemon. And in spite of having heard mumblings of iCloud bugs, we wrote it off as amateur iOS developers.
We ultimately abandoned iCloud prior to release (going with simple Python based server for data storage) -- reading the other posts I'm glad we didn't try to convince ourselves that the 'almost stable' behavior was just a quirk of our devices having been used in development mode.
My theory is that the culture of internal secrecy at Apple prevents the teams from collaborating. So you get products that work fine in isolation. Also a factor would be date-driven-development. Steve would want to demo a new offering at WWDC, so everything was driven by making that date.
My hope is that now that one person (Jonathan Ive) has been put in charge of product development, things might get better.
For one, Windows Azure is a pretty big deal. Secondly, XBox Live is a pretty big deal. Third, Bing might be struggling to gain traction but it most certainly works.
So can you point me to a Microsoft web property or service that is as wonky as iCloud at all?
Apologies not needed. That raises a question though.
I was implying that Apple and Microsoft don't do as well building web services. I wouldn't argue that Azure can't scale but the overall approach seems to be a little bit "unfinished".
Let me explain, the only true cloud computing environment that really solves 90% of any companies needs is AWS. Somehow Amazon understands what the developer community needs. I'm really perplexed by Google not getting this right.
Apple doesn't and I don't believe Microsoft does either.
I don't think this is about technical chops. It's something else that I can't put my finger on.
Similar to how I don't understand how Amazon crushed Best Buy so quickly.
Windows Azure is a big enough deal, and working well enough, that Brent Simmons of Black Pixel (and well known indie Mac developer) has been recruited to do tutorials on the Mobile Data Services component for iOS.
As an Apple user, I have often consulted discussions.apple.com to find fixes for problems. A lot of times there were thousand post long threads with no official answer. It is a wasteland. I know they don't answer questions, but they could make an exception for the most important common issues. It's as if they put their fingers in their ears and sing "la la la" while we try to communicate bugs.
How can the richest computing company not answer questions put by thousands of users who have banged their head in the same bug? It's insulting to have this circus happening on their doorstop.
The unexpected appleID problem is, in my opinion, a world stopping bug...the kind that would cause a code red meltdown if it wasn't fixed 24 hours. This bug has been reported for years, with no resolution. The apple stores pretend the problem doesnt exist or they give you the runaround because their product that "Just Works" doesnt have a fix for it yet. My wife, who was a hopeless apple junkie, has decided that her current iPhone5 will be her last apple product...with her continued frustration with AppleId authentication being the straw that broke the camels back.
I read the whole article, and I read all the HN comments (so far), but I still have no idea what the "unexpected appleID problem" is. Care to explain, for the unintiated like me?
When it prompts you to log in, it will pull up your appleID for you, but it gets it wrong. It will pull up some random other appleID...sometimes an old one you created for iTunes using a hotmail account you haven't checked since 2004, but sometimes a completely random other person's appleID. The authentication is unpredictable too...my wife has successfully signed on to her current account using an old password and successfully signed on to an old account using her new accounts password. Most of the time she can't sign on at all.
If trying to juggle more than one apple ID in the same storefront is breaking the camel's back, she's in for a world of soul crushing disappointment with any less UX polished product -- which is pretty much all the rest of them.
Someone can't use an iPhone because they're juggling Apple IDs?
First, it supports multiple IDs, the limit is one ID per storefront or sync.
Second, if this isn't about multiple IDs but about multiple addresses for a single ID, and if you don't like it supporting the old addresses, go to http://appleid.apple.com and manage them. You can update, edit, and remove old email addresses, decide what your primary one is, etc. I find it helpful to ensure 100% of all services across all devices are using the same exact ID aka email address; one way to enforce this is to be sure your appleID account only has one listed.
On the other hand, if she has inadvertently created multiple IDs over the years, she needs to either settle on one ID per service (my wife has one for music, one for apps, one for iCloud), or she needs to ditch the extra ones and standardize on a single one.
To tell if they're the same or multiple accounts, again, go to http://appleid.apple.com/ and log in with the one you think is the master, be sure the others are listed. If not, they're separate IDs.
None of this stops her from using her phone. Conveniently, all these phones fine with or without the ID.
FWIW, on my Galaxy Nexus I find managing multiple Google Accounts and cloud data sync more precarious. Sure, there's an easy radio button to pick accounts in the Google Play store, for example, but once I made sure I'm using the same ID across devices I've never had a bookmarks, contacts, calendar, or messages sync issue with iCloud, while I am having to go in and reset sync settings for bookmarks, contacts, etc., every couple weeks on Jellybean. And I usually don't know the sync is broken again until I notice something's missing, check sync settings, and see the broken sync alert.
She doesn't use multiple IDs. She only uses one. But in order to make things work, she has tried everything in the book, including trying to sign into her old account when this stupid bug pops up.
And no, she can't use the phone as it was advertised. She has multiple apps that use iCloud for storage, and every time this happens, she just stops using the apps, as it can literally take hours of trying before she can successfully sign into iCloud.
> She doesn't use multiple IDs. But... trying to sign into her old account...
Sounds like more than one. She needs to quit using the phone with "old" IDs, she's confusing herself and the device.
Get logged into https://appleid.apple.com/ and get it sorted out there. If you can't log into that, get on phone with Apple support until you can. As long as you didn't two step auth it, the phone support can and will help.
I'm not trying to debate, btw, I'm trying to legitimately help. Like most of us here, we end up being the support person for our family's devices, and I have seen people get themselves in a jam w/ the ID before. Clearly defining the Primary Apple ID won't be resolved until it's clearly resolved in the web interface.
I don't need to qualify it at all. A wife rejecting the iPhone 5 over AppleID frustration is going to have to use something else. There are things that spec higher, are more hackable, or more "open", etc., but nothing with the iPhone's level of wholistic user experience polish.
So if the corner case of swapping user IDs for purchased items is enough to dump a phone, she's going to be desperately unhappy with the other alternatives out there which currently have countless more normal user UX annoyances.
You seem to have bought into the marketing, as an owner of a Nexus 4 I am unfamiliar with the "UX annoyances" that you are referring to. Could you be specific?
As an owner of a Galaxy Nexus (TMobile) running the latest Jellybean, I suspect you've forgotten what it's like to pick the device up for the first time, if you claim the user experience is as polished for "normal" (non-tech) people. You also would be alone among the crowd of Android journalists who love the OS and talk about how each release is more polished, closing the UX gap. Skim any honest pro Android site for a catalog of annoyances remaining.
Here's one for a new user: Let's say I'm looking at apps on my home screen, swipe left or right to see more apps. Great. I install something new. Not on the home screen. Swipe left or right. Not there either. This is an actual problem for normal users.
I also don't like having to pull out the battery every few weeks because the phone is frozen when I picked it up off the dock. This can happen from any background app. In the most recent case, turns out it was the foreground app, a clock called "Alarm" that I had coming up when docked in the landscape dock. Switched to Daydream, that problem went away, at least. But having to unplug the battery before I can make a call, definitely an annoyance. Glad it has a removable battery though, disassembling and reassembling the battery saves me having to press Home + Power for 10 seconds.
Ha, here's another one, as I'm writing this.
I just took a picture to talk about the UX of finding and sending that picture, but when I went into Apps and swiped left to look for Gallery (seriously, we can't just call it Photos?), the phone froze. Now I'm looking at the first page of icons half off the left of the screen, above a dimmed set of the second page icons. It's still stuck there as I type this. But the phone isn't frozen, I was able to take a screenshot of the built-in Apps browser being stuck halfway between two pages:
As I'm finishing this paragraph, it's still stuck. By contrast, I haven't seen an iOS device crash swiping between the app icon pages.
While finishing describing the above, the phone dimmed, and then locked, as it should. I unlocked using face recognition, and was looking at home screen. Tapped to browse apps, and ... still frozen halfway between two apps pages.
This is native core functionality, simply not working. And like now, these things happen when trying to do something useful. That's a user experience annoyance.
From what you describe you have a defective phone, you should get it replaced, your experience with your phone freezing isn't typical at all. My Nexus 4 has never once froze on me.
Also the Play Store has a default option called "auto add widget"[1] that adds a shortcut to your home screen of any newly installed app, I personally hate that default behavior and always uncheck that option.
Is that all you could come up with for UX annoyances?
I personally think the experience on Android and Jellybean in particular is first class, if you just take things like the notification shade, the intents system and widgets, the UX on Android is unmatched.
> Is that all you could come up with for UX annoyances?
Nope, but I refuse to be trolled by someone suggesting there aren't any. Just because YOU may not be annoyed, does not mean there isn't consensus that it still has a ways to go.
My phone didn't freeze when swiping the apps pages. Only the UI for swiping the apps pages froze. The rest of the phone, and all other running apps, are fine. That's not a defective phone. That's defective software.
Note that I never said Android Jellybean wasn't first class. I said if the woman is ready to dump a phone over trying to use an old account ID, she's going to have bigger problems with other operating systems which are not yet as polished.
“... A related issue is that the iOS interface is simply cleaner and more user-friendly than any Android interface I'd yet to see. One of Apple's slogans is "It just works." Well, actually sometimes it doesn't work. iTunes, for example, has been annoying me for years now. But, when it comes to device interfaces, iOS does just work. Android implementations, far too often, doesn't.“
“So, yes, Android does more today than Apple's iOS promises to do tomorrow, but that's only part of the story. The full story includes that iOS is very polished and very closed, while Android is somewhat messy and very open.”
If you cannot imagine any UX annoyances, then most likely you fall into the “Android fans can be blind to its faults just as much as the most besotted Apple fan” camp.
I have been surprised at the relative lack of interest from developers about first class syncing support, when mobile blew up I expected a flurry if libraries and tools that I could use to have my app work offline and sync data wherever I am.
Sometime last year I gave up waiting and started working on PouchDB (http://pouchdb.com/), its based on CouchDB's syncing model (and syncs with CouchDB) and works anywhere that Javascript runs (the browser + node), there is an iOS / OSX equivalent (https://github.com/couchbaselabs/TouchDB-iOS).
PouchDB isnt quite ready for production but its getting close, I would love any feedback on it, although I expect most people commenting on this would be more interested in TouchDB
I'm very interested in this, in fact I'm planning on rolling an iOS based couchDB solution for a company sometime this summer. Could you comment on some key differences between pouchDB and touchDB? Did you also try to simply run ARM compiled couchDB on iOS? If yes, how was the performance there? I've read that initializing takes about 10 seconds - I'd be willing to swallow that if the subsequent interactions are smooth.
How about the initial replication of larger databases? Do you roll out an initial version directly in your app or do you have some file based download of a snapshot?
The key differences between touchDB and pouchDB is that touch is written for ios and pouch is written for web browsers, if you are building for ios and not using phonegap then I suggest using touchdb. It is also the more mature of the 2 but hopefully we arent too far behind with pouch.
I used to work for Couchbase on the arm compiled couchdb mobile versions, mainly android but either way I wouldnt hugely recommend using them, they always worked against the grain of the platform, never had great performance and due to those reasons they didnt really get widely adopted and therefore I wouldnt trust their stability across a variety of platforms and use cases.
The problems with arm compiled couchdb is the reason TouchDB was made, and similiarly with PouchDB / TouchDB for android and CouchDB I hope we get to the point were there is an out of the box sync solution that works great on every platform, it doesnt feel too far away.
For large initial datasets I have tried taking a dump of the database and importing it on first load, it worked however replication has been more than sufficient for my use cases right now. Once pouch has stabilised and released there are a lot of low hanging fruit that can be done to optimise replication (and in particular for first load).
The TouchDB project was started at Couchbase, and since then we have ramped up our investment in mobile sync. The key thing about this project is that its designed to be tiny and boot fast. So it is written in Objective-C (and Java for Android.)
We are renaming TouchDB to Couchbase Lite (still 100% open source), and have a bunch more tools for it:
We are 100% serious about building the best damn database for mobile. Bonus points that it will seamlessly interact with other projects like PouchDB and friends via the sync protocol originally written as part of Apache CouchDB.
I don't mean to be flip but how could you think a company that has to take down their store to change the product inventory would be a good bet to build solid web services?
When I still had Mac blogs in my RSS reader, the store closing down for a few minutes would actually cause wide coverage with a follow-up post on what how their inventory has changed.
In my last startup, I used TouchDB and found that 95% of the time, it worked great, and 5% of the time, the app would grind to a screeching halt.
Reading documents from the local TouchDB store would block the main thread until the upstream CouchDB server would reply with the document's current revision, even if the local store already has an existing version of the document. Using GCD to wrap around these document transactions was still clunky at best and resulted in a poor overall UX.
YMMV, perhaps these issues could have been worked around if I had had a more realistic deadline and stayed in touch with the community. The maintainer, Jens Alfke, is very active.
Thanks for the heads-up. We are very responsive to issues filed via the Github issue trackers, or brought up on the mailing list. My guess is this has been fixed since then (we've rewritten a lot of that code path), but if it isn't fixed, you can find a link to all of the relevant bug trackers on our Google Group splash page: https://groups.google.com/forum/#!forum/mobile-couchbase
Core Data now provides NSIncremental store, which lets you use any web API as your persistent data store. It's possible to roll your own alternative to iCloud in this way, if you choose.
Core Data + NSIncremental store with a web api backend does not solve the problem of offline sync. It is possible to roll your own, its hard to get it right. It is complicated by iOS restriction of not able to run your own daemon. But then if I have the resource to roll my own then I would not complain about Apple.
Dropbox does not come close as alternative. Dropbox as of now is file-based sync. Core Data iCloud sync is trying to sync an always opened SQLite3 database file, they do it through database log files. Possible alternatives are TouchDB, Simperium, Firebase but then you have to change your app from not using Core Data. Or rewrite your app to use file based storage as what 1password did. The key is offline sync. File based cloud sync is solved problem. So maybe the way to go is persistent store of client app has to use files.
Very true. The pain point seems to be largely the database merges; it's possible to roll your own db while using iCloud mainly as a key/value store for synching user preferences and status between devices. This is particularly useful if you support more than just apple devices...
Edit: regarding offline sync (which I didn't address) something like AFIncremental store is a good example of how a local persistent store can be maintained in tandem with the online store.
(But, as you say, non- trivial and hard to get right.)
We've tried to use NSIncrementalStore and we found it quite tough. Documentation is sparse. You buy yourself into the SQL-ish interface of NSFetchRequest throughout the app. There are fun bugs to discover like this one:
If I'm not mistaken, there seem to be better cross platform options in Parse and Dropbox. A case can be made for the "walled garden", iCloud doesn't seem to be one of them.
There is indeed an opportunity here for other cloud providers - but "better" is fuzzy. The nice thing about iCloud is that once you set it up on a device, there is no need to configure it or authenticate ever again.
It "just works" (if it actually worked).
With Dropbox, either you are storing everything for all of your users (and paying for it), or you integrate into the user's Dropbox account - which would necessitate a fancy ol' login/authentication system. Far from seamless, and relies on your users subscribing to something that is popular but far from ubiquitous.
So... either pay through the nose for all your users' storage, or force them to jump through some circus hoops on their own. Neither are great.
I'm not too familiar with Parse, but Dropbox is not solving the same problem as the feature of iCloud that developers are complaining about. Whole file syncing is comparatively easy from the server side. If there a conflict between the server-side file and the client-side file, the app has to work out how to handle it. Core Data (basically a local database) syncing tries to handle conflict resolution automatically, since it has knowledge of the data structures. That's tricky, and that's the part that doesn't work well.
I haven't seen many complaints about iCloud's file syncing features, which are what Dropbox is the direct competitor to.
While Dropbox is the most popular shared-drive mechanism out there, there are alternatives to it.
1) Google Drive - took my a long time before I even tried it, but it works about as flawless as dropbox for my needs. I actually like it a lot as I use Google Docs heavily, so it makes it easier to use those tools.
2) SkyDrive - offers the most base storage than the other. Has nice support for MS Office products.
3) AeroFS[1] - While it's still in beta, it looks promising (it has limited (android) to non-existant (iphone) support). [1] https://aerofs.com/
The great advantage to iCloud (whatever issues there are) is that it’s already there, integrated to the system. There’s almost nothing your users have to do in order to use it, quite unlike Parse and Dropbox. Granted, “almost everybody” has a Dropbox account and it’s free, but the nice thing about Apple devices is that they mostly work (or used to work?) even for people who know almost nothing about computing.
Tuesday's article on The Verge [1] was a lot clearer about the distinction between the various components that fall under the iCloud brand and what the problems are with the Core Data synchronization piece.
I feel like I'm the only person that has had zero trouble with iCloud. My phone backs up to it regularly (without my having to do anything whatsoever) and I've transferred my entire phone settings (and my wife's) more than once to brand new devices via iCloud.
Lots of things fall under the iCloud umbrella. A lot of them DO work. Just as you only think about the parts that work, ios developers think about the parts that do not work- so to you iCloud works and to them it doesn't even though you're talking about different things in the system.
I think the reason developers (myself included) try, and re-try to make it work (Coredata + iCloud), despite all the bad experiences and bad stories is that everybody would like badly that it worked. Because if it worked it would be awesome, when I test it and that it works it's really really nice, but I lost confidence in the overall reliability so in the end I never take the risk to ship it.
The document storage of iCloud is broken from a design viewpoint: I don't want to store/find things based on app, but rather by date/time created, content type, name, etc.
If I have many apps that do the same thing, I'd have to go through each one to remember where I saved something. No, just no.
Many points of the article are related to conflict resolution and CoreData sync, both of which features that are a) incredibly hard to get right and b) practically unsupported by Dropbox. Dropbox' method of conflict "resolution" is to keep to files around and its method of synching databases is to not do it.
There are other places where Dropbox has advantages over iCloud (like doucments being shared between applications, even though the dropbox api seem to move away from that and on to sandboxed application folders), but for the points made in the original article, Dropbox provides no help at all.
No, the article covers this. Dropbox is great, but it is based on document syncing. Apple has document syncing working just fine. Like database synching, Apple and Dropbox both have issues. To get a hint of the problem, you can read this post about why git+Dropbox is problematic: http://www.unityisplural.com/2012/02/git-dropbox-bad.html (I've used git+Dropbox and have gotten corruption issues in my git repo).
Further, by pushing an app relying on Dropbox, you're telling your users to sign up for an extra service to use your app. That's bad.
It's true that git+Dropbox isn't resilient in the face of multiple simultaneous writes, but I think the problem is overstated. You might see an inconsistent state, but you'll never get corrupted data because of it. You might corrupt the repository on Dropbox (has happened to me a couple of times), but this is trivially fixed by deleting and recreating it.
I'd never use it for a large team, but for a couple of collaborators, or syncing a repository between multiple computers for a single developer, it's a nice and easy solution.
No one really fully knows WTF is going with iCloud - documentation is basically nonexistent, support unavailable, implementation broken in trivial ways, and error messages inscrutable. I was at an Mac/iOS conference recently where the Core Data iCloud talk morphed from a standard talk into a "here's the mic, anybody have any idea what's going on with this thing" affair. Helpful, but indicative of the state of things.
IMO, from having studied this for many, many hours over many moons, is that what Apple is trying to do is fundamentally hard, if not entirely unfeasible. They're trying to replace a smart server with one that's dumb as a doorstop. More specifically, they're trying to emulate a CRUD web service with a file sync engine.
Conflict resolution is left up to individual clients, since the server doesn't do any "thinking". Likewise, there is no canonical, authoritative state of the store, since the server doesn't "think", only the clients do. Apple was hoping that by shoving a bunch of diffs of your database onto the server, that clients can reliably reconstruct a sane database by playing them back - except that multiple clients are updating the diffs simultaneously and there is no server-side conflict resolution.
Oh, and if stuff fails, there are no regular snapshot states to fall back to, because iCloud is a file store, not a database engine. Your whole store is now corrupt. Enjoy.
Oh, and there's no way to nuke the database and start over - there are metadata files that are undocumented or that we don't even have sandbox access to, that interfere with completely destroying the database.
At this point you might think "Fine! I'll go to my Settings -> iCloud -> Manage Storage menu and delete everything associated with that app and start it off on a clean slate!"
And then you find out that, even today, in iOS 6.1.3 land, there is a bug from iOS 5.0 where if you delete an app's iCloud bucket it can never be created by that app, on that account, ever again. I wish I were exaggerating. In perpetuity. Presumably some state gets fucked and now this app is forever more unable to store anything on iCloud.
To make even the most trivial experiment on the iCloud API involves creating a brand new app.
iCloud (in so far as it relates to Core Data and document storage) isn't just buggy, isn't just poorly implemented - it's non-functional.