Core Data is just not a good technology. People on the other side of this debate will say that there's an appropriate time for everything, but nothing makes sense for everything, so you have to choose based on your needs. This is half true. It misses the fact that some things are just plain bad and are never better than the alternatives in any situation, and IMO Core Data is in this category.
There are several fundamental problems with it.
First, the API is awful. If you want decent model objects in memory you either have to do a bunch of manual work, or use a third-party tool like mogenerator. Even then, the result is a massive soup of mutable objects with no intelligence. The structure of the API encourages passing the entire context around everywhere, which basically turns all of these objects into global variables. Look no further than popular Core Data wrapper APIs to see how bad this can get. For example, ObjectiveRecord adds class methods for looking up objects using a predicate. Modularity? Separation of concerns? Perish the thought.
Second, it ties your on-disk representation to your in-memory representation way too strongly, and this makes it more difficult to choose appropriate structures for either one, and more difficult to make changes to either one.
Third, it locks you into the technology something fierce. Core Data is so different from everything else that once you build your model layer on it, you're just about stuck there forever. You can move away, but it's a ton of work that reaches into every corner of your app. Other solutions, like serializing to a property list or SQLite, simply require a translation layer that you can switch out more or less at will.
Fourth, it's unbelievably slow. Literally unbelievable, as in I tell people about it and they don't believe me. It's too slow for a large number of records. It's fast enough for a very small number of records, but at that level it provides no advantage over using property lists and just loading everything into memory. At the large scale, you don't want to load everything into memory so you need a better scheme, but then you need something like SQLite so you can avoid the slowness and gain more control. Core Data is really only workable when your quantity of data is in a happy medium, which exists roughly within the range of 1097 to 1143 records. (Warning: previous numbers made up.) Since you can almost never guarantee that your data will live in that happy medium and not exceed it, Core Data is a bad choice.
I understand why its proponents advocate for it. It's shiny. Apple pushes it hard, and people tend to like what Apple tells them to like. It gives the illusion of making things easy. It does make certain simple things easy. The problem is that making easy things easier is not a virtue. Making difficult things easier is what counts, and Core Data fails at that.
For a lot of applications, a relational model will work just fine. And, the optimizations that Apple provides for SQLite are probably going to be better than the average developer would achieve by writing their own SQLite transaction layer. (A little confused here on your point about needing SQLite at scale for performance -- it's the default persistent store on iOS).
I'm not sure I agree that an app needs to be wed to Core Data -- if the transactions with the persistent store are hidden from the model layer, then they should be swappable for something else.
> The structure of the API encourages passing the entire context around everywhere, which basically turns all of these objects into global variables.
I don't see it that way. In many cases, you don't need to pass around a context, you can directly pass the objects that you need to work on. And even when you do pass the context, its not as if you go through its -registeredObjects. You would usually get your objects using a fetch request -- how is it different from fetching your objects directly using SELECT over a db connection?
> For example, ObjectiveRecord adds class methods for looking up objects using a predicate. Modularity? Separation of concerns? Perish the thought.
I really don't see how this is bad for modularity or separation of concerns? Can you please elaborate?
> At the large scale, you don't want to load everything into memory so you need a better scheme, but then you need something like SQLite so you can avoid the slowness and gain more control.
But Core Data does support SQLite as a backend, and it allows you to easily load only a subset of your data. Just create a fetch request with the appropriate predicate and you can work with just the objects you need.
One of the things that is definitely slow though is if you want to mass update or delete a significant number of your objects. In that case you would have to load all the objects in memory and update or delete them one by one, which can be a pain if you need to do this a lot (whereas with SQLite a simple UPDATE or DELETE statement would have sufficed).
Another reason for slowness historically has been that if you use two different contexts (say for multi-threading reasons), then in order to transfer updated information from one context to another you had to first save the information to disk, and only then could you fetch the updated info on the other context. The alternative to this is to lock the context and objects explicitly which has its own problems. However, with the introduction of parent contexts and new concurrency types in iOS 5, it is not a problem anymore. If a context has its parent context set, then when it saves its changes, they won't got to disk -- the changes are only registered in the parent context.
Core Data is not perfect at all, but I don't think it is completely unusable either.
One thing I’ve been doing lately to mitigate #2 and #3 is to use the repository pattern. Models become POCOs and the repositories worry about translating to and from SQLite/Core Data/etc. Is it perfect? Nope. But it does provide a lot of flexibility, and I sleep better at night knowing that I can swap out the data store much more easily should I need to.
It's nice working this way, interestingly, sans-frameworks for once. Code is a lot more maintainable too.