
IOS 5's "Cleaning" Behavior  - JeffDClark
http://www.marco.org/2011/10/13/ios5-caches-cleaning
======
lukeredpath
Lots of people are focussing on Marco's particular use case in the comments,
and I think it's a valid one, but this extends beyond simple documents.

There is a category of data that is aimed at offline use. Streaming apps like
Spotify, that let you download playlists for offline use. GPS apps that
download hundreds of MB of map data. You get the idea.

On one hand, this data is a form of cache. The data _is_ always available
elsewhere (on the content provider servers) and it can be restored if
necessary in a worst case scenario. But the key word here is "offline". This
is the kind of data that, by definition depends on being around if the user is
offline and therefore cannot be easily restored on demand, when the user needs
it.

Obviously, having all of this stuff backed up to iCloud and using up GBs of
people's capacity is not feasible or even logical. So this kind of data does
not belong anywhere that iCloud will back up. But it must be stored somewhere
that is safe from being purged.

Yes, a users GPS maps can be restored eventually but that doesn't help them
when they are stranded in the middle of nowhere with nothing but a weak GPRS
signal and all of their maps gone.

Apple have made an almighty cockup in overlooking the "offline data" use case.

In Marco's case, I'd agree that the articles represent user data that should
be stored somewhere like Application Support, which will be backed by iCloud
but I think that's probably fine in this case.

~~~
jeromeparadis
I entirely agree. Apple forgot an important use case? I hate it when software
is built to think in place of the user. There are two culprits here:

* the lack of persistent, non backable offline storage * the addition of a auto-cleaning procedure

The auto-cleanup without a storage alternative is a blunder. iOS 5 decides for
the user what should be deleted to free up space. An alternative would be to
prompt what application data you would like to remove instead of a blind
decision. Otherwise, that's the kind of programming that will one day lead
robots to cleanup the human race! ;)

Of course, having permanent local storage that's not backed up would be a more
user-friendly solution. However, it would probably lead people to run out of
space when developers begin abusing it like some did with temp or cache
storage.

It seems like a trend at Apple these days. With Lion, the OS now decides when
an application should quit when unused.

------
pohl
I'm sympathetic to Marco's blog post here, but I wonder if he's failing to
interpret the two paragraphs from the documentation within the context of
Instapaper's purpose.

With respect to point #1, and in the context of what Instapaper is for, the
user is intending to read an article offline, and the local copy of the
article was generated by the user's intent (and can, therefore, be considered
user-generated even though the user is not the author of the article.)

With respect to point #2, the articles fail the "can be downloaded again" test
in light of the app's purpose of making the articles available to the user
when the network is not available. When the network is unavailable, the
articles cannot be downloaded again. _Edit: JeffDClark makes another excellent
point here that the article may have originally been behind a paywall and
therefore cannot be guaranteed to be re-downloadable. The same is true if the
author removed the original article from their webserver._

Ergo, put them in the Documents folder. Whether this would satisfy the app
reviewer is another question, but it's worth a shot to carefully explain to
them how your app is not violating the letter of the law.

~~~
rcthompson
But it doesn't make sense to back this data up to iCloud, which is what
happens to the Documents folder.

~~~
pohl
Really? If I generate an offline copy of an article on my phone, wouldn't I
want to have access to it on my iPad? Why wouldn't I want the truth to be in
the cloud?

~~~
Mavrik
Because there's no point in burning your cloud storage, Apple bandwidth and
prolong backup times when those articles can be easly re-downloaded from
Instapapers servers on iPad.

Same result, without long backup times, Apple server bandwidth usage and your
iCloud storage usage.

~~~
mynegation
Why there is no point in burning Apple's bandwidth, but there is a point in
burning Instapaper servers' bandwidth?

I don't care how Apple (or Marco) solves the problem, but I sure do not want
to lose Instapaper articles at random.

~~~
modeless
It's not just Apple's bandwidth; it's users' iCloud storage, which costs users
money.

~~~
dagw
Also don't forget that many people in world still (for whatever reason) don't
have unlimited data plans. So if everything that get's downloaded also gets
uploaded back to iCloud, then you've effectively halved their bandwidth quota.

~~~
pohl
I think iCloud syncing only happens over wifi.

------
smokey_the_bear
I write several offline mapping apps, and this is totally throwing us for a
loop. We're recommending our power users not upgrade to iOS 5. Users download
gigabytes of maps to their cache directory, they don't want to eat their
iCloud allotment with that, or their slow their iTunes sync. But they also
don't want to have to download those maps again, or find themselves in the
middle of the woods without the maps they downloaded.

~~~
derefr
Instead of just saying "don't install iOS5", can't you continue to store files
in their Documents, but tell them to flip off the sync switch for your app?
<http://i.imgur.com/acOje.png>

~~~
smokey_the_bear
We could, but it would involve pushing an update to the app. The 'don't
install' is a stopgap for people who may be counting on using their maps this
weekend. Additionally, people collect geo information with our app, tracks and
waypoints that they like to backup.

~~~
derefr
> Additionally, people collect geo information with our app, tracks and
> waypoints that they like to backup.

That could still be persisted separately if you stuff all that into iCloud's
KV store instead of relying on file backup. (Just trying to brainstorm
technical solutions; not suggesting they're at all optimal.)

------
qjz
_Apple - iCloud - Your content. On all your devices._

That is the title of Apple's main iCloud page at
<http://www.apple.com/icloud/>.

 _iCloud is seamlessly integrated into your apps, so you can access your
content on all your devices._

This is Apple's definition for the iCloud service. It doesn't matter what the
data is, it's _your_ data and Apple is promising to sync it between your
devices, to preserve your experience.

In the case of Instapaper, the solution is obvious: Put the files in
Documents. That is Instapaper's content and part of the experience that users
want synced between devices.

If Apple penalizes developers and undermines the promise it is making to users
because it decides to be miserly about bandwidth, then it has to admit it
launched iCloud before it was ready.

~~~
wlesieutre
But it's not clear that Instapaper is _allowed_ to put the files in Documents.
According to the Data Storage Guidelines cited in the article:

 _1\. Only documents and other data that is user-generated, or that cannot
otherwise be recreated by your application, should be stored in the
<Application_Home>/Documents directory and will be automatically backed up by
iCloud._

 _2\. Data that can be downloaded again or regenerated should be stored in the
<Application_Home>/Library/Caches directory. Examples of files you should put
in the Caches directory include database cache files and downloadable content,
such as that used by magazine, newspaper, and map applications._

I don't know if this is something enforced in the approval process, but it
seems like Apple's intention is for redownloadable content to all be stored in
temporary, wipeable locations.

~~~
TeMPOraL
Still, when user saves a webpage to Instapaper he/she probably wants to see it
as it was when it was saved, and not magically updating itself afterwards. So
I think that 'exactly this article at exactly this point in time' should
qualify as 'content that cannot be recreated'.

------
ender7
Apple's in a tight spot here - either they leave things as they are and piss
off developers (and, by fiat, piss off users), or they have to start forcing
users to more proactively manage their remaining space.

One can probably easily imagine an interface for showing the user how much
data a particular app is using and allow them to nuke the temporary stuff. It
might even look beautiful. It might even be fun to use. But it's going to
introduce a lot of hand-wringing and micro-managing and lots of mental
overhead that Apple really, _really_ wants to avoid.

~~~
sudoman
This general problem is a symptom of a major issue regarding the use of
software that does not respect the freedom of its users. Every non-apple
developer and user is required to bend to the will of the programmers who
develop the proprietary iOS. Any negative choices or limitations imposed by
Apple Inc. are virtually uncircumventable.

On the other hand, if users and developers in a community are free to view,
modify and share the source code of the operating system and programs they are
using, then that creates a dynamic where software cannot remain defective by
design, something which cannot be said about Apple's hardware or software.

If we were talking about free (as in speech) software, users would have
control of their data, and their choice of how to manage it; not some distant
programmers working for a corporation's bottom line.

~~~
muuh-gnu
> If we were talking about free software, users would have control of their
> data; not some distant programmers

Example Gnome3, which is free software, even a officially sanctioned GNU
project. Overnight, a few key people decide to abandon the traditional desktop
metaphor, against the will of a majority of their users and to the detriment
of the whole linux desktop. The users hated it and are still hating it, but
nobody has the ressources to modify it. Nobody has even the ressources to keep
Gnome2 alive, let alone fork Gnome3.

Yes, in theory users could make only those changes they want, but only in
theory. In practice, if a significantly influential player says that Gnome2 is
gone, Gnome2 is gone. Even if iOS was free software, if Apple decided to make
a change, they would be still big enough to force a majority of users to
accept their way.

~~~
danssig
Wonderful comment. This is one reason I always laughed at the argument "yea,
but... we have all the source code!". Great, so the solution is we pay to
maintain it? How is this free again?

~~~
seabee
The unsaid thing about freedoms is you cannot exercise them unless you have
the means to do so. If you don't have that, the only value freedoms afford you
is aspirational.

Given that the potential to do something is worth a whole lot less than the
activity itself, it's easy to see why FOSS is in is current state and will
remain that way for a long time.

~~~
gbog
Then you would drop freedom of speech just because it is useless unless you
own CNN?

~~~
seabee
Nobody is suggesting that. In any case it's a false comparison, since anybody
has the ability to fully exercise their freedom of speech. To exercise your
open-source freedoms fully require skills, time, and/or money (to pay others
to do it for you).

~~~
gbog
If the overall noise level is too high and you don't own a microphone, your
freedom of speech can't be used effectively, you talk any nobody hears you. It
is not a reason to drop freedom of speech, right? It's just the same with free
software, the mere fact that you are allowed to read, change and distribute
source code IS the key. Doing so is hard but not a reason to dismiss this
freedom as irrelevant.

------
zbowling
Add .nosync to the file/folder name path and keep it in the home or documents
directory. Problem solved.

edit: I'm shocked this thread is so long and no body mentioned this. It's been
on the apple developer forums for months as a solution.

~~~
smokey_the_bear
Do you have any documentation about this? I see a few scant references in the
developer forum and nothing in the docs.

~~~
zbowling
[http://developer.apple.com/library/ios/#releasenotes/DataMan...](http://developer.apple.com/library/ios/#releasenotes/DataManagement/RN-
iCloudCoreData/_index.html#//apple_ref/doc/uid/TP40011298)

right there.

~~~
stevemoy
From what I can tell the use case for .nosync there is to store a large
dataset in a SQLite store in such a way that individual transactions are
replicated to iCloud but the entire SQLite store itself is not transferred
each time something changes.

The documentation seems to imply that .nosync will prevent the whole store
from being uploaded, but (1) that does not seem consistent with what others
are reporting, and (2) I cannot find any other supporting information on
.nosync in either Apple documentation or dev forums - that doc and this thread
are literally the only places I've seen it referenced.

The other underlying issue is whether Apple would be cool with storing
persistent application data in Documents provided that it is not synced to
iCloud. The use case that Marco and others have described does not really
match what's described in that document.

edit: Marcos -> Marco

~~~
zbowling
Can also store in the apps home directory (which is right above the apps
Documents directory).

They spoke about .nosync at this year's WWDC a few times. Dev forums are a
little nuts since they opened the iOS Beta 5 forums to all the non NDA
members.

------
sjwright
There are a number of possible scenarios for file storage, the problem is a
lack of clarity or documentation about the properties of the various locations
as they stand now. As a developer, I could imagine desiring the following
choices:

1\. Temp: No backup, cleared regularly

2\. Cache: No backup, cleared when space is tight

3\. Local: Local backup only, never cleared

4\. Documents: Local/cloud backup, never cleared

5\. Cloud: Cloud backup, cleared when space is tight

The problem seems to be that #3 doesn't exist. Yet you'd think it would be a
common requirement for stuff like in-app purchases of large and essential
content packs, for example, turn-by-turn navigation maps.

I'd hate to be on holiday and have a 10 megabyte podcast download
automatically trigger the erasure of 1000 megabytes of navigation data.

~~~
mw1234
You are missing "No backup, never cleared" which is what the old "2. Cache"
scenario used to be.

~~~
sjwright
I disagree. If it's not worth backing up (locally), it doesn't deserve
protected status on the phone.

Or to put it another way, if it's important enough to retain when storage
space is low, it's important enough to retain for when you have to recover
from a backup.

~~~
mw1234
I think that is a decision best left to the developer or user. Take an app
that syncs your photos from a site you may already use, like Picasa, Flickr or
SmugMug. You want to have these available on your iPad or iPhone instantly,
even when you are without internet service. But you don't need them backed up
to your computer every time you sync, causing the backup times to skyrocket,
and if you lose them on a restore, it's a minor inconvenience, but not a huge
deal. Losing them unexpectedly when you don't have the time or ability to re-
downlaod them is a much more major problem for some people.

------
smackfu
I don't understand Apple's logic here. You can't reconcile the idea of
"cleaning because you can redownload" with "available offline". As soon as you
clean up, you are going to break offline uses.

I would guess the idea is to help enable these 500 MB per issue magazine
downloads. You download a new issue, you nuke some old issue, no one cares. As
long as that wasn't an issue you cared about.

~~~
voxmatt
I had the exact same question and your idea as to the answer is interesting.
Perhaps this new behavior is in response to Newsstand's ability to
automatically download new issues--users may allow those to stack up without
any thought and run out of space. The question then becomes: why not just
enforce this behavior for Newsstand?

~~~
morrow
Somewhat of a conspiracy theory - but if the choice for developers is risk
losing losing local data or back data up to the cloud unnecessarily, I think
they'll choose to store it to the cloud, as they will be the ones blamed when
their app "deletes data randomly".

The more storage space taken up by local files being backed up to iCloud, the
more people will end up going over their 5GB capacity, and will have to
upgrade to the higher capacity plans to cover it. I'm not sure I actually
believe this right now, but if they refuse to fix the problem, then I might...

------
JeffDClark
The whole idea of an app like Instapaper (or any of the other examples
presented) is that the "stuff" that is saved for later is all user-generated
content. Some articles may even vanish (different location, move behind a
paywall, deleted, etc...). In this case those articles would become
inaccessible when the OS deletes the cache.

It seems that the argument that only the list of metadata is user-generated
can apply to any type of media (music, movies, etc...). Technically speaking
all of the music on my phone could be downloaded on demand. Of course this
requires an always connected, fat, and cheap network connection. Which is
pretty much the opposite of what most folks have.

This also breaks the user's expectations. I was annoyed/surprised when I
upgraded to iOS 5 and Instapaper had to re-download everything.

~~~
masklinn
> Some articles may even vanish (different location, move behind a paywall,
> deleted, etc...). In this case those articles would become inaccessible when
> the OS deletes the cache.

As far as I know, all scraped articles are stored in Instapaper's server and
downloaded from there, you don't get them directly from the source page.

~~~
JeffDClark
I have had articles in my list that have become unavailable. No amount of
trying to re-download them works. What you say makes sense but I am unfamiliar
with the exact way that it works.

------
jackvalentine
The first time that one of my several hundred megabyte foreign language
dictionary files isn't available and needs to be re-downloaded when I need it
in say, a meeting will trigger a severe re-evaluation of my use of the phone.

~~~
jackvalentine
I just emailed the company that makes the dictionary app in question (pleco
chinese dictionary, if anyone cares) and they confirmed this issue will affect
them.

The ball is in Apple's court.

------
rubergly
I'm definitely at risk of running into this predicament as a user. I'm more
worried about the hundreds of megabytes of podcasts I download when WiFi is
available for use throughout the day when WiFi isn't available.

To avoid this issue and enjoy the benefits of iOS 5, I'm going to have to
clear out a bunch of apps, music, and photos and ensure that I always have a
sufficient amount of "buffer" space so the cleaning is never triggered. I
cannot be alone in this, and the fact that Apple is making this kind of
thinking necessary for end users is kind of ridiculous. I really can't see
this behavior lasting for very long, and I'm sure Apple will address it soon;
this is the antithesis to the traditional iPhone experience. The only scenario
where I could see this being purposeful is if Apple is really trying to hurt
offline apps to increase data usage and appease carriers (maybe for pissing
them off with iMessage?).

~~~
phillmv
I would wait until more reports come in.

Alternatively, just turn off iCloud back ups.

~~~
rubergly
Will turning off iCloud back ups turn off the auto-cleaning behavior?

~~~
cschep
It won't matter if the files are saved in the "Documents" folder as that isn't
cleaned.

~~~
rubergly
Well yeah, but the point is developers will start moving it out of the
Documents folder.

------
euroclydon
Seem like the Dropbox app will have this quandary but on an even larger scale.

------
wrs
It seems fair to say that Instapaper's version of an article can't be
"redownloaded" for various reasons (offline, paywall, article removed, etc.)
so it would be OK to put it in the Documents folder.

The argument against that is that you're now syncing that article with iCloud
in addition to Instapaper.

But I wonder whether the correct answer is instead to eliminate Instapaper's
sync feature, and just let iCloud do it. Once you have system-level cloud
sync, don't you want to let Apple do the work? Sync is hard, and it isn't
really the core value of Instapaper.

Edit: I was wondering about iCloud only for iOS 5/MacOS 10.7.2 devices with
iCloud accounts. But that story does fall apart for people with mixed devices.
So never mind.

~~~
biturd
I see it being a long while before everyone has iCloud up and running. We
installed it last night on a relatively new MBP, to learn it doesn't work
without Lion, so now we are upgrading to Lion. It will only work on certain
phones, I believe 3GS and above.

Maybe Apple will release iCloud for Snow Leopard, but as it is now, instapaper
works all the way down to the terrible version 2 iPhone I have had to resort
to using in order to not sign into a new contract waiting on the release of
the 4s.

Instapaper would hurt too many current users who are just fine with how things
work without the cloud.

------
psychotik
Isn't NSApplicationSupportDirectory for stuff that isn't "Documents" and yet
needs to be managed as app state (not in 'Caches')? Why not just use that
instead? That's what my app does and it seems to be OK with iOS 5.

~~~
mrgrieves
I do the same thing. Apple is unclear about this, however. ApplicationSupport
is at /Library/Application Support, and /Library definitely does get backed up
__to iTunes __.

[http://developer.apple.com/library/ios/#DOCUMENTATION/FileMa...](http://developer.apple.com/library/ios/#DOCUMENTATION/FileManagement/Conceptual/FileSystemProgrammingGUide/FileSystemOverview/FileSystemOverview.html)

That said, I think we're in the clear with respect to iCloud backups:

"Devices with an active iCloud account have their app data backed up to iCloud
at appropriate times. And for devices that are plugged into a computer, iTunes
performs an incremental backup of the app’s data files."

It's ambiguous. But "app data" sounds like documents, whereas "app's data
files" sounds like big data files in /Library to me.

------
hernan7
I wouldn't say that the articles' metadata (URL, title, date of download,
maybe thumbnail of the 1st page) is downloaded content. It's clearly user-
generated, and should be in the "home" of the app IMHO.

The articles themselves, yes, send them to the cache. If the user needs to
reclaim the storage used up by the articles, let the OS delete them. Then,
when the user needs to read the article again, it will take some time to
download. But don't get into an "all articles gone" situation. Just my 2
cents.

~~~
luchak
What problem are you trying to fix? If I'm offline, what good is being able to
see my article list if I can't read the articles? If I'm online, why do I feel
any less irritated when I try to go back to an article I was just reading only
to find that I need to wait for it to download again?

~~~
adgar
The problem fixed is that the user has an indication as to what happened to
all of their saved articles. That's a very big difference. It's not a full
solution, as the parent noted, but the UX is far superior.

------
theatrus2
Has anyone figured out where the high water mark is? When will the cleaning
behavior kick in?

This is an interesting twist especially with the 16GB devices which tend to
actually be above 50%.

------
j_baker
What about apps like spotify and rdio? Where do they store music if those two
directories are constantly cleaned?

~~~
jmcnevin
For the record, I almost immediately filled up my allotted iCloud backup space
because of rdio, because I have 4.6 gigs of music synced for offline use, and
it wanted to back that up.

Solution: I disabled backups of rdio data.

------
sidwyn
I develop Definition (<http://definitionapp.com>) and I store the database in
the Caches directory as well, after receiving the email from Apple to move.
This is bad news for me, the entire offline dictionary could be wiped out.

------
Scorponok
Maybe the solution is to make it a choice for the user? An option to "clean up
documents when space is low on the device" in the instapaper options. If
checked, stuff gets stored in cache. If not, in documents.

That way, the default behavior is that "download something = want to keep it
on device", but users can do the other one too if they want. I don't think the
option is particularly useful, but it might make the app reviewer happy?

~~~
high5ths
I would bet Apple's never going to make this sort of a choice obvious to the
user -- it goes against the "just working" (whether or not it works the way
you want) they're currently pushing. It sure does worry me to read all these
comments though; it seems like you'll never be able to count on an app having
the data it needs to run, without re-downloading. As somebody who spends half
his day in an airplane or a subway, plus plenty of time in expensive places
(like overseas), that's terrifying.

~~~
lreeves
Actually a couple video players right now do make this choice available to the
user - OPlayer and AVPlayer. They both say something to the effect of "Backup
media" in their settings. This simply toggles which location the media is
stored in (AVPlayer even seems to move the media on the fly).

This is literally exactly how I would implement the choice if I were Marco.

~~~
WiseWeasel
The problem is if the user chooses not to backup the data, it'll get wiped if
she runs out of storage space. What would be nice would be to not sync to
iCloud AND not get cleaned out.

------
jmcnevin
Is it possible that a user could use Instapaper to save a document that they
wouldn't be able to download later, even if they had access to an internet
connection? If that's possible, I think Instapaper would have every right to
store things in the Documents folder, since you're asking it to create
something more akin to an archive than a temporary cache of data.

~~~
baddox
That seems to be the only way for Instapaper to give their users a reasonable
experience, but it's still a bad solution. It doesn't make much sense for the
Instapaper app to backup its users' cached articles to iCloud, when the whole
point of the app is to backup the articles on the iPhone. If you have access
to iCloud, you have access to the original article, so it's a waste of
bandwidth. Not to mention that Instapaper risks getting hounded by Apple for
the iCloud usage.

~~~
lukeredpath
I honestly don't think there is an issue with allowing Instapaper content to
get backed up to iCloud here. The analogy of a user created archive is a good
one.

If people don't want the benefit of having their Instapaper content transfers
across their devices automatically (and I do think this is a good feature)
then they are free to disable iCloud backup for Instapaper in Settings.

~~~
WiseWeasel
That's not entirely true. You can disable iCloud syncing for individual Apple
apps, but 3rd party apps are all-or-nothing. You can't disable iCloud syncing
for a single 3rd party app.

~~~
Lazlo_Nibble
You can't? What's this screenshot showing, then?

<http://i.imgur.com/acOje.png>

(That's a serious question, BTW. I would check it myself but I can't upgrade
to iOS5 until I figure out how to get Windows DEP to stop killing iTunes 10.5
every time I launch it.)

~~~
WiseWeasel
I didn't find that setting screen. Definitely didn't see that in
Settings/iCloud. I wonder how you get to that... I only had a chance to
briefly look at my roommate's 3GS on iOS 5; won't be on iOS 5 myself until my
4S arrives later today.

------
MatthewPhillips
I think the problem is that Marco sees Instapaper users as his customers. I
sympathize with this, but I think iOS is an assertion by Apple that App Store
customers are Apple customers. App Store Developers are providing a service to
Apple's customers. But everything that happens must point back to Apple's
servers, not their own.

In the same way Apple doesn't want every developer operating their own
independent payment system, they also don't want developers operating their
own cloud storage services. If Apple holds the data they can guarantee its
security. They likely see these problems as temporary, until developers and
customers learn to adjust to the fact that iOS data is controlled by the
iCloud service.

------
xpaulbettsx
It sounds like what Marco is looking for is something equivalent to Windows's
AppData\Local, machine local, doesn't get nuked, but doesn't sync over roaming
profiles either

------
nrser
manage it yourself: put things that need to be persistent in Documents. put
the rest in Cache. move 'em as needed. do it automatically by download and
access dates and/or provide an interface for people to manage it.

your app absolutely needs tons and tons of data to function? doesn't seem like
your day. it's their device, their cloud, their decision. Apple doesn't give a
shit about your day; they're going cloud. they may be wrong, but i'd guess
they're going to have to find that out for themselves.

i'd assume they acknowledge this may kill some apps. i don't think they ever
promised anyone a business; on the contrary, they seem to remind developers
that they are there at their good grace all the time. as someone that built
Facebook apps since '07, trust me, i know what this is like. start coding and
start calling. best of luck.

------
tjmc
DHH declared the solution in 2007 [1] - apparently nobody needs offline
applications!

[1] [http://37signals.com/svn/posts/347-youre-not-on-a-fucking-
pl...](http://37signals.com/svn/posts/347-youre-not-on-a-fucking-plane-and-if-
you-are-it-doesnt-matter)

~~~
shinratdr
I'm very rarely on a plane. However I take the subway every day, and I'm
extremely thankful that Tweetbot, Reeder and Newsstand cache everything so
that I have something to read. I also don't consider the subway a niche or
isolated pocket.

I just don't buy the lame justification about not always being connected. If
you don't want to be connected, turn off your damn phone. Don't insist that
everyone else shouldn't be able to enjoy their time the way they want to
because you associate your mobile device with stress.

------
charlieok
I must be missing something. What is the problem with backing up pages stored
in InstaPaper to iCloud? That's exactly the behavior I would want if I were
using InstaPaper.

~~~
hboon
1\. The user can re-download from Instapaper's servers.

2\. It goes against the spirit of Apple's policy here, which is not to backup
to iCloud what ca be re-downloaded (see 1)

3\. The user might have lots of content stored and again due to 1, those don't
need to be backed up to iCloud. And really Instapaper actually has a small
version of the problem. What is critical is applications that has huge amounts
of data like offline maps and wikipedia. Those can be re-downloaded from their
servers yet most definitely should not be backed up to iCloud due to time and
cost.

------
droithomme
That's pretty interesting. So, the three non-backed locations are tmp, Caches
and the Application Bundle. tmp and Caches are now swept. So, if you store
your real cache stuff in the Application Bundle, it won't be backed up or
auto-deleted. But maybe it gets trashed when you update the app.

I wonder if OS X, in line with its trend to be more like iOS, is going to
start automatically clearing the ~/Library/Caches directory as well.

~~~
hamrickdavid
Unfortunately you can't write data to your application bundle. That folder is
read-only.

------
zamfi
_There needs to be a file storage location that behaves the way Caches did
before iOS 5_

I'm confused. If the goal is for documents to be available in the near future
in offline form, why not keep documents in /Documents until the user has read
them (or some sane amount of time has passed), and then move them to /Caches
or some temporary storage?

I've never used Instapaper, so perhaps documents are only stored until read
anyway?

------
mw1234
This is also particularly problematic for my own apps, which are offline photo
browsers that sync your collection of photos. Keeping GBs of data in the
Caches folder was the only way to have iPhone backups occur reasonably fast.
Sure, they can be re-synced, but that will be a very time-consuming process
for thousands of photos.

------
wmf
Clearly, apps that cache data need to be modified to show the difference
between having no data and having no data cached. IMAP clients have dealt with
this, for example; they show a message like "the contents of this folder are
not available offline".

Perhaps Apple could have made cache cleaning opt-in on a per-app basis until
iOS 6, though.

~~~
smackfu
The main issue is that iOS is deciding that you don't need something offline
without asking you. I don't want my cached email to turn out to have been
deleted because I downloaded something else entirely.

Many people would prefer the old method, where it would say you were out of
space, and you would have to go free up some space.

~~~
politician
That's right; the solution is to make it easier to see which apps are using
how much space and then provide a "clear" button.

My Nintendo Wii uses a "blocks" concept for this. It works fine, Apple should
just copy it.

------
staufman
Could you use NSUserDefaults? I don't think there is a limit on the data
stored there.

~~~
apike
NSUserDefaults gets very slow when you put megabytes of data in it.

------
cschep
So, people (developers) are going to have to put their stuff in Documents, and
instruct users to disable the iCloud sync for their particular app, unless the
user really wants to have it eat into their iCloud storage.

Would that work?

~~~
marchdown
The whole quagmire is bigger than just flushing Cache. Apple prohibits
developers to store stuff that can be regenerated or re-downloaded. So,
arguably, you can't just switch to using Documents for downloadable data.

------
tvaughan
Or worse: [http://blog.foreflight.com/2011/10/14/flying-with-
foreflight...](http://blog.foreflight.com/2011/10/14/flying-with-foreflight-
and-ios-5/)

------
peterclary
What about offline web apps which are saved onto the home screen? Are they
also "cleaned"? If we can't depend upon an offline web app to be there when
offline then that would fundamentally defeat the purpose.

Of course, one could argue that the cleaning behaviour fundamentally defeats
the purpose of a lot of apps, as already covered above (Instapaper, Offline
Maps, etc.).

My iMac is packed up for building work, so I can't upgrade my iPad and check
this out for myself. Sorry.

------
minga
"Prior to iOS 5, the system never deleted the contents of Caches and tmp, so
they were safe places for apps to put data that should always be available"

Not exactly.

------
falling
He says he knew about this behavior, and he also explicitly said (on Twitter,
can't be bothered looking for it) that he was not going to report bugs to
Apple during the beta.

I guess Marco just prefers venting after the fact.

~~~
russellallen
Maybe if you are going to slag someone off you should be bothered looking for
the tweet you base your slander on...

~~~
falling
Fair enough. I tried now but that was months ago and Twitter search is
unhelpful.

However, he's being asked by multiple people right now for the radar# and he's
ignoring the questions. He just said he's known it for two weeks, which might
have been too late for Apple to fix it anyway.

~~~
dextorious
Yeah, one thing I don't understand is Marco had access to dev builds of iOS
for months and iCloud dev access I presume. Why he only found about it now
instead of doing something about it all this time?

~~~
allwein
Probably because, like most developers, he has separate development devices
that he only uses for development. So he probably never ran into storage
constraint limits while just testing Instapaper.

As an example, I have a 32GB iPod Touch that I've been using for all of the
iOS5 testing for my applications and I'm only using around 600MB total. I'd
never run into this issue either unless I loaded this up with tons of other
apps and movies and music.

~~~
dolbz
That's incorrect, he stated on his podcast that he's used iOS5 on his main
device since WWDC.

Whether he hits the limit regularly or not is another question...

------
martinbech
I dont get it.. he get furios because he saves files he needs in a folder
called /cache , and /cache gets purged when the system runs low on space....
What did you think cache meant?

Why dont you just update your app?

