If a recommender system for iTunes can predict the likelihood of me appreciating movies that contain violence against women, that information could be subpoenaed when I am falsely accused of having strangled my girlfriend.
I appreciate that Apple is trying to protect our privacy where they can. But if we want them to make predictions about or behavior, we have to be aware of the fact that we are necessarily giving up some privacy.
I understand that the database Apple wants to build does not contain accurate information about individual users. But if that database allows them to make predictions of our behavior, then there is a privacy issue. If the purpose is not prediction, then what is it?
So Apple can (for example) predict that listing to band A means you are likely to like band C, and then send a list of correlations to your device so the predictions can be made there by examining your library locally. A more probable use is analytics for marketing purposes. Another is selling just these correlations and other aggregate statistics to other parties; this is actually how Mint makes money.
And how is that different from my iTunes example?
That's what makes the data useful and that's what makes it a privacy issue at the same time.
How does sending the same list of conditional probabilities for liking pairs of bands to everyone's device and then having the device pick out the ones actually pertinent to your library compromise your privacy?
What I'm saying is that if Apple keeps data on its servers that is sufficient to predict some of my actions or likes with any accuracy greater than 50%, then that is a privacy concern.
But if you're saying that the data in Apple's database does not have any predictive power on its own, then I agree that it is not a privacy concern.
In that case, my device would have to download some of Apple's data and combine it with data that resides only on my device in order to make a prediction locally on my device.
If that's how it works then I have no concerns.
They even limit the number of samples they get from a specific person so they can't filter out the noise for that person and get their individual response.
But, keep in mind that Apple will have records of all your iTunes rentals and purchases at least for billing purposes. However, at least in the US there's a law about keeping that data private (because of Robert Bork).