I've been tracking the security/privacy problems with Instant Personalization for a while; my recent post might be relevant: http://33bits.org/2010/09/28/instant-personalization-privacy...
I'm also curious to see how things will turn out when a whole bunch of YC startups get Instant Personalization access, as YCRFS7 promises.
I consider opt-out a special case of bait-and-switch. You offered X and I signed up for X. Two months later you added Y and change my settings so now I am signed up for X + Y. Since X != X + Y, I consider X + Y to be a new product Z. I signed up for X, you switched me to Z. Bait-and-switch as far as I'm concerned. Doesn't matter to me if Z = 99% of X. Z != X and the switch happened without my prior consent.
I'm sure in the short-term numbers game, opt-out wins opt-in by far. Grab as many eyeballs as you can in the cheapest way possible. In the long term, people stop using your services. There's a reason I'll never use RealPlayer even though the company has completely changed since the early 2000s. I just don't trust them anymore. Same with Facebook. I just don't trust them with my data. Same with Google Buzz. Even though I'm comfortable with Google managing emails, I can no longer trust the Buzz team. Privacy loss doesn't have to directly happen to me in order for me to feel violated.
This is absolutely unreasonable to apply in general, because it would devastate the pace of change for web applications, and it is actively harmful to the users as well. You're going to be routinely asking them to make decisions which a) they do not want to make b) they have no information to make and c) they are incapable of making. It will merely confuse and annoy them, and the best possible resolution is that they do what they do any time people put up meaningless repetitive dialogs and click Next Next Next until you stop asking such stupid questions as whether to format C:\ or not.
("Attention, non-technical elementary school teacher in central Kansas. You signed up for Bingo Card Creator revision 1,550. Since you last logged in yesterday, we have made 5 changes to the service. These are summarized as: $INCOMPREHENSIBLE_COMMIT_NOTES. Do you want to consent to these changes, or should we keep a Rails application stuck to r1550 spinning for you until the end of time? Pretend this does not sound scary and that you understand that sentence. It is not scary and you don't understand that sentence, but you have no good way of knowing that.")
Personally, you would be the last person I would think of as pulling such shenanigans. Your work and words have shown that you care about your users more than making some extra cents in the short-term. I can't say I feel the same about others.
It seems like some site owners' dreams are to turn you into a bot for their own promotional purposes, or to just use your voice as their personal bullhorn.
I'll stick with registering to sites using a "plain" login (or OpenID, where available) -- at least that way, I have a bit more control over the way my online identity is used.
It should be a huge reason to log out of Facebook after every visit (or not use it at all), but 99% of users are never going to do that.
I had forgotten that I had turned off instant personalization (likely due to some TC article or something). So this works without even logging into the site? Jesus, it is worse than I thought!
Aside from the benefit of not staying logged in to your email/FB/Twitter when browsing random websites, this also lets you treat constantly open web-sites as legitimate apps in your switcher, frees up those permanent tabs on your browser, and provides for features like an unread count badge on the application icon when you switch apps.
 Obviously, this means that features that rely on you being constantly logged in to FB/Twitter (those Like and Tweet buttons) won't work.
The risk of unknowingly spamming people was too big for me and I just quit. I don't even care as much about the privacy issues as I care about these services doing things and post stuff in my name without any way to stop it or even just indication that they are doing it
As long as I'm given more relevant content, feel free to use my publicly posted information.
Doesn't that way of putting it, though, make it sound a bit like you are actually making the decision to opt in?
Why? Facebook provides no value to me; if Facebook disappeared from the face of the earth tomorrow, my life would change in absolutely no way.
No need to embrace it.