Zygna (of FarmVille fame) has one of the most sophisticated testing and metrics operations on the Internet today. From externally observable evidence they're probably better at it than WoW and WoW is large enough to be its own national economy. (Zynga is a lot like IMVU or for that matter Toyota: I don't care if you have no interest in their product, their processes are worthy of study.)
They aren't just throwing dung at the wall and seeing what sticks: they're very, very good at what they do. For every FarmVille there are tens of thousands of games which you have never heard of: their viral growth coefficient was .95 instead of 1.05, kaboom, that is all you need to fail. The platform is getting harder to develop for, as early entrants used techniques which were what SEOs might call "a wee bit grey" to gain traction, and these are either impossible now or will result in a new entrant getting hit with the banhammer. (For example, early social games maximized their viral coefficient by adopting a spam-early spam-often policy to user notifications, which ticked off a lot of Facebook users, causing Facebook to take a number of countermeasures. Ask somebody who builds for it to describe the specifics, but there are now throttles and some sort of system where you earn the right to use the viral channels.)
"Ask somebody who builds for it to describe the specifics, but there are now throttles and some sort of system where you earn the right to use the viral channels."
All of that is going away in five days. No more requests, no more notifications, etc.
Memo to self: add "Do not build for platforms which could pull the rug from under you" to list of tips for founders with day jobs. That sort of thing would crunch my business like a tin can if it happened to me in the wrong week. (Actually, I think all weeks are the wrong week for something that big.)
Facebook has done this at least three times previously. This time was better than previous because they actually laid out a roadmap with specific dates.
This is why the game I'm currently working on is built using facebook iframe/connect, and not fbml; the thinking being that this product should be able to stand on it's own merit regardless of which API i mash into it.
I don't have a good one-stop "Here's everything people outside the company know about Zynga" resource in mind, but my two secrets for keeping a finger on the pulse of social gaming are a) andrewchenblog.com (though he doesn't blog much anymore, which saddens me because it frustrates my hunger for testing/metrics inspiration) and b) Slideshare (It is amazing what goodies people give away at conference presentations and a huge portion of them eventually migrate to Slideshare.)
They aren't just throwing dung at the wall and seeing what sticks: they're very, very good at what they do. For every FarmVille there are tens of thousands of games which you have never heard of: their viral growth coefficient was .95 instead of 1.05, kaboom, that is all you need to fail. The platform is getting harder to develop for, as early entrants used techniques which were what SEOs might call "a wee bit grey" to gain traction, and these are either impossible now or will result in a new entrant getting hit with the banhammer. (For example, early social games maximized their viral coefficient by adopting a spam-early spam-often policy to user notifications, which ticked off a lot of Facebook users, causing Facebook to take a number of countermeasures. Ask somebody who builds for it to describe the specifics, but there are now throttles and some sort of system where you earn the right to use the viral channels.)