(If I'm a 'noted Bayesian', then Bayesianism is in big trouble. I use frequentist techniques because they're what I know, and I wanted to learn them before I learned Bayesianism so I'd know what both sides were like.)
Wow, that site is great. It's like John Nash and a librarian had a secret love child. Organized thoughts are clearly a virtue, but not quite his forte.
I only skimmed it (which is more than I like doing with gwern articles) but:
The base-rate predictor got 65% right by definition, the logistic managed to score 68% correct (bootstrap20 95% CI: 66-72%), and the random forest similarly got 68% (67-78%).
You're right that it's not that great, although it might be more or less impressive depending on how much one expects these things to be predictable at all.
But one useful way to look at it is to see this as indicating how good the base rate predictor is compared to random commenters or your gut feel - 65%? Did you know, in advance, before I spent like 3 months collecting the data, that the right rate was 65% rather than, say, 10% or 90% or 50%?
I find it refreshing. In an age where everyone on the internet is trying to "tell a story" it's nice to just have some plain facts and information presented in plain old text once in awhile. Seriously, how many "why our startup failed/succeeded/grew/shrank/lobotomized" articles do we need? It's nice to have people doing actual analysis with numbers once in awhile.
It's not my style, but i'd be interested in any other sites like his, because as I said it's not my style, but I love reading in-depth things that are style rather personal in nature, kind of an odd spot to ask, but sites like that arn't easy to find. Thanks for the link, btw!
I wonder why, in the least risky list, he puts Translate before AdWords. There's no way Google will shutdown their money making machine, much less likely than shutting down Google Translate.
Am I missing something? What's so special about Translate?
Ah, I see. I should've probably continue reading it before asking. However, it does say something about the algorithm if it scores Translate higher than AdWords (it should probably be first, even before Search - I think they're more likely to return results completely based on AdWords with no organic results at all than shut down AdWords and return only organic results).
Statistical algorithms that make predictions based on signals can't make good predictions about something that is unlike they've seen before. The algorithm being used does consider profitability. Even if you added "massive profitability" as a signal to AdWords, an algorithm would still struggle because it's the first and only data point like that for Google. They have decent other businesses (hosting, enterprise, etc.), but none in the same class as AdWords.
No, it says something about adwords compared to other google services.
Normal predictors for shutting down google services doesn't apply to it, because it is very different in its behavior than other google services. That doesn't make it a bad algorithm.
