At the risk of appearing cynical, I'll bet on money driving the ranking of potentially profitable sites above obviously non-profitable ones before I attribute it to stupidity at Google.
The "weather" page is hand coded - the term triggers a unique piece of code that isn't used for any other search term.
"Pickles" doesn't draw a bunch of icons across the top of the search results and offer short form links to "Mt Olive," "delmonte" and "vlasic." The top link doesn't give me subdomains for MtOlivepickles.com in the way Google search does for weather.com.
When I query "weather," google sends my location to the commercial sites, but not to weather.gov. Explain that with statistics.
>At the risk of appearing cynical, I'll bet on money driving the ranking of potentially profitable sites above obviously non-profitable ones before I attribute it to stupidity at Google.
Then don't think of it as stupidity then. Think of it as having poor proxies for quality that we have to mash together into a reasonable but imperfect system.
Here's a video that shows how all decisions about ranking changes get made: http://www.youtube.com/watch?v=JtRJXnXgE-A I've been in that room many times over the years, and I have never heard anyone mention or display any statistics that involve ads or revenue.
I think of it as good proxies for revenue. It just makes more business sense, and seems more in line with the fiduciary responsibilities of Google's board in regard to Google's stock holders.
And that's why I don't doubt your statements that the algorithm doesn't do this or that. There are proxies. If Billy Beene could do it a decade ago in Oakland, it is hard to see the smart folks in Mountain View depending on dumb luck to generate revenue from search. And that's the only alternative to tweeking the algorithm to generate revenue.
"q=weather" shows an observable point where hand coding for business interests is accessible and the that holes in the wall are deliberately created. It further shows a correlation between the hole and revenue - note that weatherunderground.com generates revenue outside the ad model.
It's difficult to believe that this practice isn't automated.
Ranking tries to make money by returning good search results so that people use Google. Ads tries to make money by showing good ads so that people click on them. Ranking only cares about revenue in that they want to make a product worth using.
Look, I've worked in search at Google for over 5 years. I know all of the signals that determine how results are ranked and most of the details about how they are used. I know all of the metrics that are used to evaluate and tune ranking. I know nearly everyone who works in ranking, and I've seen a decent fraction of the launch decisions and the metrics supporting them.
I'm telling you, with no caveats, that we don't make ranking decisions based on statistics related to revenue. If that doesn't convince you, then I don't know what would.