If you don't like Google’s results use another service.
Sure, but Google has two kinds of customers: searchers and websites.
Maybe you believe websites shouldn't hide from Google because it makes no strategic sense. Twitter and Facebook don't let Google index their content. Why would they let Google front-run ads on their content? It's not a catchall rule, to hide with robots.txt.
Maybe websites should just pay up: they should auction for higher results, if their business depends on search placement. But that's misunderstanding Google's role.
Google isn't a salesperson. It's more like a road. Both bring business to your door (literally), but the latter gets used by everyone for everything. And it turns out it doesn't matter how you pave the road (roads have been public since the Egyptian times, yet road-building technology has changed dramatically) or whatever.
To Nextag, Google is a utility. It is only by tradition that some monopolists become utilities and others do not. How do you regulate Google for what it is: a public utility?
If searchers can't find what they're looking for on Google, they'll go elsewhere though. Before Google downranked sites like Nextag I seriously considered moving to a different search engine altogether because they were rendering it pretty much useless for any topic they'd got their claws into. I'd search for reviews or technical information on a netbook or some other consumer electronics and get page after page of nothing but useless "comparison shopping" and "vertical search" sites, none of which actually had any meaningful information on and all of which appeared to be automatically generated.
It's no wonder that Google cracked down; they were destroying its main service - search - in order to make a cheap buck for themselves by driving out useful results and driving searchers to turn to them in desperation.
Google has to get searchers to sell to advertisers. That said, the generic 'websites' mentioned above are not Google customers. Google does not "owe" them anything – they owe the searchers the best results and the advertisers clicks at a certain cost. The only thing those other websites do with their free listings are attract eyeballs to Google's engine.
No, a searcher is not a customer, a searcher is the PRODUCT. Google is the marketplace, their customers (advertisers) are buying the product (traffic) via ad spend.
The entire reason Google improves the search engine is to keep you coming back so they can learn more about you to make their advertising even better. You, as a user of Google, are their product. They are selling YOU to advertisers.
Google is a company, not a utility, not a public service. They are in the business of profit.
However, cheap polemics (like being very adamant about who gets to be the customer) help nobody. Two-sided markets can push prices for one side of the market below zero, that’s just an inherent property of those markets. That does not mean the one side is the product.
Google has to provide value to you, otherwise they couldn’t make money. If they screw you they screw themselves.
Now, this doesn’t mean that Google can do no evil, but it does mean that you can’t just assume by default that the relationship between users and Google is an exploitative one.
(And two-sided markets are nothing new. Google has essentially the same business model as the free magazine that informs me about cultural events in my hometown. Because of their very well targeted ads – local restaurants, businesses, etc., often pointing to current events – they can offer that magazine for free. That doesn’t mean I’m the product.)
I completely understand that. My point was that Google has to keep searchers happy or they have no product to sell advertisers. In that sense, the searcher is a 'customer' (you may like 'user' better) that Google needs to keep happy in order to maintain its product and keep it's real customers (advertisers).
That said, the greatest search engine in the world isn't that great if no one uses it. Search volume (and advertising platform quality) is what drives advertisers to Google. If their results suck, users will use something else and lose the product that Google needs to be able to sell to its customers.
(You can say the same thing about the rest of their products as it is all about collecting peripheral demographic information that allows advertisers to directly target them on the content network).
This 'users are the product, advertisers are the customer' insight can be useful, but can also be overdone.
It's also useful to recognize that for some of Google's lines of business, the payment offered is not money, but attention (or perhaps habit). Then, searchers are again customers, after a fashion.
This better recognizes the searchers' customer-like power, in some dimensions. Otherwise, as just the 'product', were they mined from the ground? Grown like crops? Manufactured? No, their attention was acquired in a voluntary, market-like but non-monetary exchange of value for value.
Once Google gets competitors like Bing, the product ceases to be the user and the product becomes the paid ads. If the business isn't getting their money's worth from the ads + the free stuff from google, they stop buying the product. Google apps, analytics, etc. are also products.