Hacker News new | comments | show | ask | jobs | submit login
Paul Graham, the Commons, and How Google Stopped Being Google (theatlantic.com)
139 points by maxprogram on Mar 22, 2012 | hide | past | web | favorite | 103 comments

> It isn't that anymore, though. Searches are geospecific and social network-dependent. All of which is fine and useful, but that's not what made us love Google's search engine.

What makes you love Google is Google giving you the right answer.

If you live in the US, do a google search for dmv.

Did it pull the one from your state?

Do you really want an averaged across all searchers answer? California is probably the state the generates the most Google searches for dmv, but the California DMV is likely relevant iff you are in California.

This idea of one best search page for one query, independent of any other factors based on the user or users geography is really overly romantic and simplicistic.

On the other hand, the filter bubble stuff is a nice narrative, but it mostly makes for an interesting story rather than real problem. Having worked on search personalization at Google about 5 years ago, it's just hard to make that big impact on search results with personalization. "dmv" is a nice example where it's works brilliantly, but those ambiguous queries are pretty rare, and hence the more bread and butter stuff is still really driving most of the quality.

> If you live in the US, do a google search for dmv.

First of all, no, it doesn't give me the DMV for my state first.

And second, why would I do that when I can easily search for "$statename dmv" and have that search universally do the right thing regardless of location? I don't want hidden inputs to my search based on my location; I want everything that determines the results encapsulated in the query.

I think this debate hits at an important point; different people want different things. Some people would find it easier to type "dmv" and have the engine know they want "dmv open right now and closest to my location". Some people, like you, want the search engine to handle your input without external influences.

In the end, it might be optimal for the world to have multiple, specialized search engines/apps to please as many wants as possible.

The reality of it is slightly more meta than that. Some people just want the right answer. Some people want a tool that they can build a consistent mental model of.

Hackers prefer the latter. Everyone else? That's up in the air.

As I've said before, there are searches for consumption needs, and a completely different kind of search made for research needs. No popular search engine today differentiates between the two, but Goggle is slowly moving towards a search dedicated to consumption needs, which seems logical for an advertizing company.

I might search for pizza in order to find a place to order it from. Or I might become curious, and search for pizza to learn about its origins and history. A personalized search would only answer the first option.

Personalized searches also render it difficult to gauge how 'pizza' relates to other data in the public (sub)conscious. Google results used to (and still do, to a certain extent) provide a very good picture of the semantic structure of the Internet, if you knew how to read them.

It's not up in the air. Everyone else just wants the right answer. Even I sometimes just want the right answer, and I love mental models..

Having a mental model of how the google search ranking works sounds… hard. For me the mental model works out to be "based on various statistics, google's best match for my search terms". I'm not sure that's different from "the best answer". :-)

Is it even possible to build a search engine without external influences..?

I could not agree more.

That becomes even more frustrating when you (temporarily) are in another country and suddenly get smacked in your face by incorrect location based search results. If google wants to really retain traction with people who want to search for what they type (don't even get me started on the "showing results for: $bshere, if you want to see results for what you searched for, click here"), then they will soon have to provide a switch for it.

I really hope that google goes that way but I think it only crop up when people working at google find themselves using a different search engine to find (esp. programming related) specific topics.

Apparently I want to get you started on "showing results for: $bshere, if you want to see results for what you searched for, click here", because I'm amazed that anyone complains about it.

For the overwhelming majority of my typos, "$bshere" is exactly what I wanted to search for. I particularly appreciate this when I'm searching from my phone (where typos are more common). There have been times when it has gone the other way (searching for "WiDi" aka Intel Wireless Display is the most notable example I can remember), but those are rare enough (and understandable enough) that they don't bother me.

Yep the movie times searches on the uk are messed up beyond belife even on Google.co.uk - Google insistes on serving up me the restuls for Bedford Texas and not Bedford in the Uk

You are not the typical user that Google is aiming for. I find the unasked for word substitutions and stemming and "corrections" and the bubble frustrating, but I recognise that I'm an edge case. Google really should have some kind of "what you type is what you get" mode, with user-definable stemming and word substitution and bubbles and etc.


> we found that users typed the “+” operator in less than half a percent of all searches, and two thirds of the time, it was used incorrectly.

Check my arithmetic but that means only 1 in 600 searches used + correctly; 2 in 600 used it wrong, and the other 597 searches didn't use it at all.

0.5% == 3 in 600 2/3 == 66% = 2 in 3 are wrong

> Google really should have some kind of "what you type is what you get" mode


Doesn't work as advertised -- try searching for [["everything wrong" crossfade]] in verbatim mode.

Oh wow, if that works I'm going to use Google again. I hope there's some easy way to switch on that setting with a GET parameter so I can replace my default Google search.

..."with user-definable stemming and word substitution".

I think you have to be pretty technical to dislike hidden inputs. But hidden inputs generally please mere mortals.

> What makes you love Google is Google giving you the right answer.

No it's not. What I loved about Google is giving me the ability to search the web. If my answer wasn't on the first page (or even the second), I'd refine my query until I would get it. Which was easy because Google was very deterministic, just your basic "AND" keyword search engine. Try a few synonyms maybe, different angle, synecdochical approach[1] and if all that didn't get you your answer, it probably wasn't there[2].

Today, you type in your query, Google tries to guess what you need, and shows you your results. If your answer is not there, you can try different queries, synonyms, whatever, but you'll get mostly the same results again. The web being unimaginably bigger than 10 years ago when I used the approach described above, I refuse to believe my answer's not there. Google is just being dumb.

Anyone know of a proper Search Engine? (I love Duck Duck Go but its index is rather small)


[2]actually, you were probably not trying hard enough. Pressing on over the course of a week and getting some expert friends to help you would always show that your answer was in fact there. in some form or another.

Do you have an example of a search that doesn't currently find good results, but you suspect it would work if Google worked the way it used to?

That's a rather difficult question to answer. I usually don't remember what I've searched for recently. I'm just relating my general impression, which has gotten much much worse over the past year.

Really it's hard to say because 9 times out of 10 it works fine. Probably even more. Because most of my queries are simple ones. I'd have to go dig through my browsing history, it'd take a lot of effort. I did it once, when Matt Cutts asked a similar question, but right now it's rather late here so I hope you can excuse me :)

One thing I can say, I have tried to diversify my usage of Search Engines in the past, to not just use Google but also Yahoo, Bing, and others. Even changed my default search engine sometimes. But it never stuck. Google just was too good--its speed was a big factor btw, even when the results weren't great, the fact that you could quickly try different queries made up for it. But since a couple of months I find myself more and more going for DDG at the first query--also because if DDG doesn't have my answer it has a convenient link to Google to retry, if Google would have the same I might even still use Google with DDG as a fallback instead :)

Another tweak I did was disable Javascript on the Google result pages. There are a lot of shortcut keys I use in Opera all the time and Google's persistent autofocus was really getting in the way. Especially with the instant-refresh feature, your whole results would change just because you pressed some shortcut key even though the input box wasn't even focused!

Try the literal search feature on Google.

You mean this one? http://googlesystem.blogspot.co.uk/2011/11/google-verbatim.h...

Yeah I'll investigate that tomorrow.

I'd prefer the local DMV be a special result, clearly separate from the universal search. See how "current time" gives you a special result on top - I think they could give similar treatment to a link.

More generally, I think users appreciate when recommendations are transparent. Amazon and netflix are great about this - users get to implicitly choose when they want "recommended for you" or just basic results. I'd like Google search more if it had the same model.

I did a Google search for "dmv". The first result, an ad, is for Florida, where I live. The next three non-ad based results are for California, Virginia and New York. The fourth (and on) non-ad results are for Florida.

A Google search for "florida dmv" returned Florida-based results.

What I am tired of seeing are Eames chair ads all over the place. Do one search for Eames chairs, and click on one paid result, and now all I see are Eames chair ads. Sigh.

At least it's a break from the casino school ads I was getting.

This is called google "remarketing". Basically what happened is when you visited the Eames chair site, you received a remarketing cookie.

Then when you visit other google properties, google automatically gives you ads from that Eames chair site.


I did the search from my home computer. The Eames chair ads are showing up not only at home, but on my computer at work as well. Seeing how I've never logged into any Google property at work, I find that interesting.

The first result, an ad, is for Florida, where I live. The next three non-ad based results are for California, Virginia and New York. The fourth (and on) non-ad results are for Florida.

So you click on the relevant link ("happened" to be an ad,) as the Stanford MBAs high five each other at Googleplex. I remember reading a very interesting comment about the incentives of providing the best results and ad revenue. It's clear to me that ad revenue has been winning at Google.


I work on Search Quality at Google. We have a firewall between Search Quality and Ads. We don't answer to Ads. Revenue is just not my team's problem. We come into work every day and look at our search results and try to get the most relevant answers ranked as #1, #2, etc. I never think about the impact of my ranking improvements on ads. I've been personally involved in dozens of changes, and I've never once heard anyone argue for or against releasing a change because of ads.

I can understand why someone might be mistrustful because of the apparent incentive to sandbag the algorithmic results. But Google has the long-term view: if we can keep satisfying our users' needs, the revenue will follow.

You can only speak about yourself, you do not know what goes behind the scenes. Firewall, like the investment banks have? Don't take personally, enjoy your paycheck and stock grants, you work for a corporation that answers to Wall Street. Listen to the earnings call once and analyze what the ad people say about "innovation."

But Google has the long-term view: if we can keep satisfying our users' needs, the revenue will follow.

Personally I no longer buy that. The revenue is now #1 and now Google is spending like crazy to keep the users. It started last year full speed

Logged in, and in Texas, my search for 'dmv' returns the California DMV.

Then New York.

Then Virginia.

The only results for Texas DMV are two ads at the top of the result listing for sketchy drivers-license.org type "businesses".

hmm that might be because we don't use the DMV here in Texas the same way - TxDOT and TxDPS are the two agencies you're going to interface with to handle the stuff that California's DMV does.

I've been in TX 5 years now (and owned a car that whole time) and never even knew we had a department of motor vehicles.

A little off-topic, but...

Why is this getting down-voted? Is it because you disagree with what this person is saying? I thought down-votes were only for things that didn't contribute to the conversation.

It feels like this is getting down-voted because people don't like that Google is doing these things. So, this person is getting down-voted 'cuz of Google? Huh?

Sadly, Paul Graham has endorsed the idea of using the downvote to express disagreement. I personally dislike this use of the downvote, but it does have official sanction. Read this:


Just because something CAN be done doesn't mean it SHOULD be done. I think the community benefits from people expressing counter-arguments far more than grey-ing out minority opinion.

I'd hope a majority of users here feel the same way and act accordingly - pg's comments not withstanding.

> I think it's ok to use the up and down arrows to express agreement.

There's a dangerous consequence to saying this. Here's how: 1. Voting is easy.

2. Voting is valueable: it literally changes HN for everyone else.

3. Agreement/disagreement is easy: we're taught how to do it as children.

4. Agreement/disagreement is value-less. Saying that you agree with pg on 'Hackers and Painters' is valueless. Saying "prior to reading H&P, I was in mindset A-B-C supported by points of view D-E-F, and here's how they were modified, and here's the new mindset H-I-J" conveys the same agreement, but it provides value.

We shouldn't incite people to use up/down votes to express something value-less. Agreement and lack thereof (alone) is valueless.

> I thought down-votes were only for things that didn't contribute to the conversation.

yeah, well.. you need to see my posts on Apple threads.

Some people might have downvoted based on the DMV example apparently being wrong. Factual inaccuracy can be a good reason to downvote.

>Do you really want an averaged across all searchers answer? California is probably the state the generates the most Google searches for dmv, but the California DMV is likely relevant iff you are in California."

My state has a Motor Vehicle Division.

"DMV" gives me California's.

This page which does not appear on Google's first page: http://en.wikipedia.org/wiki/DMV - though perhaps it should.

Two points.

> it's just hard to make that big impact on search results with personalization

Trying to make a big impact on something that was more or less working might be the problem. A "don't fix what's not (too) broken" could be a good idea.

For your dmv example, actually, I din't know what dmv is, and my fisrt page (french localized) is about a medicament dictionnary for doctors (DMV, didn't know it either).

"dmv usa" doesn't get me the dmvusa.org result firt either (it's the one I get on the US google)

This seems to be the best example of why geospecific data can be widly irrelevant, and using the default setting, I get this kind of unusable results for half of what I search.

I was expecting that DMV is shown as special result on top - something like "are you searching for DMV office next to your location?" ...

But you explained the core of the problem. There are two kinds of searches:

1) discovery of information

2) finding information I probably know but I'm just too lazy to go that site directly (or something in that sense)

The first kind of search is what made Google the most popular search engine. The second kind of search is something new for Google and it seems it is implemented as a unsophisticated simplification of the complex problem. It is like AOL search all over again.

That's ok until Google infects a current search with a previous search as it seemed to do for me the other day. In some cases that might help, and although they were in the same domain I was actually looking for something else. Maybe it was an accident, but the result (at position 3 or 4) looked very much like it'd been boosted from my search a minute earlier. Anyone else experience this?

I feel like there's a recent anti-Google sentiment among the hacker/techie community which seems very unsubstantiated by solid reasoning or logic. The argument in particular, that search quality is getting worse, is a little lost on me because, for me or anyone I talk to at least, this is simply not true.

In fact, I find myself spending less and less time on "Google the Search Engine" because it's gotten so damn good at giving me the right result right away based on the simplest and vaguest keywords (I use the omnibar so I don't even go to google.com anymore). I still remember not so long ago having to try different phrasings of my search keywords a number of times before I would get the result I was looking for on the main page (or in cases of extreme frustration clicking through pages and pages of search results). This is largely not the case anymore.

From my general interactions with people and how they use the web, anecdotal as it may be, they simply doesn't give a shit about ads, filter bubbles and so on. Google Search hasn't changed for them in any meaningful way except that it's gotten a lot better at figuring out what they wanted to find (and in most cases they are not even conscious of that improvement over time).

With regards to other other free G-services (Gmail, Docs, G+, Blogger, YouTube), if you are consciously buying into them, it is implied that you know you are entering into a relationship where parts of the real-estate you're interacting with might be used to display ads and I think everyone understands and accepts that.

I am an avid conspiracy theorist, however, and have more long-term concerns about what happens if Google turns TRULY evil and starts to use its data to spy on citizens on behalf of some authoritarian government. But as of right now I feel a little foolish reacting to this concern in any significant way.

I have stopped using Google Search entirely very recently. The main reason is that Google too often misinterprets my search queries, and I got tired of typing quotes around the search terms.

I should stress that like any software developer (or any power user, probably), my queries tend to be very specific. I'm looking for particular information (which may or may not exist) that takes more than a couple of words to describe. It is with these queries that all too often Google simply decides that I meant something else and replaces one or two words with what it has decided are synonyms. Except they aren't.

Unfortunately I have not documented all these queries (I should have). But to give some examples, it would not be the first time that Google misinterprets a specific query about "sqlite" and only returns results related to SQL Server. Or it decides that a "scapegoat tree" is equivalent to an "AVL tree". They aren't synonyms. The results are actually worse than irrelevant.

It's like typing a query such as "honda brake problems" and getting results that are about "toyota brake problems". Yes, both are Japanese car manufacturers, but they are really not equivalent.

Of course correcting typing errors and searching for synonyms is useful. But I'd like to have control over that behaviour, and most of all: I want to understand what the search engine does. The best interfaces are those where a user can develop a mental model that helps them interact with it. It seems I'm no longer able to create a mental model for how Google works.

> "The main reason is that Google too often misinterprets my search queries, and I got tired of typing quotes around the search terms. I should stress that like any software developer (or any power user, probably), my queries tend to be very specific."

So who do you think should invest the extra effort into performing searches (aka adding quotations.. or "taking them off"): the hackers with specific, but edge-case search requirements, or the 99% of casual users who just want to learn about "how do helycopters fly"

P.S. I tried both the sqlite and the scapegoat tree searches without the quotes and Google seemed to "know" what I meant... but if you encountered that problem perhaps it's an algorithm that unnecessarily "kicks in" depending on some other circumstances of the search. I would definitely classify this as a problem, but one that should again be easily solved with the use of quotation marks

I agree 100% with you that there is a large difference between search engine use by hackers/power users and by casual users. But given the frequency of the problems that I have, I definitely believe that 1) I can impossibly be the only one that experiences this and 2) therefore there may be a lot of room for improvement. Room for Google, or for a competitor. (I just don't expect it will be Google, since they are the ones that made it worse.)

By the way, the scapegoat tree search query was "scapegoat tree inorder append". Try that and observe 90% AVL tree results on the first page.

For me the feeling of reduced quality is that when i don't find the answer on the first page, that i have less luck with additional keywords and more pages then i think i used to have. These queries are mostly trying to cast a wide net with some errorcodes or stacktraces and seeing if others reported similar problems.

For me there are two problems.

1) Google focuses more on commercial sites, especially branded sites, than research or personal sites.

2) Sturgeon's law (90% of everything is garbage) and a much bigger world wide web means that there's a lot more garbage indexed, and returned, by Google searches.

I remember that I'd try a search phrase, and I'd be taken to some Web 1.2 site with black text on a grey background; some university professor or doctor would have written a really interesting article and linked to other sites or research. Now I get some reputable site selling stuff, or a bunch of shitty link farms offering me god-awful "reviews", or awful awful blogs.

I remember when search terms would return less than a gajillion hits; you could tell by the extra 'o' in Goooogle how many hits you'd got. When was the last time GoogleWhacking was possible? A result of "zero hits" is important. It doesn't mean the search failed and that Google should feel free to substitute words and try different stemming and remove the quotes. "Zero hits" information for me.

I strongly agree that I'm an edge case, and that Google is great for most people.

I personally have to modes:

1. I already know what I'm looking for: a particular website, post, comment, the lyrics of a song that I know a part of.

2. I have a question, I don't know the answer, but I'll recognize it when I see it.

In my experience Google is getting better at type 1 '(re)discovery/popularity' search and worse at type 2 'exploratory/specificity' search. Depending on the ratio of type 1 vs type 2 in your search habits, Google results will get better/worse for you.

They introduced Verbatim search to solve the specifity problem, but in my experience it hardly ever works :(

It reminds me of HN itself.

I don't want to know what the "average" likes to read, or even what people like me like to read. I want to know what smart, interesting people (not me!) like to read.

In other words: I want curation by an expert, not the average of everyone's inputs. Google is supposed to know the right answer, not ask all my friends what the answer is and report back.

I completely agree with you on this topic. In a similar vein, I don't want to read the average Joe's take on why he loves the iPad 3 so much (I usually abstain from commenting, though lately I've said what the hey) - I want to hear what smart, more enlightened people have to say. That was the real value of HN for me when I first got on years ago. It was hearing from people like patio11 when the topic of SEO came up.

The thread recently where Jordan Mendelson (napster hacker) chimed regarding Napster's back end is the quintessential example of what I'm talking about.

this is really interesting. What People WANT and what they THINK they want are two different things...How people define and Perceive SEARCH and how they use search are two different things.

I think Google is facing a SELF-DEFEATING reality that as they serve people with better results, they erode the perception of their brand in peoples eyes.

There is clearly an incongruity between what people "THINK" they want from a search engine( and how they perceive Google) versus, what they "REALLY WANT" from a search engine.

Conflicting priorities are always a struggle, but this is an especially interesting one. One side has a DATA supported reality that can't be disputed about what searchers want and what puts bread on the table. On the other side is the more important, but less quantifiable reality that their BRAND is what brings searches back to the site, not the quality of the results, but the perception of GOOGLE and SEARCH being synonymous. This brand value is clearly getting diluted by delivering better results, and could in the long run be their demise.

This is more insightful than the article. The problem isn't optimizing for the "community" as the article seems to think. The problem really is that what people think they want and what they actually want are opposed. Facebook faces the same problem.

I think the unfortunate endgame here is legislation that will require what people think they want (e.g. privacy, absolute control) at the expense of some things they actually want (e.g. frictionless social interaction, simplicity).

(To forestall some argument: Yes, I realize that privacy/social interaction and control/simplicity are not always mutually exclusive. However, there are cases where they interfere with each other, which requires compromise.)

The problem with the approach Google is taking, the reason why I 'left': I felt that it became harder and harder to actually get what I want (using your caps: WANT). I know it. No, you don't know better. No, I didn't misspell that, no geolocation is a bad idea for results or - worse - language.

It's a mediocre service now. Became big with search, became worthless for searches for me. I cannot search anymore, I can play games with software that tries to second guess my intentions.

I almost wonder if SEARCH by definition can't and shouldn't be the end answer, but rather the journey to discovering new things and finding answers.

Google is assuming that when people search, they want answers, but maybe what they really want is to SEARCH for answers.

This is a really good article. Succint, reasonably well written and interesting. That being said, one imagines that Google (despite being an advertising company) have a huge investment in the quality of search.

However, because they are essentially a monopoly, many people try to game the system for their benefit. This creates an arms-race, and may be one of the root causes of this personalisation. I do agree with the main point, that this could potentially make Google less useful for us all as a whole, and will probably lead to their disruption, sooner or later.

Alexis Madrigal reaches a subtle and thought provoking conclusion "The aggregation of individual data does not a commons make." This would seem to be at odds with a lot of "Big Data" evangelism but I think he is right.

This article is custom tailored for the HN echo chamber. The link to PG for this article is tenuous at best.

I agree Google has gone into a filter bubble, but the article very speculative and provides no statistics or examples to make a good point.

This is a pop-science article, so I don't have high expectations.

Why do you agree then that Google has gone into a filter bubble?

Based on various other articles I've read. I would not go to an extreme and say "this is a terrible idea," but I would assert it's worthy of serious thought on how to ensure we are not limiting ourselves to the filter we've built.

Nonetheless, my criticism was directed at this article for not providing at least a few examples of the results of a filter bubble.

The author is prompt to admit that this is pure speculation on his part, and so we don't know how and why Google became what it is; but I find his explanation very convincing.

Serving personalized results to users may increase click-through rates but it degrades the idea we form of what Google is and what it does.

> Serving personalized results to users may increase click-through rates but it degrades the idea we form of what Google is and what it does.

If it serves me better results, then how so?

Who says it serves you better results? If I type in "tea", am I searching for Tealuxe in Harvard Square or am I searching for the botanical and scientific material on a common drink or am I searching for recipes? The new "social+local" Google biases my results towards the former.

A search engine used to be a tool for reaching out into the world and finding something new. Now it searches through what I already know? What's the point in that?

Search engines were never a content discovery tool for me. I use HN, Twitter, Facebook for that. Some others use Digg, Pinterest and Reddit. When I search on a search engine, I am looking for a solution to a question that I have.

Better can mean 'more liked' and 'more accurate' which are not necessarily equivalent. Accuracy being something of a fuzzy category, of course.

Here’s how: http://dontbubble.us

I always thought that search personalization was of course better, until that page made me think about the downsides.

The problem of course is that what Google actually IS and what people THINK Google is are two completely different things. Google is an ad company, always remember that.

The argument that "it's a business" is weak and not very helpful.

Google makes its money from advertising, but in order for it to place ads it needs to serve relevant results; serving "tailored" results because they generate more clicks is short-sighted (well, that's the argument anyway). If people become so distrustful of Google that they stop using it, and use a combination of DDG or Wolfram Alpha for example, then there will be no more ads for Google to serve.

We're admittedly very far from this, but the fact that high-profile people such as PG start to complain about it publicly (and not just old anonymous grumps like me) should make Google think.

We'll see.

"It's a business" is a weak argument, but potentially a helpful one. Your optimal search experience generally won't line up with theirs. In retail, for example, there are all kinds of tricks used to keep customers in the store as long as possible without making the experience so unbearable that customers just won't come anymore. So if you go to the grocery store for some fruit and vegs and yogurt, you will need to visit opposite ends of the store, through the seasonal items and past racks of candy, because this apparently results in increased sales. If they were optimizing for you, they would have the basics near the entrance, and the Peeps and cinnamon brooms at the farthest corner. They might still have the best produce in town, but as they figure out how to wring more money out of their customers (on average), the experience becomes grating for some portion of their clientele.

>"in order for it to place ads it needs to serve relevant results;"

or Google could serve ads by developing a suite of web apps, or a mobile operating system, or a even a social networking site.

Google could even serve ads on other people's websites when people visit those websites without going through their search service.

Google is not an ad company. People just like to say that because they think it's clever or surprising or something. Google does not produce ads. They sell ad space. So does Facebook. So does NBC. These are product companies selling ad space.

And without a great product, the ad space wouldn't be very valuable.

Totally agree. This is why Google cares so much about the quality of results.

Are you aware that Google runs an entire ad network? It's said to dominate the field of online advertising.

Anyone know what proportion of their revenue comes from ad placement on their own product pages?

I didn't understand the reasoning behind the retort.

>Are you aware that Google runs an entire ad network?

Which would mean that they would want to keep such third party sites interesting, fast and high quality as well. Chrome, Analytics and their other similar products/services are hence justified.

I'm saying the search engine, email, phone OS, and social forums are Google products which are just tangentially related to their advertising business. It would be interesting to know what the real split in revenue is in the discussion about what their real "product" is.

Its called "the filter bubble" http://www.youtube.com/watch?v=B8ofWFx525s

With the "old" Google, a search made me feel as though I'd stepped into the potential realm of answers to my query. The most relevant, to me, was the stuff that mapped to my query the best -- nothing more, but nothing less.

Now, I feel like I'm living in the law of averages. I'm not interested in the average; I want the top of the scale. I really do not want social and local and all the other unknown contexts involved in giving me what's right "for me". With all the contextual crap, I feel like now I get the "less".

I wish they let me have the option of turning it all off, and just going with content-based search. Of course, they have to filter out the crap that's being generated to game their search index, but outside of that -- give me what the web has.

You can get the unpersonalized version when needed. Simply log out of your Google account. Yoi can use a different browser, too.

Discovery and personalization are not compatible with each other. So if you use Google search as a discovery tool, then personalization hurts.

There is no alternative. Maybe DDG but results are not so good.

The Dr. Frankenstein in me loves the idea of an automated data-driven business: generic algorithm + A-B testing + payment mechanism. If you could also automate litigation... running many in parallel, a useful monster might one day emerge. Evolution has worked well elsewhere. The real problem is an exponential search space, and time/experiments needed to explore it.

The problem of giving people what they want is also the problem of markets and democracy in general. Fortunately for us buyers/citizens, we usually eventually realize what we really want - unfortunately for the vendor/politician catering to what we previously thought we wanted.

There's an appeal to Jobs' approach: make what they will want once they've see it. Far superior to creating/governing by numbers.

The article points out reasons as to why many of us are leaving Google for other search engines that protect our private data by not recording our search information. Startpage for instance does not track your private data. You will not have your IP address recorded or have tracking cookies put on your computer. I know we now live in a world where it is difficult to get away from Google altogether. It seems like they want to collect data on everybody in various ways. However, many of us are beginning to try. My gmail account may be the next to go.

The "personalization" of the web is not a Google-specific phenomenon. I am, at this very moment, sitting in a session at the Adobe Digital Marketing Summit where a marketer from Toyota is espousing personalizing site content to users.

This practice is being pushed _hard_ to all businesses here as a central theme of growth of the internet. In the sense that this drives better business, it certainly will be. Don't expect to go to Toyota.com and see the same experience as your neighbor anymore.

Does anyone know an easy way to prevent Google from tailoring my search results based off my location? I don't want to use a proxy, because then the results would just be tailored to that (false) location.

I know pws=0 works for turning off personalized search.

More Search Tools : Verbatim?

What I find fascinating is that with the sheer unlimited amount of data they have the quality of their search engine results have been stagnating for almost 10 years and no substantial break through been seen so far.

The only somewhat "break through" could be seen in local searches - and saving people from adding their zip code, city or street name to their search phrase seems to be not that much impressive either.

It's nice to romanticize what Google might've once been, but at the end of the day Google is a tool that provides a service. To have a qualm regarding the current user experience is a different story, but in terms of the delivery of sought after content I don't think one could argue they have taken steps anywhere but forwards.

At first search signals were mainly provided by the content itself and now signals from the user are being incorporated.

There is already an established term for this and it’s not “filter bubble” or “tailoring” the term is “relevance” and it’s the core of search.

If the author is suggesting that data about a user’s identity, history, and location isn't usefully to providing her with the best possible results then he should stick to writing about stuff he actually comprehends and not be the umpteenth author resorting to use “Google” in the title to whore traffic.

>"They make changes to the user interface and collect more data, analyzing what they've got to see what users "liked" more."

I find this so utterly naive that I am incredulous.

Google is a business. They don't tweek results based in favor of user preferences.

They tweek results based on the amount of revenue they are likely to bring in.

It's not an accident that [in the US] http://www.google.com/search?q=weather provides top links to commercial advertizing driven sites for one's local weather and only portals into the main page of weather.gov rather than the local forecast page.

Google search results aren't the best possible for the most people. They are the worst tolerable for maximizing revenue.

[edit] do employees have to use 20% time for downvoting on HN or is it part of their job?

Either you didn't read the article or you didn't understand it. The author doesn't complain that Google search results aren't "the best possible for the most people"--au contraire!

He complains they're not the same, that if you search from a given location or being in a given demographic you will get different results than other people, and that this fact erodes the trust users have in Google: how do I know if this set of results is the absolute best on a given topic or if Google is simply showing to me what it "thinks" I will like?

It isn't showing you what it thinks you will like at all.

It is showing you the most amount of pertinent advertising that it thinks you will tolerate while promoting sites containing similar advertising as high up the page rank as it thinks you will tolerate.

All those links to your G+ friends are promoted to keep you on Google's properties where they can make the most money when you click on an advertising link - think of Google like a DisneyWorld's Magic Kingdom and G+ like Downtown Disney.

"They tweek results based on the amount of revenue they are likely to bring in."

This is entirely false, and I wish misinformed people would stop parroting it. There's an extremely rigid firewall between the ad side of the company and the search side. People working on search don't even see metrics on how their changes effect ads, let alone make decisions based on them.

When I want change the way ranking works, there's a big stack of statistics I have to collect. None of them involve ads. All of them involve how useful the results are to a user.

I'll believe it when someone can explain why a search for "weather" returns the results it does.

The Weather channel utilizes Google search.

The Weather Underground...well name a Google service they could use and don't.

Accuweather provides plausible deniablity.

On the other hand, weather.gov employs some of the best meteorologists anywhere, provides the official hazardous weather warnings the other sites rely on, and has a wonderfully informative hourly weather graph for any given location.

In other words, Google doesn't provide the most relevant results first when it comes to local weather in the US, it provides the most profitable.

But I'm willing to read your statistical explanation for why this is so. Go ahead and make your case.

Be sure to explain how the special format for the weather page is based on statistical data and not hard coded.

I get weather.gov at #3 after weather.com, and sfgate.com/weather (probably because I'm in the bay area.) Where does it rank for you?

I can't get into detail explaining what goes into our rankings to you. I can tell you that I can bring up a debug view that shows all the signals that enter in to this ranking, and none of them involve "uses google products."

We love for people to complain about queries. That feedback is very useful, and we always need more of it. But when our search results suck, the reason is that our system is too stupid, not that it's perversely smart at funneling you into money making opportunities.

At the risk of appearing cynical, I'll bet on money driving the ranking of potentially profitable sites above obviously non-profitable ones before I attribute it to stupidity at Google.

The "weather" page is hand coded - the term triggers a unique piece of code that isn't used for any other search term.

"Pickles" doesn't draw a bunch of icons across the top of the search results and offer short form links to "Mt Olive," "delmonte" and "vlasic." The top link doesn't give me subdomains for MtOlivepickles.com in the way Google search does for weather.com.

When I query "weather," google sends my location to the commercial sites, but not to weather.gov. Explain that with statistics.

[edit] At the risk of further cynicism, I'll explain it with the relationship between WeatherUnderground.com and Google: http://en.wikipedia.org/wiki/Weather_Underground_%28weather_...

>At the risk of appearing cynical, I'll bet on money driving the ranking of potentially profitable sites above obviously non-profitable ones before I attribute it to stupidity at Google.

Then don't think of it as stupidity then. Think of it as having poor proxies for quality that we have to mash together into a reasonable but imperfect system.

Here's a video that shows how all decisions about ranking changes get made: http://www.youtube.com/watch?v=JtRJXnXgE-A I've been in that room many times over the years, and I have never heard anyone mention or display any statistics that involve ads or revenue.

>"Think of it as having poor proxies for quality"

I think of it as good proxies for revenue. It just makes more business sense, and seems more in line with the fiduciary responsibilities of Google's board in regard to Google's stock holders.

And that's why I don't doubt your statements that the algorithm doesn't do this or that. There are proxies. If Billy Beene could do it a decade ago in Oakland, it is hard to see the smart folks in Mountain View depending on dumb luck to generate revenue from search. And that's the only alternative to tweeking the algorithm to generate revenue.

"q=weather" shows an observable point where hand coding for business interests is accessible and the that holes in the wall are deliberately created. It further shows a correlation between the hole and revenue - note that weatherunderground.com generates revenue outside the ad model.

It's difficult to believe that this practice isn't automated.

Ranking tries to make money by returning good search results so that people use Google. Ads tries to make money by showing good ads so that people click on them. Ranking only cares about revenue in that they want to make a product worth using.

Look, I've worked in search at Google for over 5 years. I know all of the signals that determine how results are ranked and most of the details about how they are used. I know all of the metrics that are used to evaluate and tune ranking. I know nearly everyone who works in ranking, and I've seen a decent fraction of the launch decisions and the metrics supporting them.

I'm telling you, with no caveats, that we don't make ranking decisions based on statistics related to revenue. If that doesn't convince you, then I don't know what would.

You could actually show us what goes into the rankings for [weather]. Until then, some will be cynical, whether you think it's warranted or not from the inside.

As noted previously, explain "q=weather"

When I go to q=weather, I see a local weather summary above the search results. Is that what you're referring to?

I think of it as additional information Google supplies, I don't think of it as part of the search results.

I liked the old Google better. Today DDG has the simpler interface and gives me 50% more search results to choose from. Google looks more like the old Yahoo or Netscape portal every day.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact