Yesterday I was surprised to find myself trying Yahoo search because I couldn't get satisfactory search results in Google. It was the first time in years. I started thinking about this, and I realized that in the past few months I haven't been getting particularly good results from Google. I don't get spam or anything, but a lot of times I don't get useful results.
The thing is, I'm not sure if it's because I do a lot of very specialized stuff these days, or because the search quality really has gone down. Consider these two examples:
Search for "Linux asynchronous IO". You'll get a lot of articles, but most are four years old (which is an eternity in the Linux world). These results aren't very good - posix AIO is implemented in userspace threads, and io_submit and friends don't work in many cases. Which cases? Hard to tell - I couldn't find any information in the results no matter how long I searched. I couldn't find any benchmarks either.
Perhaps it's because there is no good info on this on the web (hard to believe). So let's try something else - search for "concurrent hashmap in C". After hours of searching and playing with keywords, I got almost no useful results (other than Intel's libs, but not too much info on that either). It's difficult to believe that there are no good implementations out there.
So, is it the specialized nature of my searches, or is it Google? What do you think?
SEO: When you cut through all the BS, the entire goal here is to make a less good match come first. And it works (sorta). Just consider crap sites like Experts Exchange that we've only learned about because they pollute many searches.
Feedback effect: Thanks to google, less people do less collecting of good links. Why bother when you can google for it? So there's less good information for google to use in ranking links. Bear in mind that when google started, nearly every home page had a long list of links to all the pages that particular user liked and frequently used. I used to have one; I've long since deleted it; my blog has some outgoing links that I like, but relatively few. If I twittered, I'd probably post a lot of outgoing links, but of dubious value; there's no gardening of just the perfect page of 100 links going on anymore.
(I think this also partially explains why some (generally more specialized, so less effected by other things) results feel dated -- legacy links that are still hanging around from days when links were still used that way.)
Feedback effect: Thanks to google, ten sites tend to be more important than any other sites on any given topic. This results in certain sites becoming increasingly important. Wikipedia is the chief example here. Why is there only one Wikipedia and not a dozen? Chiefly because it's gotten all the google juice. If you want your wiki article on foo to show up in google, you naturally write it on Wikipedia, not Fooipedia. The result here is that all google searches feel increasingly the same -- of course Wikipedia is always in the top ten, or maybe something like Stack Overflow for a technical search.
----
So, these days, if I don't see something interesting in the top ten, I often click on the link to page 10 (or 20, or 100) of the results. Often more interesting. For example, google for "mashed potatos".
Top 10 results: "Perfect mashed potatoes" (SEO), allrecipies.com (always in top 10 for any recipe search), foodnetwork.com, Wikipedia, about.com, nytimes, etc. Pictures of mashed potatos. All generic and useless.
Page ten results: Dairy-free mashed potatoes. _Potato_ free mashed potatos! Caramelized Onion Horseradish Red Mashed Potatoes! A poem about eating them. At least marginally more interesting and quirky. What I would have expected out of google circa 1997.