I don't see anyone addressing the question of whether good content actually exists for certain keyword searches. How can you blame Google for giving you bad results when they are the only results?
For example, if you search for "digital camera reviews", the first hit is dpreview.com, a fantastic site. Is there an equivalent site for appliances?
The real message here is that keyword optimized sites are addressing a real need, but doing it poorly, which suggests that there might be big opportunities for people who can beat them by producing better quality content.
But as soon as you search for a specific model, the spam gets out of control. Try to find a review on a particular model, or whether it's comparatively good, and you'll get very little material but tons of price shopper sites and other spammy reviews sites.
Hm, I don’t know anything about specific dishwasher models but I tried current digital cameras. I got Amazon, Wikipedia, various reputable review sites and one or two reputable price comparison sites. No spam.
When it comes to dishwashers the cause of the problem may be very simple: there are no reviews online (or anywhere). At least not when it comes to the dozens of same old and bland models dishwasher companies tend to spew out.
(And since I don’t know anything about dishwashers this statement was most certainly a gross oversimplification for which I apologize to all the busy dishwasher engineers out there.)
"Long-tail" SEO is definitely a weakness in Google quality control. It is remarkably easy to get yourself into the first search result page by autogenerating specific pages for specific models, a trick I learned recently from patio11, and for which I surely owe him a drink (my mom, a teacher, already has enough bingo cards).
The 45 second explanation of this: Fat head SEO is dominated by the link graph, which is (by design) difficult to influence in a scalable algorithmic fashion. Long tail SEO is dominated by on-page factors, domain authority, and information architecture [+]. If you have a reasonably trusted domain name and a way of generating hundreds or thousands of pages of more than minimal value, you'll tend to rank for those subjects by default.
This doesn't have to harm the user experience, incidentally. I am fairly upfront about what I'm doing: there are about 700 pages on my website which cost me about $3 each to write, and they are all designed to rank highly for a single search term or small basket of search terms. Probably 95% of them deserve the #1 ranking on Google because they are the best content for that search term on the Internet. (Better than 80% of them are the only content for that search term on the Internet.)
Similarly, the context the discussion I had with Thomas was also about creating user value out of a resource he already possessed but wasn't surfacing in a manner optimal to either searchers or his business interests.
[+] Edited to add information architecture, which I forgot to mention because I haven't worked on improving it in a while. Effective content siloing and interlinking is a major factor in why my site works as well as it does.
Epinions, CNet, and HowStuffWorks are also spam. Because they don't have any actual reviews, just lists of models, and a buying guide. (Although the domains themself are obviously not spam, just the results for this subject.)
ConsumerSearch and Consumer Reports are the only real results.
You're one click away from reviews on Epinions and HowStuffWorks. I'd say the same for CNet, except the first 3 dishwashers I clicked had no reviews, just a template dishwasher buying guide, and so I gave up. Of those three, only CNet comes close to spammy... but CNet's dishwasher buying guide is at least decently written.
I haven't found any reviews yet on Epinions, and I've clicked more than three times. They do have a nice side-by-side feature comparison feature, but reviews appear to be entirely user-driven so if no one has posted a review you won't find one.
I think ConsumerSearch, as a made-for-AdSense, made-for-referral-revenue site, is part of the problem Kedrosky is highlighting.
Also I wouldn't try any of your generic/naked term searches if buying a major appliance. The results don't really help discriminate among choices on major axes.
I would instead mix the categories with descriptors ("best", "reviews", "compact", "most reliable") or brand names. But unfortunately, that nowadays results in a maze of repetitive e-commerce pablum sites, many running AdSense ads. It's possible to pick out some useful information with effort -- but it's hidden in a lot of muck of Google's own making.
I hadn't come across ConsumerSearch before. Its content for the example 'dishwashers' query isn't that impressive, but isn't that evil, either. They're trying to be useful, but the busy, cramped, SEO- and AdSense-optimized layout makes its excerpting/summarization of other reviews very hard to trust.
Thus it's a lot like other About.com sites -- sometimes useful, until everyone else gets into the same 'landing page' game. Then they're all just variated regurgitation of the same C&P'd wisdom from elsewhere.
It aggregates and summarizes reviews from Consumer Reports, a bunch of other resources, and Amazon.com user reviews, breaks products down into three categories, and provides a top recommendation for each, runners up, and a methodology.
It's indistinguishable from what a pennies-per-paragraph SEO contractor might write, just excerpting and summarizing arbitrary reviews from other sources, chiefly as crawler-fodder.
It might be a wise synthesis of key points from many other reviews... but it'd take a while to earn that reputation as a trusted voice. In the meantime, its design is not giving me signals of reliability, though I can see it has those aspirations (from its 'about' page).
If it were signed copy, more clearly dated, with some quantitative indicators they'd done deeper research, it'd come closer to impressive for me. But, then the source sites might object about the quantity of material excepted.
How valuable is a review-of-reviews, by an unnamed someone who hasn't themselves ever tried the product?
I could believe that done right the review-of-reviews approach could theoretically provide some value. But it veers so close to pay-for-keyword-stuffed-article index-chaffing behavior my presumption starts out against it.
I see the problem as Google having reached a large enough scale for them to affect the web in the same way that you affect the price of a stock if your trade order is large enough.
When Google first wrote their search algorithm, they tailored it to the web in order to produce good results. Now we're in the opposite situation where everyone tailors the web to fit the algorithm. I believe this is a worse world. I would rather see site creators focus on creating the best experience for visitors to the site instead of trying to play the SEO game, either white or black hat.
In addition, it probably creates a feedback loop that stifles innovation in search algorithms. Google issues recommendations on how to do SEO, so sites follow those guidelines, which in turn leads to the web conforming to what Google's algorithm wants the web to look like.
A concrete example: Newspaper editors used to pride themselves on writing witty and clever headlines for their stories. But SEO guidelines dictate putting lots of search keywords in the headline and page title, leading to bland and boring headlines. Google could change their algorithm to accomodate the way headlines are written instead of the reverse.
"...churn out content cheaply and regularly, and you're done. On the web, no-one knows you're a content-grinder."
Spoken by someone who has never run an active content-based advertising revenue business. What you don't see behind that site with crappy content are 100's if not thousands of links pointing to that site. And THAT my friend is the real problem with Google search. Simply cranking out content is only a small part of the equation and will get you no where.
I have never understood why google doesn't have a down vote button next to each link. If a domain gets too many down votes (from valid Gmail users) it is marked as spam. -- at the very least allow users to block the bad domain from coming up in their own personal future searches.
Any google search employees have an answer?
My only guess is that google seems to be anti crowd source and pro algorithm?
I understand there are content grinders out there that are made to rank high for Google (but see, they are not made by Google nor are they Google itself). It sounds a lot more like giving a snake what it likes to eat--say rats--rather than a snake eatings its own tail, but since you say it's clear, please do enlighten me because I honestly don't get how the metaphor is supposed to work.
This is a bit (way) overblown. To me it sounds like a rant looking for a place to happen. I will freely acknowledge that people are creating content that provides little value to the searcher but rank highly in result-listings and picking up the ad money. But even if you get to a good appliance site, manufacturers are also gaming the review process so that you're unlikely to be able to tell the difference between somebody who really liked a product and some shill from corporation X.
Simply because sites appear in the listing that you don't like doesn't mean they are spammy. People are also writing good content for the same reason. If folks are searching for material and you can provide something unique and valuable (which is really only determined by the consumer), there is a role for additional content providers in any area. Heck, there was a HN article a week or two ago about a company that is making huge bucks providing random video instruction. Remember these guys? http://www.wired.com/magazine/2009/10/ff_demandmedia/
Finally there is existing content that is gaming Google's system to gain ranking where it probably doesn't deserve it as well, keying off of strategic partnerships and other things that just make them look better.
Once again, boys and girls, Google is an algorithm, not a way of life. Whenever you put an algorithm at the center of your company people are going to manipulate it. Google doesn't get to have the "magic" or right algorithm that is somehow non-game-able any more than Microsoft or anybody else does. This is an evolutionary process and we've got to expect change and adaptation. Seeing a lot of spammy sites in your results just means that the system is gearing up for the next evolutionary leap. Can't wait to see what it is.
As a personal story, I have a massage chair that gave me fits when I bought it a few years ago. So I put the name and model in my blog.
To this day I get about a comment a month with somebody asking questions about the chair and somebody else responding. Was that a spam article? Would it have been a spam article if I had purposely targeted that particular chair's keywords?
I think it's just content, and I think the rest is just content, and I think people write content for all kinds of reasons. If you incentivize it, people are going to do more of it. Search-by-keyword incentivizes creating content.
The problem is very definitely real. "Helpful" comparison and review sites come up far too often, filled to the brim with affiliate links and, if you're lucky, excerpts from and links to reviews only tangentially related to the product at hand, when you go searching for a product that doesn't have dominant niche sites.
And I don't see how Kedrosky is a black pot in this case. He is not a search engine, nor is he a spammer, nor a purveyor of affiliate-linking review sites.
"Find some popular keywords that lead to traffic and transactions, wrap some anodyne and regularly-changing content around the keywords so Google doesn't kick you out of search results, and watch the dollars roll in"
Is he not doing exactly what he describes? He is writing about a topic popularized earlier in the day by TechCrunch. There are no objective measurements of the assertion that searching for dishwashers produces spam. As comments here note, searches for "dish washer" product pretty good results. Face it, it's the HN equivalent of Google spam.