Google has always had "bad neighborhoods" -- places where results weren't so good. What folks are finding is that the bad neighborhoods are on the rise, at least when it comes to short, popular searches. Now it appears the screen scrapers are busy at work targeting tech questions. In the last couple of months, when I had a technical question I got total junk for an answer -- lists of questions that took me to landing pages, re-dos of Stack Overflow pages, and random questions that didn't even have answers.
I use Google extensively for search. About once a month or so, I'll be looking for something in a bad neighborhood. It's not a pleasant experience. It's a shame to see tech questions end up like this.
But the problem, as another poster pointed out, is that nothing is for free. You are either paying money, in which case you are the customer, or you are the product. There's no "in-between" In Google's business model you are the product.
I think the business model can continue for a good, long time, but there is always going to be cross-incentives between people who want free stuff and providers who have to pay money to provide you with stuff. Not everybody can be a wikipedia and raise money with pictures of Jimmy Wales. They are an outlier.
My conclusion is that these are browser problems. After all, it's none of my business what people put on the web, and aside from liking Google and wishing them well, I really don't have a dog in the fight for their struggle. In fact, it's better for me to have a dozen search companies all using different algorithms -- makes it harder to game the system.
So what I want is a browser. A browser that uses multiple search engines automatically and completely eliminates any "fluff" from rendered pages -- perhaps even combining various pages into much simpler displays.
I'd pay for that, and that would make me the customer. Then I would have whatever web experience I desired, instead of the one that I get for free. I'd much rather be in the position of writing a check to the best browser provider that condensed and filtered information than the situation we have now.
(By the way, if anybody is interested in this browser project, please contact me, as it's been a pet project of mine for some time)
Hold on a tic. You're complaining about clutter in the search results, but didn't you just say at http://www.whattofix.com/blog/archives/2011/01/confession-i-... that you make several sites like facebook-login-help.com to target specific phrases? So you're complaining about junk on the web, but you also own domains like buy-fresh-blue-cheese.info, and at the same time don't want such sites to clutter up search results? My head hurts.
I believe that there is no single metric for search results -- that there is no universal answer to the question of "iPad games"
So I try to provide original content in niches that don't exist.
I apologize if you don't like the quality of what I am doing. I will gladly delete the sites if they hurt people. I am simply trying to learn how to provide content that people want.
You are welcome to contact me offline and I'm happy to do whatever is necessary to prove that I mean well. I don't know what else I could offer to prove my good intentions.
And please do something about the StackOverflow problem. This is clearly an issue with Google, which is highly visible to ALL programmers. I couldn't imagine a worse problem to have for a company that is interested in hiring the best engineers.
I did ping the right people about SO and added a new example that someone mentioned. The right team (and their manager) are actively discussing what steps are doable to improve these searches.
I'm coming at this from the other side, so perhaps sometime we can compare notes. I'd like to learn.
I'm a writer -- I love writing. I have a blog that I've been writing on for several years now. I think I'd write even if nobody wanted to read what I wrote.
A year or so ago, it occurred to me that, instead of just writing whatever I feel like, perhaps I should try to write something that people want. So I use tools to see what people search for, and what kinds of content already exist.
I've written some really corny sites, as you point out. But I hope that each site gets a little better, and I know from the emails and return traffic I get to some of my sites that people are getting value from them. Some -- probably not. That's okay because I'm learning.
If I'm doing something wrong, please just let me know. I'm happy to change things. In this field we have the big companies, the scammers, and the folks somewhere in between. Most guys who make money off such sites probably aren't honest (stupid?) enough to blog about it. I know it's been very difficult for me to find guidelines on what works, what to do, and what not to do.
So I've created my own guidelines for now: no rewrites, no link-spam, no tricking the user, no link-bait. Perhaps I need more. Don't know. That's all I have so far.
Like I said, I'm not putting myself out there as an example of what to do or how to act. Beats me. This is a complicated problem and there are multiple valid points of view.
Hey -- the first thing I install on my browser is AdBlock. I hate all forms of advertising, on any medium. I suggest others do the same. That doesn't mean I have to give up writing about stuff and providing content. And it doesn't mean I can't put ads on the content I provide. Not crazy about a lot of stuff I participate in.
> I've created my own guidelines for now: no rewrites, no link-spam, no tricking the user, no link-bait. Perhaps I need more. Don't know. That's all I have so far.
"We strongly encourage you to pay very close attention to the 'Quality Guidelines,' which outline some of the illicit practices that may lead to a site being removed entirely from the Google index or otherwise penalized. "
The http://www.hamburger-casserole-recipes.com/ site gets a lot of traffic, and a lot of it is return traffic. But, since recipes are fairly standard, it was very difficult not to in some way re-fashion what was already out there.
When I was doing the research on the recipes site, I found a lot of what I consider spam -- sites full of recipes with the intention of tricking users into downloading spyware, buying SMS subscription services, and other such stuff. I felt like a simple, easy-to-read, targeted site based on just a few recipes people want might have value. It's not a general purpose recipe site, and the content is presented as simply and openly as possible. (And it was all entered by my wife)
It's interesting that it would stand out as being dubious, as it seems to me to be the most straightforward trade of content for eyeballs of the sites I have.
As an aside, it doesn't make much money, perhaps because nobody advertises about casseroles or cookbooks? Beats me. But I plan to keep it because of the positive response it gets, including people writing to ask for new recipes.
I didn't have a problem listing my other sites on my blog, since, well, they are my other sites. It's not like I'm trying to keep them secret or anything.
I don't see any contradiction, even if sites like 'facebook-login-help.com' are exactly the kind of clutter Markham would prefer did not rank at all.
One complaint is wishing that the search results were better. From the perspective of one person, that might or might not ever happen – one person outside Google has very little influence on overall search quality trends.
The other action is living in the world as it is. If cynically-optimized results are going to dominate the top spots, and being in the top spots is a way to make a living, even people who would prefer cynical-optimization not be rewarded have to engage in it. (I'm sure there are lots of good people creating awful content for Demand Media and other mills; they're just responding to bad incentives.)
It's the same for anyone who's had to twist their original web writing/design/structure in a way that was solely directed at search engines, rather than primary users. Yes, lots of SEO advice is making the web more usable for people, as well. But not all of it, axiomatically.
Note that the area StackOverflow is in has been a "bad neighborhood" for a long time... StackOverflow has played a big role in cleaning up.
About six months ago, just about any question about "doing X" with Windows or .NET would come up to some page on ExpertsExchange that would promise to give me the answer... if I paid. Yeah right.
It's no wonder the spammers in this space are fighting back against S.O.
> So what I want is a browser. A browser that uses multiple search engines automatically and completely eliminates any "fluff" from rendered pages -- perhaps even combining various pages into much simpler displays.
Aren't you describing a "metasearch engine"? They have been around for quite some time (which does not mean they could be much improved, of course).
Most of the good search engines have a TOS that disallows you from creating a meta search engine from their results (API or otherwise). And they WILL cut off your access if they catch you. Learned that the hard way.
The browser would solve the cross-purposes problem which exists not only in search but everywhere else.
So, for instance, Facebook provides you this cool platform for chatting, playing games, and keeping track of your friends. But, of course, it's not really free. You'll see ads, get pitched products by your friends, and all kinds of other stuff that has nothing to do with (perhaps) your primary purpose for being there.
Since this "Hey! We're free! (Except not really)" problem is going to continue to exist on the web in various formats and places, fixing the browser is the only way to keep control over your experience.
We're already seeing this type of work in developers who build add-ins to control the user experience. Hell, I'd pay for a subscription that kept my browser updated with all the appropriate plugins, but I think we can do a lot better than that. For instance, if I want to send a friend a message, I could care less whether they are on LinkedIn, FaceBook, MySpace, or just somebody from my address list.
The purpose of HTML is to separate the data from the presentation. It's the only way you can make the web work. But the implementation of html is full of walled gardens, addictive-play websites, and using your own friends against you. We need to get back to basics.
I use Google extensively for search. About once a month or so, I'll be looking for something in a bad neighborhood. It's not a pleasant experience. It's a shame to see tech questions end up like this.
But the problem, as another poster pointed out, is that nothing is for free. You are either paying money, in which case you are the customer, or you are the product. There's no "in-between" In Google's business model you are the product.
I think the business model can continue for a good, long time, but there is always going to be cross-incentives between people who want free stuff and providers who have to pay money to provide you with stuff. Not everybody can be a wikipedia and raise money with pictures of Jimmy Wales. They are an outlier.
My conclusion is that these are browser problems. After all, it's none of my business what people put on the web, and aside from liking Google and wishing them well, I really don't have a dog in the fight for their struggle. In fact, it's better for me to have a dozen search companies all using different algorithms -- makes it harder to game the system.
So what I want is a browser. A browser that uses multiple search engines automatically and completely eliminates any "fluff" from rendered pages -- perhaps even combining various pages into much simpler displays.
I'd pay for that, and that would make me the customer. Then I would have whatever web experience I desired, instead of the one that I get for free. I'd much rather be in the position of writing a check to the best browser provider that condensed and filtered information than the situation we have now.
(By the way, if anybody is interested in this browser project, please contact me, as it's been a pet project of mine for some time)