Hacker News new | past | comments | ask | show | jobs | submit login
How Google is Killing Organic Search (tutorspree.com)
535 points by akharris on July 1, 2013 | hide | past | favorite | 209 comments

The auction model for ads basically ruins internet searches in transactional categories. If you want to win at the "italian restaurant" search game, you have to bid the highest for the ad, which means you must have the highest margin. The best way to win is to open a restaurant with ridiculously high margins (over priced wine and cheap ingredients).

Want the "cheapest car insurance"? Google is zero help. It sends you to Geico or a bunch of lead-gen sites, and no matter what, you will end up at an insurance company that makes the biggest margins.

If you really want the cheapest car insurance, you need to find a company that doesn't advertise, and is non-profit, that way your premium is spent buying insurance, not TV ads and Berkshire Hathaway's stock appreciation.

Whenever I see a SERP full of ads, I search for something different. When there is nothing but ads, Google is sending you to high-margin crap.

Internet searches in many categories have always been useless. Searching for a restaurant or car insurance has never ever returned anything useful on any search engine, ever. it's not as though google used to have incredibly relevant results for a search term like "cheapest car insurance" and now it's been replaced by ads. The only thing a search like that has ever gotten you is the websites of the insurance companies who are well established brands.

The categories where google is pushing a lot of ads or supplementary features like knowledge graph are the categories where organic results are not a good experience. 9 times out of 10, when i search for something on google i get a traditional SERP with little to no advertising. If you're searching for something that doesn't necessarily lend itself to a web search, google should be trying to find some way to make that useful to somebody, whether that is a feature that gives you better results or an admission that their organic results for that query are useless, so here have some ads instead.

In regards to your first paragraph, you are completely right because the product has always not been the consumer of the search.

I dont if it would be possible for a subscription based search service to exist, but as long as they are selling to advertisers and not to the people there is very little incentive for this to change imo.

It's not quite like that.

What advertisers can afford to pay is margin X conversion rate. If people search for cheap socks and you have high margin socks, your conversion rate will be lower. In super competitive markets, the conversion rate is usually what makes one win. For something like insurance quotes, that probably means the one with a known brand, supporting advertising in tv & radio, etc.

The quality of the sites and the margins will probably be pretty similar among most players.

Where the dynamic you're describing happens is between product categories where you have a high margin per unit in one product category. Eg. If you want to sell a 'diy divorce kit with a $5 margin' you will have to outbid divorce lawyers who expect to earn thousands per client. The lawyers will drown everyone else out.

I don't know about the sock market, but there are lots of categories where the arms race for user revenue-maximization has either not yet started, or everyone is using very old techniques.

A better example would be someone selling "cheap" socks but with expensive shipping. Or, aggressively upselling additional sock products in the checkout, possibly with FTC-unfriendly opt-in vs opt-out methods. Or, hammering their users with advertising after the sale by email and phone.

When a particular vertical market gets aggressive, the item being advertised usually brings little, no, or negative revenue to the advertiser, and all of the money is made on the back.

> The lawyers will drown everyone else out.

The worst lawyers will drown everyone else out, including the good lawyers. The lawyer that has a high conversion rate by promising it will cost $5000, but ends up costing $30k drowns out the lawyer who legitimately quotes $10k.

I agree, for certain searches, the ads are helpful, but I stand by the notion of avoiding SERPs with tons of high value ads. The person paying for those ads will likely be you.

Yeah, from what I've seen fraudulent advertising, bait and switch, and various other shady tricks are dominant strategies in adwords advertising within some categories, and I haven't seen any obvious effort by Google to curb it. If there was some way for Google to act as a reliable Better Business Bureau replacement, that rating would be a great addition to their ranking algo.

I've been doing a lot of thinking about this topic recently, so it's nice to see a discussion on HN.

AdWords ad placement is determined by a mix of quality score and bid size. It's good that they're using quality of links to influence ad ranking, but here's what troubles me:

We're allowing money to influence information salience.

You could say this is how the world works, and I can't deny it. Money = power = eyeballs. But is this a good thing? Is this a dynamic we should be actively supporting? I honestly don't know.

Possible counterargument (because I can't help but play Devil's Advocate against myself):

Maybe the auction model is about determining the true "market price" of an ad position, in which case maybe this is a good thing?

Is there any hypothetical search model in which money doesn't influence information salience? High performers for competitive searches in organic search have usually invested heavily in SEO and/or brand-building too. Open reviews can be astroturfed and social likes can be astroturfed or incentivised. At least with "sponsored results" it's blatant, and declared.

Ironically, considering the original article starts off by complaining about its effect on search results, the most-free-from-financial-influence way of ranking information salience is probably the Zagat review model (selling aggregated secret shopper info)

I don't claim that advertising is the only vector of influence, and I realize it is possibly the least worst version (because of labeling), but that hardly seems like a reason to consider it a Good Thing.

Your argument seems to be that if we get rid of sponsored search results, this will push advertisers to invest more heavily in less visible forms of opinion manipulation. This is actually a very interesting viewpoint and possibly a valid argument, but it also feels like coercion in some way ("give us ad space or we'll reach you via less ethical methods").

I've been thinking about this too. IMHO, the only reason to advertise that is beneficial for both the consumer and the advertiser, is to provide information. Ie.: our product is objectively better than the competition for certain consumers, but the problem is, they don't know about us. So we create a ad campaign, that will be beneficial for both the consumers and our company.

The goal of connecting people with value-providing businesses is a noble one, but the problem is the difficulty in using advertisements as reliable indicators of a product's merit. Economists have written about the prominence of advertisements as signaling mechanisms, but the bottom line is that I would rather rely on customer reviews than ad placement when it comes to determining if I want to buy a product.

I absolutely agree, most of the time ads don't help consumers make better market decisions.

I think your hypothesis is quite far off. Can you give an example of an auto insurance company that is a non-profit? I couldn't find any. Perhaps you mean a mutual insurance company? Furthermore, when I bought car insurance last month, Geico gave me the most coverage for the least amount of money (after quite a bit of research), so I also don't agree with your assertation that Google is no help in that regard.

While I agree with the concept, as I work for a non-profit and belong to a credit union instead of a bank, I don't think your hypothesis holds without some data to back it up.

As someone who sells car insurance leads for a living, this is an interesting discussion.

First, Geicko isn't ranked number 1, affiliates who mask themselves as Geicko are. In high rewards markets like insurance (where 30% of the overall price of insurance goes to affiliates) it's actually about which affiliates convert the best and have the best payout. Those SERPs are dominated by black hat SEO. None of that is organic.

The larger issue (percentage of the page dedicated to SERPs vs ads aside) is that Google's formula worked best in a world where people weren't trying to game it. In the end it breaks down to relevance and links.

Relevance is based on tags and keywords, which everyone can get a perfect score on. So in the end it comes down to links.

Is a link there because someone just loves the site, or because some SEO schmuck built it? There are correlations, but there's really no way of knowing. The key to spam is mimicking a human process closely enough that a computer (or algorithm) can't tell the difference, then making that scale.

Google does manually review the SERPs for the highest grossing terms basically daily to try and find foul play, but those can be beat, too.

So why doesn't Google just manually find and serve up the best pages of content? Because you just described Yahoo.

So it's a very complex game that evolves every day as both parties advance and become more sophisticated, often in response to each other. If you think you can do better you either have to out-engineer Google or find a way to rank sites based on something other than links.

Could Google Analytics data be used as "something other than links"? They can track traffic, bounce rates, how long people stay, how often the same people return. Would that provide a better indicator of quality pages? It would only apply to sites that opt-in by using Google Analytics, but if it made them rank higher than everyone would do it. (How many don't already?)

Just because blackhat SEO ranked it doesn't mean it's not a page people will use or stay on. That would also require every site to have Google analytical installed. Google is a big company for sure, but not quite that ubiquitous.

Why use Google Analytics, when there is Google Chrome browser?

No Google already know too much about all of us.

This is hands down the best company with which I have ever done business. Their technology is amazingly far ahead of others in the banking and insurance space. I was taking a photo of a check with my smartphone to deposit it years before it became mainstream.

It is a great example of how constraints build value. By constraints I mean, they have had to deal with military customers stationed in far flung places for years. This built their capability to conduct business virtually.

I will second pretty much everything you just said. I have renter's and auto insurance, checking, savings and a credit card all through USAA and they are fantastic. Many banks are just beginning to catch up with their technology and none can match their customer service.

i like the business model of USAA, where it isn't a corporation, but that everyone who becomes a member gets some sort of benefit. its almost like voluntary communism. I wonder if this model would work for other sorts of institutions.

> Can you give an example of an auto insurance company that is a non-profit?

Privilege Underwriters Reciprocal Exchange (PURE) is not "non-profit" but theoretically "policy holder owned" with surpluses to the benefit of subscribers.

To your point about credit unions, insurance 'unions' are interesting:

Overview: http://www.pureinsurance.com/about/index

That said, there is a marketing arm that extracts value as fees for marketing and management, and a financial stakeholder umbrella seeking return on capital. It's an interesting structure:

Structure: http://www.pureinsurance.com/about/pure_group

In any case, my experience with PURE has been superior to my experience with other insurers (except Met-Life who was excellent but unable to competitively insure my property), and my costs have been 30% less than the most competitively priced usual players.

> Can you give an example of an auto insurance company that is a non-profit? I couldn't find any. Perhaps you mean a mutual insurance company?

Aren't mutual's non-profit? They aren't trying to extract a profit for investors or other owners, but just need to charge enough to pay business expenses (including salaries, as well as the main expense of an insurance company, paying out claims and funding reserves to pay out claims, etc.)

USAA mentioned in some other followups is... even weirder than a mutual insurance company, but it's legal structure is effectively the same as a mutual insurance company.

"Aren't mutual's non-profit?"

No, they distribute the profit to policyholders, who are effectively shareholders of the corporation. I belong to one, for auto and home. I get a dividend each year of 20 - 25% of the yearly premium, which is competitive even without considering the dividend. It's not open to the public.

> Can you give an example of an auto insurance company that is a non-profit?

AAA who do my insurance: http://en.wikipedia.org/wiki/American_Automobile_Association

What's funny is how different using Consumer Reports online experience is. What? There's no free content? I have to pay a monthly fee to read content and it's not trivial. Best investment I ever made. Especially when buying new appliances. Ha.. It's kind of pathetic that I flinched at spending a few bucks a month for good content.

If you're getting information for free your information likely has a cost you're just paying for it in an obfuscated way in the form of higher margins when you shop at a place that advertises heavily.

If you ever look at Hulbert's Financial digest, which tracks the total return of investment letters you'll see that the ones that do well are $120/year. Oh yeah, you have to pay for the digest which will tell you which newsletter performs the best and that isn't cheap either.

Of course free investment advice is available everywhere online and usually makes a good contra-indicator. If I see a hype filled story show up in the freebie investment news it usually means the stock is about to take a short term down turn.

If I see a hype filled story show up in the freebie investment news it usually means the stock is about to take a short term down turn.

Good time to short it.

This will be solved using big data, and it will be free not subscription. Working on it right now.

Best investment I ever made. Especially when buying new appliances. Ha.. It's kind of pathetic that I flinched at spending a few bucks a month for good content.

I have an entirely different perspective of Consumer Reports (the online variant which has external site ads on it).

Understand that they are entirely biased (as we all are): They take home the lucre by keeping you thinking that you need them, and that they're doing an important service. Hence why the big noise about things like not recommending the iPhone -- all just noise to keep the subscription dollars coming and to create a sense of utility. They're also prone to taking complex products and then trying to distill it into easily quantified values that they can put in a chart and categorize, leading to a detachment from real-world value and "need to fill this issue". I was a subscriber for several years and eventually came to realize that their top picks were usually not really top picks, and they were often in areas where they were entirely out of their depth (I discovered that particularly after buying a set of their top pick speakers).

After that print subscription ended, years later I was looking at getting a new furnace and saw that the online site listed furnace reviews, though they were hidden behind a paywall. Okay, bought a subscription to find that there were no reviews at all, but some useless summary about basic efficiency levels. Well that's useless. Oh well, a year subscription wasted. A year later I discovered that they auto-renewed (never signed up for that, but whatever), and put it on my todo list to jump through hoops to cancel at some point, wasting my time (as they give NO clear way to cancel, despite subscribing being a trivial painless step). The next year they sent me notice that my credit card had expired -- awesome, easy way to finally unsubscribe from an auto-renew that I didn't opt into without sitting on hold for half an hour! Imagine how funny I thought it was when they guessed my next expiry and just used that, continuing the subscription.

I have an extremely poor opinion of CU, and I think the belief that they're more pure or their opinion more useful is a bit naive.

>> Imagine how funny I thought it was when they guessed my next expiry and just used that, continuing the subscription.

They are not guessing the expiry date - I believe it is a service provided by some credit card companies, possibly to large merchants. It happened to me for several years with my Yahoo Email account - that would renew for $20 / year even though my card expired twice during that time. It stopped after I reported my card as lost/stolen - and got a replacement card, so the actual card number changed. Fastest - albeit a bit drastic - way to get out of a recurring billing if you are having trouble cancelling.

It's relatively easy to update the expiry date, since the month doesn't usually change. You just add two years, try the charge, and if it doesn't go through, wait a day and try it again with the next year. This is pretty standard for services with a recurring subscription model, and they don't really need to pay an outside provider for that info (although it would be trivial for any credit agency to provide.)

That said, I'm not sure he can complain that a recurring subscription continued to, well, recur.

Isn't that fraud though.

I'm not a lawyer (nor an expert on payments) but generally it doesn't seem to have any element of fraud - it's collecting payment for services fairly rendered via the credit card account provided by the customer. Fraud would involve some kind of intentional deception.

If their was fraud, you could get a chargeback, and I don't see that happening very often.

You provide payment using the card. The card expires and can't - supposedly - be used beyond the expiry date to make a payment. There is no expectation of continued payment beyond the expiry date.

Guessing the new expiry date in order to exact a payment is taking an unauthorised payment. It can't be authorised because the card-holder can't authorise payments beyond the expiry date.

Taking a payment that isn't authorised by the card-holder by guessing some of the details of a card against the expectations of the card-holder appears to be fraudulent to me.

No, you provided initial payment using the credit card account, but you signed up for the service until you cancel it, not until your method of payment expires.

Even if you cancelled the card outright, you're generally technically (and legally) still on the hook for a subscription if you don't go and cancel. Read the ToS for just about any recurring subscription. It's your obligation to cancel, not their obligation to guess when you want to cancel. Honestly, I'm not sure how that's unfair; it's not a service provider's business to interpret your payment methods and how they might map to your desire to cancel or retain a service.

The only reason most services don't bother to pursue payment for expired cards is because it's not economic to do so. So when a card expires, they eventually mark the account as overdue and disable it. That's not the same as a clean cancellation; technically you still owe them for the services rendered between the last invoice and the final cancellation.

>you signed up for the service until you cancel it //

Maybe, that would depend on the legal terms under which you signed up. Mostly you sign up and pay for a fixed term. So contractually the expectation is that you don't continue payment.

If legally the payment is due, then that's still doesn't mean you can have authorised it on a card beyond the expiry date. Moreover you haven't authorised the payment with the new card (in the circumstances we're considering).

So the payment may be due, but it's been fraudulently acquired. There are procedures for collecting overdue amounts. Of course the companies don't want to follow those procedures and air their unfair [attempted] retention practices in court.

Hardly anyone offers a subscription service without an automatic renewal of some type. It's counter to the entire business model, which is that you spend once to acquire a customer but then earn back multiple subscription cycles from them (as the first cycle almost never pays back the cost of acquisition.)


I'm guessing that their definition of "CREDIT CARD" is wide enough that they can roll the expiration date; when your card expiresd at Amazon you can just update the date without entering a "new card" - they consider the new physical card to be the same "payment method."

interesting - so there is a business opportunity to collect these overdue amounts in large numbers, and exact the payment along with a "fine"…

It's your obligation to cancel, not their obligation to guess when you want to cancel.

You have a remarkably apologetic attitude towards a scumbag industry.

Just to be clear, here's the modus operendi for the retention industry-

1) Trick or outright scam people into recurring subscriptions. I never told Consumer Reports I wanted my subscription to recur, and it provides absolutely zero value to me that they did so. If they hid text somewhere for this, it is a clear and obvious example that the industry desperately needs onerous rules and restrictions prohibiting this, such as a customer yearly opt in and double-positive recurrence opt-in (rather than some hidden subtext that oh by the way we have your credit card so have fun cancelling sucker).

2) Hide methods of cancelling said subscription behind as many layers as possibly: It is shocking how easy it is to sign up for many things versus what is needed to cancel the same (as if charges on my credit card are less of an issue for me than the possibility that online access might be temporarily delayed). In Consumer Reports case, you could not email or fill out an online form -- the single and only option was a limited hours telephone number that required lengthy wait times. They know that many people will avoid the time sucking tax. Xbox Live had a particularly hilarious system where not only did you have to phone a call center and sit on hold, your cancellation had to be immediate (despite the fact that all payments were for future services, when you disabled recurrence they immediately downgraded your account) which of course is meant to delay people from jumping through the hoops until the last day when they might forget or give up and boom, another month.

While much noise was raised about Apple shaking down the subscription industry on the iOS platform, they will have to simply eat that cost because people like me have absolutely zero trust in the industry, and I greatly respect how Apple not only constantly reminds me of subscriptions, they make cancelling any of them the easiest thing in the world, so those services that want subscribers need to actually work

Recurrence is a tax on the world. It is inefficient and obnoxious companies that turn their business model towards unsavoury tactics rather than actually providing value that keeps people from wanting to be their customer.

1) By reading this post you agree to subscribe to future posts at the economical cost of $14.49 per month, billed bi-annually.

What you're describing is the MO for dishonest subscription services. But there's lots of subscription services that aren't dishonest and have very happy customers.

To your points:

1. While there are dishonest sites, I'm sure you also have numerous recurring subscriptions with a whole bunch of services where that's exactly what you agreed to, and what you want. Your internet provider, your mobile phone provider, your cable company, etc. I realize you may have had an issue with one company, but that doesn't mean the subscription model can't be used in an honest, customer-focused way. I have subscriptions with utilities, online content services (Spotify), and even a podcast network and for 90% of situations, it works fine and there's no trick or scam.

2. I totally agree that companies should make it easy to cancel, and it's despicable if they don't. It's also generally bad business practice because it hurts your relationship with them. But again, there's lots of companies with recurring subscription models that handle this right: just look at Netflix. Not only will they cancel easily, they'll also put your account on hold if you're going on vacation, etc.

"only option was a limited hours telephone number that required lengthy wait times."

You don't need to play their game. Send them a letter cancelling and revoking the authorization to charge your credit card. When they charge you anyway, send a letter to your bank informing them of the unauthorized charge. You'll get the chargeback. Send another letter to the AG of your state, too, complaining about the credit card fraud.

An important point is that it isn't for services rendered -- it's for services to be rendered in the future on receipt of payment. At the point of billing I owed them $0, and had received no service from them. A credit card number is the compound of the number and expiry day, and they essentially fabricated a new expiry date.

There are whole industries built around billing people for services they don't actually want, and this is just an example of it. I completely understand that it is normal across the industry, but it is incredibly scummy and puts deception and evasion ahead of providing actual value.

This depends a lot on the particulars of the individual subscription agreement. But in most cases, you owe for the services once the new subscription period starts, not when payment is received. (In most cases, your card company actually pays long after the services are rendered (30-60 days), anyway.)

You're saying this is a service you don't want, but it's unclear how the provider is supposed to guess that. You signed up for it, you haven't cancelled it, and now they're going to provide it and then seek payment for it. The failure to cancel is on you, not the provider. Even if you cancelled the card outright, in most good ToS, they'd still have a right to bill you directly for the service if you don't cancel.

Let's say I run a gym and you're a signed up member. If a new month starts, and you've got an active account, even if your card expires I'm still going to bill you for the month unless you come and tell me you want to cancel your membership. The two things are unrelated.

That said, I'm not sure he can complain that a recurring subscription continued to, well, recur.

Yes, I will complain. Intrinsic in the notion that I'm not bothering to update billing information after numerous updates telling me that it will cancel if I don't is the conclusion that the service, and billings, will cease. Credit cards have expiry dates for a reason, and ultimately it is fraud that they effectively manufactured my info for me.

I'd check the wording of those updates. It's usually that your service may be interrupted if you fail to update your credit card when it expires. And usually, the terms of the site say that you're on the hook for the recurring charge even if your card fails to clear.

You're hoping that the natural expiration cycle of your physical card - not the account - will impact a transaction that doesn't actually involve that physical implement. I can see where you're coming from, but I don't know that you really have a right to expect the expiration of your physical card to impact an existing business relationship with a third party that involves recurring charges (that you have likely agreed to pay regardless of your card's validity, or credit limit.)

I've worked on these kinds of renewal systems for large sites and generally, users don't mind them because the reverse case - where 4% of the users lose access each month because their physical card expired - sucks for both the site and the end users who don't expect access to be cut off.

If you want to use credit card changes as a shortcut for properly cancelling your subscription, you probably need to actually change the card number.

Since you tried to cancel and it was difficult to find how, can't you issue a chargeback at that point?

I am happy to see other people finally pointing this one out.

I will refine your hypothesis a little.

a) It isn't companies with the highest margin who win, but the ones which are able and willing to devote the greatest percentage of revenue to performance driven advertising. If the company has a slim margin to begin with, they will never even be able to start in the "race."

b) Education is a better example than car insurance. While there are geographic variances to what ads are shown, nearly all of the ads will be for for-profit schools (many of whom spend more on acquiring a single enrollment than on educating them.) Additionally, the organic results will either be for performance marketers driving leads right back to these schools, or it will be objective journalism plastered with banner ads and contextual text ads for for-profit schools (Forbes, Washington Post (who owns and receives half of their revenue from Kaplan but thats another story..))

c) Some markets get so abusive Google puts very heavy handed and aggressive rules in place. For example, if you read the FTC's news releases regularly a common scam is someone operating as a payday loan operator simply emptying bank accounts (not sure why this isn't an FBI/DOJ issue.) Its easy to bid high when your end result is the entire content's of your customer bank account. If you are operating in a market and your competitors start doing this stuff -- your fucked. Google can take years to catch up.

d) The "quality score" means little. It largely has to do with how a site looks, not what it actually does. Misleading ads tend promise more and thus get better click through rate which means they are even more likely to show up higher, not less. At the end of the day, Google will let low quality site run, they just charge them a little more.

e) The most disturbing thing to me is how Google is trying to wash away organic altogether. The original article points this out. Aaron Wall (SEOBook) has been pointing this out for year. Ben Edelman has been doing an excellent job of busting Google's bullshit PR (even though he may be paid by Microsoft for consulting its academic level work of the highest quality.) Google wants to keep serving that organic content to users, but block them from flowing through to the original source. Google Images' recent changes reflect this exactly.

f) All of this makes me think Google's original business model is in serious crisis. Ironically, Google is doing the same revenue escalation to their own users as their advertisers are forced to employ. If desktop revenue is dropping fast, Google has to crank up the "aggressive" meter to milk out more revenue from a diminishing ad clicker user base.

g) This is great news for the next search companies. Google is dependent on their ad revenue to fuel their behemoth enterprise. They are sacrificing their own product in an escalating fashion. (If you use AdBlockPlus you probably never even noticed..)

It is worth pointing out that Google is internally very aware of this situation. They have attempted to do what is best for users on several high value searches such as "Mortgage Refinance". At one point they had a fixed fee comparison widget, that would actually suggest banks with the lowest rates.

Unfortunately all these attempts were killed, because it is so much more valuable to sell an ad to a shady comparison site, than to run a transparent one.

I also don't think their business model is in any sort of crisis. Google gives people great search results consistently. It takes crazy logic to see good Google search results consistently everyday, and then not trust Google's results for a high value query.

My assumption & opinion is they know exactly what they are doing. I assume every decision has firm logic & metrics to back it up. However, the very nature of the marketplace is for advertisers to squeeze as much value out of the customer as possible in order to purchase higher visibility. I don't see it as a mistake or a flaw, it is just the end reality of a set of circumstances.

What makes Google's marketplace uniquely challenging is the following: In the physical world, you can go to a store and make a pretty good judgement that the product being sold is real, it is high quality, the cashiers are happy, the (real) customers tell you they have shopped there for years.

Online, everything becomes about human perception of data. It is easier to represent quality when there is none. It is easier to sell a customer one thing and extract a great deal of additional value from them. When that happens, the winner is the one with the most visibility (I think Americans can greatly relate this to US elections.)

Are user reviews and testimonials the solution? Amazon is loaded with fake ones. Many companies exist already to help small business owners "tweak" Yelp reviews. Even when you put a real face behind a review, it can be gamed. A static review will always be quite different than judging something with all of your senses in person.

Load your own site or app up with ads and see what happens to user retention. Prediction based on my own experience: it tanks.

Google is in crises because their behavior tells me their revenue would drop immensely if the result pages had as many ads on them as they did 6 years ago. Google is in crises because the ad revenue is their life blood. They have a massive organization which must have this revenue stream to remain in business.

There is a segment of the market that will click on and respond to bad ads over and over and over again. There is another segment that is more perceptive. There is a whole other segment that doesn't even know any of this has happened because they have used ad blockers for years!

Google might be really good at avoiding the tipping point. I assume that right now they are balancing the flight to mobile with the drop in user retention from "too many ads." Or perhaps I am overestimating the impact of ads, and the real game for Google is making customers think they are just clicking on content.

Some verticals pillage their customers. Many verticals do not, the arms race has not happened for them yet. That is a good thing for Google, and an even better thing for Google's users.

Google can control what they do, however with every step they take in the "show more ads" direction, the opportunity for someone else gets that much bigger.

As pointed out in the original article, in some result pages 90% of above the fold is ads. Would you watch Game of Thrones if it was 54 minutes of ads and 6 minutes of content? I wouldn't.

I wonder when (if at all) the pendulum swings back and users actively avoid clicking on the ads so much that makes Google scales back the number of ads shown. It's essentially a race to bottom and no matter how big a margins the current winners can make, it's gotta get thinner and thinner everyday.

I've read in certain places that most users are not aware that the top few links are ads.

With the new trend of usage of laptops increasing, a little shift of certain LCD screen on laptops changes the contrast of the advert background (very light pink) so much that it is impossible to determine that the first few links are ads.

I think people will continue clicking on ad links without realizing that they are doing so.

for some keywords search ads, are also a free service to users if you make a Copernican turn in your perception. the problem for end users for ads is the auction mechanism. it may seem that from quarter to quarter GOOG has to report better earnings and more ad placements seem to be an easy way to achieve increased revenue, albeit jeopardizing their main organic search experience.

at my zombie startup, "Vadlue" (http://www.vadlue.com/node/en/contact), we tried to strip ads out of search results and them searchable, browseable etc. maybe future lies in not making ads look more annoying by mixing them inside content, but making 'some content' behave like ads.

" … maybe future lies in not making ads look more annoying by mixing them inside content, but making 'some content' behave like ads."

And so online advertising re-invents traditional media's "product placement".

To paraphrase a comment upthread "Would you watch Game of Thrones, if The Freys had used Smith & Wesson branded crossbows to massacre the Starks? Or if The Lanisters only used AT&T brand carrier pigeons?"

Google is in crisis because they need 2-3 times more ads than before to keep earnings growth. At some point everything will be ads, even page 2.

Also many users do not know they are seeing ads, imagine if FTC cracked down on shady labeling? People do not like seeing that many ads and Google's revenue and reputation would go down the drain.

>> It takes crazy logic to see good Google search results consistently everyday, and then not trust Google's results for a high value query.

Why, maybe Google manipulates only the high earning keywords? There is no money on "Snowden leaks"

Business owners should band and push another search engine. Google has some guy name Mat Cutts offering "do this and that and your ranking will improve from [update a-z]" but apparently these are lies by Google, Google is actively taking away traffic to send it to ads. Some site that paid $50 for a click, will have to get that money back, and you, the user, will pay for it.

They are cracking down. Just a week or two ago a new statement from the FTC came out with new ad guidelines specifficly targeting search engines.


"many [for-profit colleges] spend more on acquiring a single enrollment than on educating them."

According to the DoE, 4-year for-profit colleges spend 2.9x more on administrative expenses (albeit including student services and academic support) than instruction. Unfortunately I can't find a better breakdown of this category, but I'd assume this is mostly leadgen.

Do you have a comparable number for non-profit colleges? While I doubt it's 2.9, I'm sure it's still high.

Non-profit 4-year colleges actually spend more on instruction than on those other categories, by 1.1x.

yeah, I'm going to have to ask you for a citation there.

It's interesting to note that across 165 comments, only 2-3 have even touched upon Quality Score let alone attempted to discuss it in detail.

Based upon personal experience (worked at a company that spent 8 digits on Google Ads on an annualized basis, profitably and successfully over a long period of time) I can assure you that Quality Score does NOT mean very little.

It is also a falsehood to state that Quality Score is entirely about "how a site looks". Google actually even measures how quickly users come back to Google after clicking on an Ad - and if users come back too often in too short a period of time (signal for poor quality ad landing page), the advertiser's Quality Score can be penalized (to what extent, exactly, is of course unclear).

"g) This is great news for the next search companies. Google is dependent on their ad revenue to fuel their behemoth enterprise. They are sacrificing their own product in an escalating fashion. (If you use AdBlockPlus you probably never even noticed..)"

This. I sometimes feel like I'm using a different internet than most users.

True that car insurance ads deliver little or no value - as in the end you always end up at one of the major carriers - and the price is no lower than if you had gone to them directly. The restaurant example assumes that you can continue to fool your customers with overpriced wine and low quality food. If you could stay in business long enough with that strategy you don't really need to run PPC ads! But these two examples completely distort the way Google's Ads are used by the vast majority of businesses - to whom they can and do deliver real customers and value. This includes the vast majority of cases - the long tail of search traffic that converts well for niche products and services that have little or no other way to directly market or get customers. When used correctly, Google Adwords is a godsend for such businesses. This could be a maker of high-end custom furniture cabinets in a little town in the middle of nowhere, or a small software company in a small town in India that has produced a killer app to learn Yoga, or a software developer in New Zealand, that has developed the perfect SaaS app for monitoring your server, and so on. These are the businesses that Google empowers - by giving them access to a potential worldwide market for their products.

find a company that doesn't advertise, and is non-profit, that way your premium is spent buying insurance

Does economy of scale not apply to insurance?

I've long ago taken this approach to all ads myself. Whenever I'm forced to see an ad (because I block or skip ads as much as I possibly can) then I make a mental note not to buy from that company. Whenever I go to buy something, I always go for a brand I've never seen in an ad. I find that this on average saves me money.

A thousand times this; when I tell people I prefer buying from companies who aren't blowing my money on superbowl ads, they just don't get it. The ads aren't for you, the existing customer - they're to get new customers. There's a reason I like Vanguard and USAA.

So this comment really resonated with me because I am competing in a space like this. I have a better product but a better product means making substantially less money.

Do you have any ideas on how to better compete without the same type of margins?

You are 100% correct! I've thought this for years, but gave up trying to convince friends. It seems like people like paying more for cute advertising?

The best way to win is to open a restaurant with ridiculously high margins (over priced wine and cheap ingredients).

That's a rather terrible example: a restaurant that had over priced wine and cheap ingredients would fail miserably in short order, regardless of how many Google ad auctions they won.

If you really want the cheapest car insurance, you need to find a company that doesn't advertise, and is non-profit

I doubt those two traits have any measurable correlation with providing the cheapest car insurance, or any other service or business, just as it's possible that a company with rampant ads on every medium and high margins provides the best value.

> a restaurant that had over priced wine and cheap ingredients would fail miserably in short order, regardless of how many Google ad auctions they won.

unless if there's so many restraurants, and the quality of wine is so subjective that most customers cannot really differentiate it. Then exposure is really the only thing a (time poor) customer can rely on, and thus, by massively spending on advertising, and not on ingredients, they can convert more paying customers than the small, but quality restaurant who nobody knows exists.

I want some way to have word of mouth recommendations work over the net, without it being game-able. perhaps a system where you must pay (a very small token amount) to review?

>I want some way to have word of mouth recommendations work over the net, without it being game-able. perhaps a system where you must pay (a very small token amount) to review?

That would make it even more of a money arms race. Ultimately you want something like personal recommendations where people attach their own reputations to their recommendations; google is trying to push something on these lines with google+.

but a person's reputation on the net is inherently non-existant, unless they're forced to use their real name, and is somewhat known/famous.

Being able to create fake reviews and fake recommendations is the same problem as spam in email (cept spam is easier to filter out). If sending email costs the sender some token amount (say, 1cent per 1000 email), it would probably stop majority of spam, because it just became cost-ineffective. That was why i was suggesting paying token amount for writing reviews - tho i have no doubts there are flaws with that idea….

> That's a rather terrible example: a restaurant that had over priced wine and cheap ingredients would fail miserably in short order

Have you ever travelled? There are restaurants like that surrounding every airport, convention center, etc. because they know people will value convenience, speed of service, ability to handle large parties, etc. more than great food. The situation is similar near large office parks — it may be mediocre but it's the closest choice! — areas with lots of tourists or transient populations (how many college town date-night restaurants are great?), etc.

Actually, it's even simpler: Applebee's does not appear to be disappearing.

there is no such things as "internet searches". Because all search engines behave differently and nobody can tell for sure how panda works , except people working on panda. So the rules are decides by whoever develops the search engine. Like there is no such thing as SEO , Google has a monopoly and the rule is pay to play.

Nit: this article is really criticizing a UI change which makes local results more prominent (in that they take over the entire page). The local results at the top of the page when you search for "Italian restaurants" are all still organic. In fact, whenever there's a map on the right-and side of the page, it's displacing ads.

Now, you can be against Google ever making a UI change, or even against Google showing different results for "Italian restaurants" in New York and Chicago, but this doesn't really have anything to do with "organic" search results.

You may have missed this part of the article:

The scariest part of this is that, if you sell something using the internet, regardless of whether or not you see yourself as a “local" business - or think you’re competing with Google - Google sees you as competition. Searching for “Camera" or “Buy a Dress Shirt" gets you a nearly identical split of screen real estate as that of “local" searches.

The issue there is that the "organic" results you're talking about that show in either the map or the carousel aren't organic at all - they're google local listings. As a restaurant, you are now incentivized to a) buy ads to get traffic from the ad units b) give google data (effectively building a whole other website) through google+/local.

You are very much being de-incentivized from investing in a page that is outside of google's universe, because you're only shot there is pure organic, which matters less and less.

I find this interesting. I didn't think of Google local listing as not organic but I did the search in the article[1] and sure enough it takes a lot of the real estate. Perhaps, Google could publish a standard way to write your own self-hosted Places page? We will still have questions on how the ranking will work and whether Google will prioritize its own Google+ place pages over self-hosted place pages but I think this would be a start. What do you think?

[1] http://i.imgur.com/Bf74Dd7.png

Well, thereby hangs a tale. But the short answer is that, by and large, Google seems to be aggressively uninterested in scraping that kind of structured/semantic data from the public Web. In fact it even seems to have been active in trying to prevent that kind of data from finding its way into webpages at all: remember the arcane but nasty scrap about metadata in HTML5? Well, this is what that was all about.

Scraping/doing it semantically has copyright implications (Re: Europe) so realistically the Google-owned approach is the most viable one.

Does Europe think that the robots.txt approach is cumbersome for "content" owners? If someone doesn't want anyone to crawl, they'd put something like

User-agent: * Disallow: /

First, why do you think Google Maps doesn't crawl the web? Second, why do you think that every business either has a web page or wants a web page?

Google local listings come from a wide range of sources. It's basically the phone book.

> Nit: this article is really criticizing a UI change which makes local results more prominent [...] The local results at the top of the page when you search for "Italian restaurants" are all still organic.

I have yet to experience the Google carousel for myself, but aren't these results all coming from Zagat or G+? So these may not be paid (or pay-for-prominence) results, but they're not the results of a Web search; instead they're the results of a search of a proprietary, Google-controlled database. That's not just giving local results more prominence, that's moving a large part of the page to a whole different kind of search: from Web-crawling to searching a Google-owned and -"curated" database. And in many ways that's more important than whether the results are 'organic' in the narrow sense of 'not play-to-play'.

I'm pretty sure Google Maps predates both Google+ and the Zagat acquisition by a large margin.

Of course it does, but what's your point here? I said that the carousel is space given over to results from non-Web search, not just space given over to local results. The fact that there was already a chunk of the results page given over to non-Web search results doesn't alter the fact that the carousel is a new, further large chunk of space handed over to non-Web search results.

The point is that calling Google Maps anything but organic is wrong by any useful definition of organic.

It's fine to argue about whether Google should be showing anything but ten blue links (maps, images, calculator results...) But refusing to acknowledge the legitimacy of searching anything but web pages is just absurd.

In the search market, ANYTHING related to UI impacts clicks and views -- ever since it was repeatedly shown in study after study that after the top 2 results in the SERP click rates drop drastically.

It's not that organic search results are changing, but their VIEWABILITY, and therefore relative importance, is drastically altered

This is true of course, Google doesn't make any money on organic search, nobody does, so it gets the short shrift.

At Blekko we built a search engine (crawler, extractor, ranker, fetcher, the whole stack) and let people know exactly how we rank things, for users who created an account and logged in they could turn off everything except organic results. We have been moderately successful (we were the first to use a 'masked results' option [1] to show how much better our results were than Google's in contested searches) but there isn't enough outrage about that yet to build a business yet.

Most of our business (people who pay us money) comes from folks who either want to figure out how to game Google's results, or want some organic results to create a 'search like experience' much as Google does. When you think about it with only 7% of the page dedicated to organic results that is like 2 or maybe 3 actual search results and the rest of the page is a carefully crafted advertising vehicle. Sort of like a free 'newspaper' which has one article of editorial content and the rest are all ads. You can serve that market quite effectively with a relatively small index (a couple of billion URLs).

If you look at Google's financial performance over the years you can see how this evolution has affected their bottom line. Today you see it in declining revenue per click sorts of things. It feels to me as it did when banner ads went from this massive cash cow into something less useful.

One thing is true though, the world is changing yet again.

[1] http://techcrunch.com/2011/06/21/be-the-mark-in-blekkos-3-en...

Hi Chuck,

I'm glad that folks like Blekko are trying to improve the search landscape but do you have any figures to back up the declining revenue per click?

I feel as though Google are becoming more and more focussed on profits (evidenced by their (not provided) data hypocrisy, role out of "enhanced campaigns, etc.) but their profits seem to grow[1] and they seem to focus on margins over advertiser or user experience with increasing focus.

I'm glad that folks like yourselves are in the market as I say but until someone develops better algos and better results (for the majority of queries) it's unlikely that folks will get angry enough to move or see a viable alternative.

In spite of these changes I haven't really seen a decline in Google's market share[2] drop consistently or significantly so I don't really see a change coming any time soon.

References: [1] http://www.blogcdn.com/www.engadget.com/media/2013/01/1-22-2...

[2] http://searchenginewatch.com/article/2275863/Google-Bing-Bot... (June, '13) "Google has the identical search share of 66.7 percent when comparing May 2012 and May 2013."


http://searchenginewatch.com/article/2269591/Googles-Search-... (May, '13)

Read Google's financial releases if you want to see evidence of their declining revenue per click, of follow the news [1][2][3]. Its a combination of factors, the market is maturing, there aren't too many people left who don't advertise on the Internet, and other sources are getting more market share (Bing, Facebook, Etc.)

  > I'm glad that folks like yourselves are in the 
  > market as I say but until someone develops better
  > algos and better results (for the majority of 
  > queries) it's unlikely that folks will get 
  > angry enough to move or see a viable alternative.
Actually one of the things Blekko has taught me is that search market share is no longer about search quality. At Blekko we demonstrated that we could provide better results in important queries and that how we did that would scale as we got larger. But we didn't get a lot of traction. Microsoft however (who we talked to) were so impressed that they essentially ripped off everything we were doing and put Bing on that path.

Now on the one hand I should be outraged Microsoft ripped us off [4], but I'm not. Its been interesting to see how much time and money they have spent making our strategy into a 'full size' search engine, and realizing that we were not going to be able to raise close to a billion dollars (which is what they will have easily spent by now) to take it to market against Google. The Bing challenge, the editors picks, the whole thing taken right from our playbook. And while I believe they are having the best success they ever have against Google to date (its a good strategy), there is another 3 years for it to run before consumers think that they are equal services. And there is a whole lot of dirty politics between here and there.

As for market share, its a curious thing. You can earn it or you can force it. Look for more and more "search defense" going on, for example non-Google searches getting harder on Android, or in Chrome. Conversely non-Microsoft searches in Windows products. With people like Firefox getting played in the middle (try to search on Google on Firefox on the standard Linux Mint distribution for example :-) Gone are the days where simply having a better product will let you win that prize. Way too much money involved.

[1] http://www.marketplace.org/topics/tech/weak-ad-revenue-pulls...

[2] http://news.slashdot.org/story/12/07/24/1553214/the-decline-...

[3] http://www.forbes.com/sites/roberthof/2013/04/18/google-tops...

[4] It would have been polite of them to have at least offered to buy us out :-)

Maybe this is a misconception on my part, but isn't local search listings and map info a form of organic search, just derived using different signals for authoritativeness and relevancy, of which proximity is only one variable?

If the above is true, Google is dedicating way more real estate to organic than ever. It's just a different form of organic.

I came here to say the same thing. But one of the other comments [1] points out how the "local" listings (Google-owned Zagat) or maps are actually Google's curated business listings, not the business' website. Same goes for Google shopping, Google books, etc, where in popular categories, Google slurps up all the content.

On the one hand, that does level the playing field, because little guy doesn't need huge website and SEO to compete against big guy. But fundamentally, everyone is now handing over the control of their data to Google. Let's say you change your business hours (and as a potential customer, I do love Google's display of today's business hours) or phone number, how quickly does Google update all that? Probably very quickly, but it's now out of your hands.

And as that comment points out, positioning within those "local" results may be determined by ads (and they are not divided into paid and organic sections)--but we don't always know. Google is definitely trying to blur the line by moving away from the be basic search results listing.

I wanted to add that the problem of organic search results is always going to exist for any sort of business (as in both examples of the original article)--as opposed to information. Let's say I'm searching instead for a performance of Beethoven's 9th. Hopefully, people around the internet are linking to good performances, and Google will show me the "best" one. And it will have ads for ringtones and buying CDs of Beethoven's 9th, which is to be expected. But I can be fairly confident the organic part of the results are meaningful. With local search or business search, there is no such thing--Google is not indexing the real world, just what people put on the web. And when there is money on the line, what people put on the web is designed to make them money (SEO, ads, paid reviews, etc.).

[1] akharris: https://news.ycombinator.com/item?id=5972516

> Let's say you change your business hours (and as a potential customer, I do love Google's display of today's business hours) or phone number, how quickly does Google update all that? Probably very quickly, but it's now out of your hands.

On the contrary, its completely in your hands: http://www.google.com/business/placesforbusiness/

The point of contention is that its a Google controlled affair instead of going through an external standard.

I know several small businesses (hairdressers etc) that have a wrong phone number or other data in Google's "zero click" information. One even has the number of another hairdresser by the same name.

Now while it's nice that people can in theory update their business information using that site (that I myself had never heard of before either), but the reality of the matter is that, even in the 21st century, the modal local business owner just isn't very good with computers and the Internet.

The difference?

The website they get from some (probably also local) webdev company comes with customer support. Google has none.

Wrong or outdated information? One phone-call with a human being and it's fixed.

Try that with Google. They'll be talking to a robot and filling out form after form after form, in no-time. Believe it or not, but "figuring out this Google stuff" probably takes them at least a whole afternoon.

This is absurd.

First, the author factored the screen space taken up by the google search bar into his calculations. Really? You want to press the back button each time you want a new search? And don't get me started on the fact that the author factored in the goddamn page margins as part of his percentage.

Secondly: the 7% of the page that is taken up by a map? Yeah, those are also organic results. In fact, they are probably more helpful to most people than the author's version of organic results (just links to websites).

Thirdly: after you scroll down (takes about 0.1 seconds) the percentages change significantly, as the entire box of sponsored results is no longer there.

So. If we eliminate the search bar and the margins, and we include the map in our percentage, and we scroll down a tad so that the sponsored ads disappear, you're looking at a page that is about 70% organic results.

FINALLY AND MOST IMPORTANTLY. Google did not become popular because it showed the highest PERCENTAGE of organic search results per page. It won the search wars because IT'S RESULTS ARE MORE RELEVANT THAN OTHER SEARCH ENGINES. Percentages DO NOT affect the relevance of google's results. It's all about relevance.

DuckDuckGo is far less cluttered, which makes sense considering they have less things to push aside from the results: http://i.imgur.com/OnBJHkS.png

In light of "designy" things like this, and other "ideological" things that have transpired in the past few weeks, I've been forcing myself to use DuckDuckGo.

It's been about a month now, and I find the results terrible. I want to love it, but the first 10 results are pretty much never what I want. Sadly, Google still performs better on this front for me. If anyone has tips to make it better, I'd be very happy to hear them.

I believe it might be because DDG is not Google. What works on google might not work the same in DDG.

How long have you been using google? Probably long time. During this time you (unconsiously) learned how to ask questions "google way", now you need to learn to quack :p

I also found that DDG sometimes (not very often) does not find what I am looking for. Mostly it is because I know nothing of the subject ie. using a description of a "thing" I am looking for instead of its name, etc. - theese are the things google is better at.

Regardless, if you find such query than please submit it to ddg, they are very responsive and results do get better.

>How long have you been using google? Probably long time. During this time you (unconsiously) learned how to ask questions "google way", now you need to learn to quack :p

I understand this, but most of us are looking for an alternative to Google and not a completely different search engine.

Google with Adblock does the trick.

"Out of the box" I don't find that to be true, at least not with Linux 32 bit Firefox 21.0 and Chrome 27.0.1453.93 just now. Same results for auto mechanic and Italian restaurant, except my local market is too small for anyone to have bought ads for the latter.

I assume I could configure Adblock Plus to hide the Google ads, but at least for me and what I normally search for they're not really an issue.

Take a look at StartPage (http://startpage.com) - similar Privacy Policy.

I did, but it still includes local results (it thinks I am in New York City). I signed up for a Google API account and tried the Google custom search engine and searched for "pizza restaurants" and received different results than with Startpage. Startpage tried to sell me NYC pizza.

Google Custom Search: http://i.imgur.com/PLZ79Vk.png?1

Startpage: http://i.imgur.com/5ui5LQc.png

Bing is a little bit better in terms of screen real estate. They seem to push the top organic results at the top of the page, and put local results in between the top 2-3 and the next 6 results.

There are barely any ads.

Also, after a year of non-stop use, i can tell you right now that Bing predicts what I'm looking for far better than Google ever did.

Are we using the same bing? The searches I just tested returned just as many bing-centric results as google does.

I've found DDG to be better at finding obscure content, and the Goog to be better at finding what I call "canonical" content.

For example, if I search for "binary tree" I probably want Wikipedia followed maybe by StackExchange. Google does that, while DDG sends me deep into the netherworld.

One awesome thing about DDG is their !bang searches. It won't automate an unknown search, but if you know the first thing you're going to want is the Wikipedia entry you just search ["binary tree" !w]; for related StackExchange questions ["binary tree" !so]; you can even do a straight goggle search ["binary tree" !g]. For the available !bangs check out - https://duckduckgo.com/bang.html

You don't need DDG for that, when you can use Firefox (or Chrome).

All you need to do on a website with a search box is to right-click on that search box -> Add Keyword for this Search... and then add a shortcut.

The cool thing about it is that it is extensible. You don't need to settle for the fixed "bangs" that DDG provides. For example, I have a "dex" shortcut for searching an online dictionary of my native language.

Also, it takes less time to add a shortcut of interest than it is to read the DDG documentation page to find out the right bang for you.

If you care about privacy, then this is also a no-brainer, since you don't go through a third-party server anymore.

If I already know where I'm going, why involve a third-party?

I have my Firefox url bar set to DDG, so it's way faster for me to:

1. type ["binary tree" !w] in the url bar

than it is to:

1. type [wikipedia] in the url bar (I don't even know if it's a com or org, but DDG would be fine with just [wikipedia]. If I knew whether it was com or org, then I could skip the next step.)

2. click the wikipedia link in the now displayed DDG results

3. find the field to enter the site-specific search terms (for many -- but not all -- it's in the upper right corner)

4. type ["binary tree"] in the site-specific search box

In Chrome, that's:

1. Start to type wikipedia 2. Hit tab as soon as wikipedia.org is the first choice 3. Start typing the query and note that the suggestions are coming from Wikipedia, not google

Significantly faster than waiting several seconds for DDG to redirect


For me:

1) Wikipedia (in featured box), 2,3) Some Exchange/Notes migration software called "Binary Tree" that owns binarytree.com, 4) cslibrary.stanford.edu on binary trees w/exercises, 5) "Binary Trees in C++" from cprogramming.com, 6) "Binary Tree" at dictionary.reference.com, and 7) "Binary Tree" on mathworld.wolfram.com

...above the fold.

You've got to admit that 2 and 3 are pretty glaringly bad in this case. My fold ends halfway through 5. Half of the top 4 links are for the data structure, half for some irrelevant company, so DDG is effectively placing this company on par with the data structure. If you didn't know quite what you were looking for in advance, this could lead to serious confusion.


1) en.wikipedia.org/wiki/Binary_tree

2) binarytree.com - Binary Tree: Exchange Migration and Notes Migration Software

Yeah, sadly that happens sometimes. I revert to Google (with bang syntax) when I am looking for something programming related and want a lot of results from stackoverflow. However, DDG is definitely better at finding some things. I recently searched for gentoo+postgresql related stuff and got highly relevant results, which weren't in Google's SERP.

If you want just stackoverflow results and are already using bang syntax to get them from Google, then you should just switch from !g to !so.

An additional bonus of "!so" is that works with stack overflow search syntax like tag names between brackets.

I've been using DDG for about 1 year now, and I haven't missed google at all. The few times I don't find what I want, I !g to see google results, but they haven't seemed any better.

I find as more subject specific sites have grown, that generic search like ddg and google aren't as important to me.

I do more searches on sites like stackoverflow, yelp, maps.google.com, tripadivsor, etc. than ddg.

I've been doing the Duck about that long now, and I miss google enough that !g gets used daily. But, I also try to do all of my initial searches, even !g, !so, !w, !gi, and !a through DDG. And, I've set my Firefox address bar to default to DDG.

Try startpage


It's basically google, but without the tracking. (Or so they say.)

Not exactly a fair comparison considering you are using some sort of software to block the ad I get on that search. I only get three search results on my screen, if you consider the Wikipedia info and ad to not be organic, very similar to Google search.

And frankly, those search results are useless. If I am searching for auto mechanic, it is highly unlikely I am doing so to know the definition of an auto mechanic. Google may be creepy in its tracking, but at least by tracking me they can point me to a mechanic in my area.

> Google may be creepy in its tracking, but at least by tracking me they can point me to a mechanic in my area.

Surely you mean "at least by tracking me they can point me to a mechanic in my area without having to go through the trouble of specifying the area in the search box".

You almost make it sound as if Google is providing you service that is worth being tracked over. They're not. It's perfectly possible to find what you're looking for without the search engine knowing who or where you are.

The only thing you get in exchange for being tracked is the "convenience" of not having to type "New York" after your query.

I've really tried to make a go of getting out of the google verse. I set DuckDuckGo as my default but it was interfering with my ability to work, when searching for code. I'm using Bing as my default but occasionally I still use google for searching some snippets.

I've been using StartPage.com which another HNer had recommended. I like it better than DDG because it seems faster and it also uses Google's search results instead of Bing's.

I turned on my television and none of the programs were organic. Instead all the content was controlled by my cable provider. I'm beginning to suspect that they just put on those shows to make money off the advertising.

It's rather simple.

When you type terms into the search box, just imagine the words, "Please show me advertisements for " in front of it.

of course, but once you get to a point 50 minutes out of every hour is a commercial, and only 10 minutes is an actual TV show... you start to see a breakdown in the whole business model. Perhaps Google has reached the point where they will soon start experiencing a breakdown that forces them to show more 'content' and less 'ads'. However since the general public cant easily tell what links are paid and what are true searches anymore (and perhaps they dont really even care), maybe this business model will go on for a long time.

That has always puzzled me - why does pay TV have ads within the programs. I can understand outside the programs to round out to half-hour slots, but handing over money to get shows with ads inside them? I just don't understand why people go for these things.

Cable TV didn't have that many ads when it started out. They increased the amount as it became more stablished and more people started beeing invested in it (at least thats the impression I get in my country).

The organic search results are the loss-leader.

I'm typically a Google fanboy, but spot on. Here's a search for sushi for me: https://www.dropbox.com/s/q3epmld2ffqzg3f/Screen%20Shot%2020...

It's like full circle back to Yahoo.

Maybe this is a result of listening to the data too much, and not depending on the greater vision enough. Or maybe it's the future and you're gonna love it.

Maybe I'm the weirdo, but I think that's an extremely useful results page. What are you looking for when you search for "sushi" besides (more likely) nearby sushi restaurants or (less likely) information about what sushi is. The screen real estate seems to reflect those two possibilities in what seems to me to be a sensible proportion.

Then we're both weirdos. These results (both OP and child thread's) look extremely helpful, both to someone looking to eat some sushi nearby or wondering what sushi is exactly. Local places, definitions, pictures, how-to videos, all kinds of stuff someone searching for "sushi" might want. I really don't understand the complaint. Is it that Google no longer returns a page of text links indexed solely by pagerank?

It used to be, when you clicked a link on Google, you left Google to go elsewhere. Now, you search for something like sushi, then go to Google Maps, then see the phone number, then call.

I'm not _necessarily_ saying this is bad or unwanted, but it's a vast philosophical and product shift for Google. And it has implications for the trustworthiness and long-term usefulness of Google.

I do not welcome a future where Google invalidates the business models of businesses like Yelp by delivering their content in such a way that makes going to Yelp unnecessary. I don't want all of my websites to be subsumed by a corporate behemoth, so I get short term better products, but long term stifled innovation.

Or you can go to the restaurant's website through the link either on the SERP or Maps page... 8/10 though I don't want to see what Wordpress template a restaurant is using this week, I just want to know if it's open, what other people think about it and how to get there.

I find this SERP to be vastly more useful than how it used to be. Restaurant owners should too, I no longer will skip visiting your place because your website sucks or is not mobile friendly.

> Or you can go to the restaurant's website through the link either on the SERP or Maps page...

There are no links to restaurant's websites on that SERP page. Except one link to a Facebook page nearly falling off the bottom of screen.

If you click the photos or the red balloons on the map you won't go to the restaurant's webpage, but to another page on Google that presumably has more information. Sometimes the link to the restaurant's webpage isn't there either and I have to use yet another query (name of restaurant + location) to find the actual webpage.

The point/complaint of the article is that Google's results seems to be less and less coming from the Web as their index and more and more from information sources that Google controls.

Even though the change for the typical end-user might be small ("I can find the same stuff"), if continued, it signals a huge change in what Google is. It used to be a service to help you execute a text search over billions of documents, which I still think is a very neat thing to be able to do.

That's simply not true. When I click one of the carousel results, I get a search result for the exact restaurant, with their page at the top:

Initial search: http://files.calebegg.com/sushi-search.png

After clicking an item: http://files.calebegg.com/sushi-result.png

At least use the damn feature before you join in bashing it.

It's not really a philosophical shift at all, but a realization of the types of results Google has always wanted to give. People have this idea that Google always was, and always should be, intended to direct people to web sites, when all along the company has said they want to organize the world's _information_. Web sites just happened to be the easiest thing to do first, but the fact that they bought a mapping company very early on should have been a huge signal where they were going.

From Google's point of view, the question is: If you search for "sushi" are you more likely to mean "nearby sushi restaurants" or "a website that lists nearby restaurants"? Which results get you closest to your goal the fastest? Depending on your answer showing local results is absolutely the right thing to do for the user.

I would really like to see the websites of nearby restaurants.

Not another website listing them, not dots on a map with phone and address (although that is also useful), I want the actual sites, so I can compare them, see their menus and whatever else they have decided to share with visitors.

I don't think I ever got a result like that from Google when searching for <type of restaurant> + <place name>, it's always been just a few restaurants mixed with way more restaurant-listing-sites.

But currently Google sort of tries to hide them, it takes you a few clicks before you get to a link to a restaurant site. So no quickly opening 5 of them in background tabs, for instance.

Data mining? If they have your IP address, they have a decent way to generally locate you... and they obviously have your IP.

mine is even worse, because google apparently has trouble with me not being in the USA http://i.imgur.com/sQ5lue3.png

What did you expect it to show you? It seems like a decent result to me. Possibly google doesn't know any sushi restaurants near you in Warsaw.

pointless zip-code form.

It's not pointless if you are looking for a place to eat sushi. You're performing a hypothetical search for something that you aren't actually looking for and then calling the results pointless.

Google can pinpoint your location very accurately, and they are usually quite agressive in promoting local results (including for tech queries, which is a pretty bad idea), yet they ask people in Poland: "In which US city are you ?".

I find that quite funny, to be honest.

In Australia get the same search results as you, but there's no 'local search' section. I'm logged into google in FF

In chrome, with no login, I get a different set of results, including some local restaurants. Curious.

I'm not convinced with the arguments in this post. Auto Mechanic and Italian Food are both vague queries. Auto Mechanic could mean the definition of the term or places around you or even . And its not super hard to get rid of advertisements. If you really want better "organic" results, type in a better query -- "Auto Mechanic, Bay Area" or "Italian Food, New York".

I think its almost taken for granted that all search is interactive. If the first try is not precise, refine the query and try again. Its not correct to compare the Google of today with the "organic" search engines of a decade ago. The world, and the web have changed tremendously since. It can be argued that Google is trying to help the consumer by showing multi-faceted search for vague queries (which can potentially narrow search requirements) or provide cues about forming the next query -- and if a company can make money doing that, whats wrong in it?

This is making the arguments based on only things "above the fold". There are plenty of people that have dispelled the myth that content NEEDS to be above the fold (1)(2).

That said, the implication with Google results is that the higher up on the page the link is, the more relevant that link is to a user's search. In my experience, very few people ever get to the second page of Google Search results. I'm all but certain that if we looked at heatmaps of clicks across all Google search pages, we would see this quite clearly. It's the entire reason that SEO has been an industry at all.

It gets more troubling with products like Google Cars (3)(4). It is literally another layer of PPC results specific to cars. I've got to think that other car retailers (including manufacturers) can't be too happy about this.

(1) http://iampaddy.com/lifebelow600/ (2) http://blog.kissmetrics.com/why-the-fold-is-a-myth/ (3) http://www.drivingsales.com/blogs/InvestigativeReporting/201... (4) http://searchengineland.com/google-tests-new-car-leads-produ...

The first result for mobile is organic as far as I can tell (it just happens to be a google property when looking for Italian Restaurants in Tribeca). Try switching the query for mexican restaurant and just a few blocks to the west in SoHo (hi Aaron) a menupages result is first. Also, on the mobile front, I believe the quick results are from places and not Zagat, since with the same mexican restaurants query there are a number of places with just ratings and not "Zagat" ratings (you can see this in the blog's screenshot for Olive Garden as well).

I actually don't mind this on mobile (but I also generally use something else for finding restaurants) since the majority of restaurant websites are basically impossible to navigate on mobile and all I really need to know is 1) where are they 2) what's their phone number. How organic that database is, I'm not sure, but it doesn't seem to be showing specifically paid placements.

What you don't understand is Google's been doing this for almost a decade now. It's about profit optimization while not pissing off your users, or sometimes helping them even.

A paid result can be a useful thing and might even be better than an organic result in many cases, but both have their pitfalls and it's not surprising for them to keep going this route more and more.

Google is a company, not a nonprofit. They are in it for the money. Why is anyone surprised?

Not surprised, but they have 65% share of search in the US, so it's a disturbing trend for high quality sites ranking organically.

There is a way to increase the percentage of organic search results on a Google page, with relative ease. Just scroll down.

I mean, really, what is the article complaining about here? The most often clicked on organic search results appear above the fold. If you need more, scroll down a bit.

Maybe I'm the minority, but if what I'm looking for isn't one of the first few results, I modify my query. Google is showing the stuff that makes it money, and the organic stuff we're most likely to need.

And at the end of the day, what we're talking about is having to scroll down on a page. Faux blog outrage.

Indeed, I wasn't really aware that the "fold" was really much of an issue any more with modern web design. Users expect to, and do, scroll down; it's no big deal :/

Agreed! Personally, I use DuckDuckGo for privacy reasons, but I have a ton of respect for Google. The blog post is complaining about some trivial stuff, that I'm sure Google has spent (hundreds of) thousands analyzing.

If you have ever looked at a heatmap of where users scroll down to on a given page. It will clearly show you that the prime realestate is above the fold (before you start scrolling)

I agree with the arguments here on the whole. However, if you declare a local search on your own, by simply including "italian restaurant in tribeca", Google offers far more organic results. https://dl.dropboxusercontent.com/u/32342947/Screen%20Shot%2...

For some reason I was surprised to see bing's results even more cluttered and useless... http://i.imgur.com/kuRTMhO.png

You would think that bing would be trying to win the popularity contest by actually providing more relevant searches in the short term and then expanding ads in the future...

I'm not sure why Google's results might be said to be /useless/? There's definitely an interesting argument to be made about Google using a monopolistic position in search to cross-promote some of their other products (local, ads, maps), but the OP's examples seem to show results that quite likely exactly what the user would have been searching for.

I guess what I'm saying is that pure organic search isn't necessarily the optimal way for a user to find the information they're looking for.

This article seems to make a point of showcasing Zagat's rating in search above others, somehow indicating that Google is unfairly manipulating search results to it's own gain (as Zagat is Google, Inc. owned).

However, there's an important point here to be made, which has been said in numerous antitrust arguments against Google, and that is, "what if the product in No. 1, Google's, is in fact preferred by consumers and therefore is, to use language of the tutorspree authors 'organic'?".

It's an interesting question. I find much of the article interesting and of course the screen space dedicated to search is a hot topic, and Google's minimal style still remains in my favor, I just wanted to briefly object to the claim that somehow showing a non-ad Zagat page is "0%" (in the author's numbers) organic search.

Measuring the usefulness of a page by percentage of screen real estate only seems useful if the user is looking or clicking randomly around the screen. The structure, layout, and clarity of communication are far more important.

> 7% is all that’s left for the entrepreneurs and restaurantuers who believed Google over the years when they were told that good business with well structured pages would be able to get in front of potential customers searching the internet.

I'm really curious how the author would propose to fix this. Even with all of the other stuff cut away, there are only still 10 blue links to give.

Screen percentage is also incredibly dependent on your screen size. When I replicate their examples, I get far more 'organic' searches on my screen, because I have four times the screen real estate. Google has to delicately balance their advertising with organic results, and this balance works less well at smaller screen sizes.

> Measuring the usefulness of a page by percentage of screen real estate only seems useful if the user is looking or clicking randomly around the screen.

You should tell Google that. They claim to be penalizing web sites with too many ads "above the fold": http://searchengineland.com/google-may-penalize-ad-heavy-pag...

The bigotry in this has been pointed out by many people already ... Google is basically forcing other sites to use less attractive advertising while offering extremely annoying ads "above the fold" themselves, smells like unfair competition to me.

Try thinking about this from the user's perspective rather than the site owner's perspective. How do you like it when you click on an ad that looks like it has useful information behind it, and you come to a page that's mostly ads?

> How do you like it when you click on an ad that looks like it has useful information behind it, and you come to a page that's mostly ads?

Was that a Freudian slip? If I click on an ad that doesn't fulfill my expectation, the advertiser failed and will feel it, the market fixes this in the long run.

If I click on a search result (perhaps you meant that) and I get to see mostly ads, I can a) install an ad blocker, b) go back and click on the next result, c) ignore the ads or actually click on the ads. But there's nothing about this that warrants special treatment for Google's pages, which are full of ads. If Google thinks, users will not like pages full of ads, they should stop cluttering their search results with them. If they don't, they're just giving themselves an unfair advantage over the indexed pages.

The numbers are somewhat misleading: they don't add up to 100% because whitespace is counted as well (although not displayed). I personally like the whitespace on the results page.

Also the ads on the right side are delineated very liberally - one could argue that the actual area is half as wide.

I see lots of comments bashing Google, ads are evil, all that. But honestly how many people search for "italian food" on the web expecting global results to be relevant locally? Google is providing exactly what people are looking for when they google for Italian.

This feels somewhat disingenuous. Naturally a site making it's money off of ads is going to give them an above-the-fold and prominent position. Just scroll down... the value add is in the qualify of the organic results (which is still there), not in the absence of scrolling.

If you think the ratio is off, then write an article about it... but this is hardly "killing organic search".

Naturally a site making it's money off of ads is going to give them an above-the-fold and prominent position. Just scroll down...

Google penalizes sites with above-the-fold ads. It doesn't take much at all to earn the penalty. I think that's one reason why people are miffed with Google's excessive ATF advertising on their SERPs.

To reduce the amount of useless clutter that Google displays in search results, try verbatim mode.

You can turn on verbatim mode on the search results page. Click on the Search Tools menu, then look in the All Results menu for the Verbatim option.

The verbatim option tells Google to use all your search terms, as-is, and (at least at present) the search results are much less cluttered.

To always use verbatim mode, remove the search terms from the URL in the address bar (leaving intact the other parameters that turn on verbatim mode) and create a bookmark.

Don't tell Google about this feature... I depend on it!

Let them greed themselves into irrelevance I say. That leaves the door open for some YC startup to implement some type of peer to peer search protocol that will leave google in the dust.

It remembers me this http://yacy.net/en/

To put it simply, if you're top 3 on organic results, Google is useless for your business. And sometimes not even being top 3.

Nowadays you're supposed to buy ads to stay competitive. Google grew so much that it's now a monopolist behemoth, and that's why they're swimming in money. But because they keep all the nerds in love with them, they manage to get zero flak for it.

While this is true, users are still clicking organic results much more often than the paid options even though the organic results real estate keeps shrinking[1].

Even with the shrinking organic screen space, the first three results still capture the vast majority of the clicks, so results 4-10 receive fewer clicks regardless of the amount of space given to organic results.

I'd like to see a study where organic results 1-10 all appear on the page above the fold, giving users the ability to see all the results without having to scroll and measure click through rates at that time.

1. http://searchenginewatch.com/article/2200730/Organic-vs.-Pai...

To be fair searching for "italian restaurants" in San Francisco the first result returned is a Yelp link on mobile, so I guess that the Zagat one wasn't a way to show necessarily a Google owned property. Everything else is the same.

Google is becoming like a newspaper. A connecting point between people, ideas and information.


1) Google has steadily moved away from optimally presenting search results which they possess.

2) In being this connecting point, ads increasingly have taken over the space where organic results are published, much like ad space is sold first in a newspaper and articles are squeezed in around it.

Everything changes, or does it? Content is king, except finding content is a new king.

3) Google generally has the content we seek - in the form of search results. The results are squeezed around ads. This creates a cognitive cost to separate ads from results, instead of just finding what we need, which was the original promise of Google. This implies, at some point we're giving into reading the ads like they were the results?

Users have only ever been interested in search results from a search engine, not ads. In a way, good search results was a promise shared by AltaVista and Google. Being able to easily access them is what is changing.

Last year I was irritated enough to build my own search interface, focused on presenting results how I wanted to see them.

4) The cognitive cost had added up to be too much to scan Google constantly to find the one weather, or movie link I have looked at for the last 10 years, and becoming increasingly harder and harder to find.

My thoughts come back to the announced shuttering of AltaVista, in wondering whether we really ended up ahead. Google certainly has done things no one else has, getting search so good, and now finding what we want is a little more work.

5) Google is a business, and need a financial engine. As much as advertising currently seems to pay for the internet, I wonder if a day is coming where we're more willing to put our money where our mouth is.

It may take some time, for the majority of internet users to have 10+ years of online experience of doing the same searches and finding them harder to find.

What to do? I feel like brushing up my custom search interface and use it a little more.

What kind of startup could exploit that situation with Google SERP?

My guess would be that some sort of rating site that better answers questions like "what car to buy?" should benefit from Google removing organic results from searches like that.

The rub is that a paid result can be just as useful from the consumer perspective as an organic result. If that's the case, Google will show the paid result every time.

It gets even worse, on some searches I have to scroll a full page down, just to see a result not pushed by a google asset (ads/map/etc). In essence it takes up about 90% of screen space on a large desktop.

Example: http://imgur.com/AJtOPkf (this is zoomed down to 65% to capture all of it....)

They need to remember that there are alternatives just a click away, and serving too many ads degrades the entire internet.

From a network centrality metrics perspective, is there such a thing as "organic search"? Is the premise flawed or is the Page Rank metric considered organic?

Before Google, people used to use Yahoo directory to find links.The only all encompassing directory in competition now with Google search is Wikipedia (not even its intended purpose)

People can use Yelp to find Italian restaurants in San Francisco but try typing the same in Wikipedia.

Is there any personalized search engine which lets you choose Yelp/Zagat/Tripadvisor when you type in that keyword? and maybe pull down Google map so Google doesnt feel left out ;)

Do results from https://startpage.com work for people in the US? I've been using it for a couple of weeks now (I'm in the EU) and it seems like it works very well (it claims to get its results from Google, but without affecting privacy). It shows some display ads, the percentage of screen real estate used is much smaller than for Google itself.

I just tested this myself, and got 43.8% of organic results (I'm counting the image result preview as organic results), 18.9% of google UI (the bars on top), and 35,2% whitespace. They do not sum to 100% because I just roughly measured stuff at photoshop. Zero ads or curated content on either search, logged in with my personal google profile.

I guess adblock and not living in US has some advantages.

I wonder if there is interest in someone building a search site that uses the google custom search API to search. The CSE that Google provides does not include advertising or local results unless specifically requested. The only problem would be that it costs money for large volumes, so you might have to charge or use ads to pay for it.

I disagree with the premise. "Organic" search will not exist, barring a radically new business model. Any search engine with the resources to produce good search results must carry auction-based advertising in order to fund the quality of their results.

Good luck finding that business model, and being able to reap the profits from it.

Great article! Sad in a way how Google continues to dominate and impose it's will on us.

Yeah, fancy showing relevant local results when you search for a Restaurant! The bastards!

Google Search was totally awesome when it first came out. Disrupted the whole industry and set new standards of search on the web.

Now I find myself longing for a new company and a new disruption of what is essentially Google monopoly.

Google == advertising company Google != search engine

I'm happy with Google because no ads while using www.google.com ( not .hk ) in Mainland China where Google quited from. Thanks Google! :)

And this is a further proof: https://www.google.co.uk/flights

Those percentages don't add up to 100%. Seems disingenious to count the white space in the display just to make a point.

On desktop, set http://duckduckgo.com as your search provider. Problem Solved.

Even when I !g to see google results, I see 100% organic results because I have Adblock Plus plugin installed on firefox. Problem Solved.

Make sure you have 'Hide placeholders of blocked elements' option enabled, otherwise AdBlock will show a blank rectangle where the ads used to be.

You're not getting 100% organic results because the author is using screen space as the measure, not returned results (and apparently a custom map doesn't count as an 'organic result'). White space at the sides of ddg? Doesn't count. ddg UI bar at the top? Doesn't count.

Panda and Penguin are euphemisms for Google A/B testing revenue.

You either die a hero or live long enough to became the villain. At least we have duck duck go.

Big dog's gotta eat.

This is why I block ads.

Adblock,FTW ! Never seen an ad in several years now.

goto.com anyone?

There's the other problem of decreasing contrast and lack of borders separating the last ad and first organic search result.



A few years ago this was the style http://www.ismoip.com/blog/wp-content/uploads/2011/01/Screen...

Before that http://cdn.userstyles.org/style_screenshot_thumbnails/58617_...

Now http://i.imgur.com/Wmdd0.png

Some of those keywords are worth tens of dollars per click so no wonder,the colors have been A/B optmized to get the most clicks from people not knowing they are ads.


Google: #fff7ec Bing: #f9fcf7 Yahoo: #fafaff DuckDuckGo: (255,212,0,0.18)

Not to say that Bing, Yahoo etc. are much better but I expect more from the "Do No Evil" Google rather than increasing the next quarter's earnings instead of targeting older people and people with bad monitors and hurting people who did a lot of good work to come in the first few in organic results but don't and/or can't pay Google for expensive ads. Also, Bing and the rest continue to mostly lose money and they can't afford to separate ads while the big guy continues to reduce the difference between ads and search results.

"Study:Contrast sensitivity gradually decreases with age" http://www.eyeworld.org/article.php?sid=818&strict=0&morphol...

Note that the FTC agrees with you and recently made an announcement[1] regarding the shading of the ad result box, among other requirements for search engines.

The study the FTC cites[2] indicates that fully half of individuals didn't even realize the page contained ads.

[1] http://www.ftc.gov/opa/2013/06/searchengine.shtm

[2] http://www.seobook.com/consumer-ad-awareness-search-results

I would use a paid search service with a quality search, no ads, no tracking, and no selling my data to third-parties.

scroll down.

Applications are open for YC Winter 2022

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact