Hacker Newsnew | comments | show | ask | jobs | submit login

Matt, this does not address whatsoever the point being brought up. And in case you missed it, I will try to cite it as clearly as possible here:

This is a case where google wants people to mold the internet according to what their current incarnation of their algorithm says the way it should be.

Rather than google trying to mold its algorithm to understand the internet, natural links vs. spams, and their respective authority and value.

Google here is saying that all links from this site are suspect, even when the link in question is 100% valid, quality and relevant.

Instead of going back to the drawing board and trying to fix the algorithm, google has discovered that since they are so big now, it is easier to play the "benevolent" dictator and dictate to the whole internet, how and what they should be doing.

And to mete out knuckle raps when they get out of line.

Do you see the point now?

The model (google's algo) should fit reality. Rather than forcing reality to fit the model.

I believe this is what the author is trying to communicate by saying "google is breaking the internet" - what he means is that the internet is becoming a 'google' version of itself, rather than what it would naturally be.




> Google here is saying that all links from this site are suspect, even when the link in question is 100% valid, quality and relevant.

I read the article and didn't see any mention of what site the link was on or even what site it was linking to. It's impossible to say if it was legitimately spammy or not, but either way Google never asked him to do anything. Someone who didn't know about the Link Disavow Tool decided on their own to email him about taking the link down.

You would have a good point prior to the Link Disavow Tool, but it has solved this (very tricky) problem.

-----


Someone who didn't know about the Link Disavow Tool decided on their own to email him about taking the link down.

FTFA:

"In short, the email was a request to remove links from our site to their site. We linked to this company on our own accord, with no prior solicitation, because we felt it would be useful to our site visitors, which is generally why people link to things on the Internet."

"Apparently Google convinced them, via their Webmaster Tools portal, that the link looked 'unnatural', and that they should use the Link Disavow Tool to discredit the link. Furthermore, they thought it necessary to contact us to manually remove the link, which is something I’m not going to do (out of principle)."

-----


... Exactly. Someone took it upon themselves to hassle the OP to remove a link. Google has made a tool to allow you to have links that you don't vouch for to be not counted, but the webmaster in question decided to try and do it "manually".

It used to be you would get sketchy emails asking you to link XYZ and now a few years in the future the sketchy emails are asking you to unlink XYZ.

-----


I think you missed the point of the disavow tool. If Google hit you with a penalty, they still expect you to put in a good faith effort to remove the link naturally first.

https://support.google.com/webmasters/answer/2648487?hl=en

-----


the doc says it's your responsibility to remove spammy back links that you (or a SEO you've hired) have introduced to improve your ranking.

it cannot possibly be your responsibility to remove links that other unrelated entities have towards you! (unless it's an obvious link scheme)

-----


I think Matt and Google are getting an unfair rap on this. Google provided a search engine that worked better than any other so people started using it. People are still using it because it's still better than the others. It's not a monopoly. You can use another search engine if you want to.

The site that emailed to get a link taken down employed an SEO agency to improve their rankings and they created a bunch of spammy links to try to manipulate Google into ranking that site higher than others that probably deserve to rank higher due to their content. If Google did nothing about it we would get some crappy search results.

Also, if you read what Matt Cutts is saying, it implies that if they only had a few unnatural links then they wouldn't have received a webmaster tools warning telling them they had spammy links. They got the warning because their link profile suggested that it was obvious they had been trying to manipulate rankings, not because they had one or two iffy ones.

It's likely that the site then blanket emailed anything and everything that they weren't sure about to get the links taken down. Is it Google's fault that this is how the site decided to deal with the warning? Should Google just make it really easy for them by saying, "OK you disavowed those crappy links, you're off the hook now"? If the consequences are so weak and short-lived, where is the deterrent effect? No, Google doesn't own the internet but they own Google and the site is just trying to provide users with the most relevant results to their searches.

Re, the nofollow aspect. Google didn't "force" sites into using the nofollow attribute by scaring them into thinking that their rankings would drop. Nofollow was in place before there was any notion that who you linked to could harm you. Sites like Wikipedia and others were only too happy to add nofollow to links because it put spammers off from crapifying their sites.

-----


No, they aren't. Google is using FUD to scare sites into requesting links be removed that they never built on legitimate sites. No one can possibly guess which links an individual Googler will think "might" be paid. I assure you they guess wrong most of the time.

As Google penalizes the best writers and blogs, one of two things happens:

1) Entire sites are being nofollowed - removing links to deserving sites.

2) The site refused to nofollow, so they stay penalized with PR0 no matter how good the site may be or how high the PR once was.

The end result will be that even more black hat spammy MFA sites will rank. Sites that deserved to be written about and linked to will drop and be invisible.

It is high time people stop defending Google when they are destroying small businesses and costing much needed jobs. Now Cutts has declared war on bloggers who often make a living writing.

Google wants to destroy any site that enables advertisers and businesses to connect with sites where their target audience already is by forcing publishers to decide whether they want to risk losing organic traffic and being branded with a scarlet PR0 or to nofollow everything they ever write.

Nofollow = NOT TRUSTED. We do not invest time writing about anything we do not trust. (On the VERY rare occasion we do we do not link to them.) We do not publish content that does not fit our audience.

Every link is a potential risk. Nofollowing links may damage the sites we link to so what is a writer who already cares about what they write about supposed to do?

Google has a monopoly on search because the wealthy elite who own the media handed it to them. People use it because in many cases they do not even know there is an alternative. Many use whatever is installed on their computers without even having any idea what it even is.

People who are trying to clean up unnatural links penalties are not SEO experts. They rely on tools to tell them what links to remove. These tools are not very accurate. I've had them flag DMOZ, Best of the Web, and Business2Community as "toxic". Finally many remove and/or disavow every link in desperation.

How does all that disavowing affect search results and individual sites? How many verify that requests to remove organically given links are from the actual site and not their unethical competitor. (Hint: almost none.)

Many businesses fail waiting for recovery. Others get penalties lifted, but having removed links they no longer get any traffic.

"Google didn't "force" sites into using the nofollow attribute by scaring them into thinking that their rankings would drop." YES, they most certainly DID!

-----


Lots of misinformation in this post. It is almost hard to pick a spot to start.

>Google provided a search engine that worked better than any other so people started using it.<

While at the beginning this may have been true, over time that search engine has grown more parasitic in terms of things like scrape-n-displace knowledge graph results, AdWords ads on branded keyword terms where ads with junk bundleware rank above official sites, etc. http://blog.brandverity.com/2738/software-bundlers-target-br...

>People are still using it because it's still better than the others.<

Care to explain why Google is spending over a billion Dollars a year buying default search placement in other browsers like iOS & Mozilla Firefox? Any thoughts on the Flash security updates which hit other browsers and bundle Chrome's web browser in with it? Or how about the Android contracts with default search placement (and other forms of) bundling baked into them?

Google is spending well over a billion Dollars a year on the thesis that your thesis is wrong.

>It's not a monopoly.<

The hell it's not. At least if we use any of the standard definitions.

>You can use another search engine if you want to.<

And while an informed individual may choose to, the majority of people are driven by default settings which are purchased, as per the above.

>The site that emailed to get a link taken down employed an SEO agency to improve their rankings and they created a bunch of spammy links to try to manipulate Google into ranking that site higher than others that probably deserve to rank higher due to their content.<

Are you suggesting there are not false positives, or that competitors do not buy links to torch their competitors? Either such assertion is simply untrue.

>They got the warning because their link profile suggested that it was obvious they had been trying to manipulate rankings, not because they had one or two iffy ones.<

The second they there presumes a competitor didn't do them for it. Only a person ignorant of the field of SEO would presume this to be true in all cases.

>Re, the nofollow aspect. Google didn't "force" sites into using the nofollow attribute by scaring them into thinking that their rankings would drop. Nofollow was in place before there was any notion that who you linked to could harm you. Sites like Wikipedia and others were only too happy to add nofollow to links because it put spammers off from crapifying their sites.<

This is a complete misunderstanding of history, on numerous levels.

Nofollow was introduced as a (ineffective) solution to blog comment spamming. To help aid further/wider adoption, at some point some Googler's even suggested things like pagerank sculpting could be useful, up until some large sites started doing it excessively. They were looking for reasons to justify its widespread use, because Google intended from day one that the tag could then be spread onto paid links & other links they didn't want to count.

And the other level of absolute misunderstanding was that (before Google went on their fearmongering campaign about links) they in the past suggested that you couldn't control who links to you, but you could control who you link to & sites which linked to bad neighborhoods could indeed be penalized for it.

Very rarely is there a comment which is that long & that wrong. Impressive!

-----


A spammy link is a link created on behalf of the target site.

In this case the author says he created the link independently, because he thought it would be useful to his users. That's, by definition, a genuine, organic link.

I wonder if a link flagged by the disavow tool and confirmed by the user weaken the Google juice of the source site.

@MattCutts (or another Googler), is it the case?

-----


"I wonder if a link flagged by the disavow tool and confirmed by the user weaken the Google juice of the source site."

We haven't been using disavowed links as a reason not to trust a source site. So it wouldn't weaken the Google juice of the author's/source site if the company that sent the email had just disavowed the link.

-----


This in no way implies Google won't use the disavow data to not trust source sites in the future.

-----


Good to know, and thanks for the answer.

-----


> In this case the author says he created the link independently, because he thought it would be useful to his users. That's, by definition, a genuine, organic link.

Genuine organic links can still be spammy. The OP seems to be a genuine affiliate marketer/spammer, it's not inconceivable that his links aren't treated with a ton of respect.

-----


Affiliate marketer & spammer are not natural synonyms.

Some entities might try to say it is six of one & a half dozen of the other, http://www.seobook.com/brand-vs-affiliate-vs-spam but those same entities do not view their own affiliate marketing efforts as spam, even when they are pure spam http://www.seobook.com/google-lends-a-helping-hand

Look at the cloaked affiliate YouTube links to iTunes on music videos. And Google has invested in Viglink, RetailMeNot, etc.

In addition to those types of investments, it was Google's ad programs which funded entities like Mahalo & eHow.

There are zero affiliates which have made as much off affiliate marketing as Google has.

-----


That's a genuinely good point :-)

-----


It seems like it would be odd to not incorporate a "spamminess" measure into the strength of the link, so hopefully you'd be losing very little.

-----


"This is a case where google wants people to mold the internet according to what their current incarnation of their algorithm says the way it should be"

You do realize, of course, that Google is part and parcel of the internet, and that much of the way the internet works "in reality" is in response to external perceptions of how googles algorithms work?

Google's early dominance in search was a result of their algorithm's ability to surface relevance. If they had never staked out that Site A linking to Site B means site B has more relevance, then the internet would be an entirely different beast, and SEO would take entirely other forms that do not amount to linkstuffing.

The internet hasn't existed 'naturally' since search engines first came around.

-----


It's not just Google that recommends to avoid link spamming. For example, at http://www.seroundtable.com/bing-links-knowledge-18535.html you can read where Bing said "You should never know in advance a link is coming, or where it’s coming from. If you do, that’s the wrong path."

When SEOs break the quality guidelines of all major search engines (not just Google), then they might need to do some work to clean things up if they get caught.

One of the reasons that Google is such a popular search engine is that we do take spam seriously and we take action to counter spam.

-----


When SEOs break the quality guidelines of all major search engines (not just Google), then they might need to do some work to clean things up if they get caught.

This still seems to be missing the point.

As I understand it, sites that are not doing anything in the realm of SEO can get penalized because of the actions of people outside of their control.

If the wrong sorts of sites (however defined), sites outside my control, are linking to my site, why am I penalized, and why should the burden be placed on me to go fix it?

-----


Frankly a bit worrying that Matt Cutts doesn't seem to get the fairly straightforward point (to the rest of us) being made here. Or he does and the criticism stands. Either way not so great.

-----


Put the pitchfork down; he's just giving non-specific answers in response to non-specific situations. For all we know, the two domains in question are `bestquickdietfatloss.com` and `levitracialisfreedietpillscanadamexicoblog.com`, in which case it would be correct of Google to indicate that the author's link is linkspam.

-----


Even if the sites are not relevant, the existence of a link does not mean the site which the link points at was responsible for creating the link. In some cases (especially with website hacking) those who create the spam will link to innocent third parties to create a smokescreen. Google can & does burn some of these "drive by shooting" sites to a crisp.

Check the people who try to drop profile links & comment spam links on your blog and such. Are there any websites which have as many spammy inbound links as YouTube does? How many people does Google allocate to cleaning up YouTube's spammy link profile?

There's another factor in that Google may shift what is considered reasonable over time. A buddy of mine got an unnatural link warning where the link cited was a link that has been in place LONGER THAN GOOGLE HAS EXISTED.

Now consider that some websites are bought & sold, change ownership, etc. ... does it make sense to penalize today for something which happened 10 or 15 years ago?

Another factor is that just by ranking you will pick up some scraper site inbound links you do not want. Many of those sites in various forms or fashion are monetized via Google ads. Google continues to run their ads on many of those aggregation sites, yet you may get an unnatural link penalty for having links from the same sites.

Good enough for Google, NOT good enough for you TM

-----


Except that I and it seems many commenters here have seen similar things occurring with non-spam sites. Regardless of the specific domains referred to in this post, a general pattern is being talked about - the fundamental criticism is that we are getting a google-shaped web rather than a web-shaped google, due to google's de facto monopoly position. If this wasn't the case, it wouldn't be a talking point here.

-----


"we are getting a google-shaped web rather than a web-shaped google"

Very well put.

-----


We were getting a google-shaped web from the moment SEO started optimising for Google rankings.

-----


Frankly a bit worrying that Matt Cutts doesn't seem to get the fairly straightforward point

Oh he gets it, it's his job to act like he doesn't and to change the subject "...Ummm...Bing does it too..look I made a video"

-----


"If the wrong sorts of sites (however defined), sites outside my control, are linking to my site, why am I penalized, and why should the burden be placed on me to go fix it?"

Well, if a website was link spamming then it seems appropriate that the site should attempt to clean up the spam they made before they can rank well in Google again. Otherwise it's not fair to the other websites that have been trying to rank fairly and would like a level playing ground.

In general, if you weren't trying to create spam links, then it's very unlikely that any of this is an issue for you and the burden isn't on you to fix it. I made a video about the "what if the wrong sorts of sites link to my site" question here: http://www.youtube.com/watch?v=HWJUU-g5U_I

-----


>In general, if you weren't trying to create spam links, then it's very unlikely that any of this is an issue for you and the burden isn't on you to fix it.

So if Bing wants to rank Bing.com higher than Google.com on the keyword "Search Engine", all they need to do is to pay link farms to link to Google?

Like everything this could go both ways. If you penalize sites for having links from link farms, what is stopping your competitors from paying link farms to link to your website? or worse yet, some link farm owner comes up with an ingenious idea to hold other websites hostage until they pay up?

It would be more understanding if Google din't have any other resources on hand to make smart decisions but heck you guys have hands on realtime experience with most traffics via Google Analytics, Google Adsense and even DNS. All these resources and can't even distinguish a link farm from a legitimate site? Doesn't sound believable.

-----


> All these resources and can't even distinguish a link farm from a legitimate site? Doesn't sound believable.

The Google webspam team seems to prefer psychology over technology to solve the problem, especially recently. Nearly everything that's come out of Matt Cutt's mouth in the last 18 months or so has been a scare tactic.

IMO all this does is further encourage the development of "churn and burn" websites from blackhats who have being penalized in their business plan. So why should I risk all the time and effort it takes to generate quality web content when it could all come crashing down because an imperfect and overzealous algorithm thinks it's spam? Or worse, some intern or non-google employee doing a manual review wrongly decides the site violates webmaster guidelines?

-----


>prefer psychology over technology<

yup.

>Nearly everything that's come out of Matt Cutt's mouth in the last 18 months or so has been a scare tactic.<

the bizarre thing is the gap in perceptions internal to Google versus external. https://twitter.com/mattcutts/status/449008706655510528 https://polldaddy.com/poll/7916680/

>IMO all this does is further encourage the development of "churn and burn" websites from blackhats who have being penalized in their business plan.<

Absolutely. churn & burn sites, and then some mixed in parasitic hosting on sites which are already highly trusted.

-----


>So if Bing wants to rank Bing.com higher than Google.com on the keyword "Search Engine", all they need to do is to pay link farms to link to Google?

During this "crackdown campaign" Google has been on, I have sat in on meetings at large clients where they've pulled sites that are supposedly link farms (according to google) and put together an ad buy on behalf of competitors meant to cause these penalties to be levied and WMT messages to be sent.

-----


Sounds like a classic Bait and Switch, albeit a good 10 year long one on Google's behalf.

They needed to provide a useful service to begin with to get the traction (good search algorithms for the web-that-is), but now that they are the incumbent, it is more in their interests to penalise website owners and force them to buy ads.

It's sad, but it's normal monopolist behaviour.

-----


Now you're talking about a malicious attack. If that's what it takes for this to be a real issue then I don't think many people need to be too concerned.

-----


The cost of building a quality website & promoting it can easily hit deep into 6 or 7 figures.

The cost of throwing scummy links at a website is under $100.

And if their first $5, $10, or $20 test doesn't work? Then so what for them. They put it on cron job and throw another $20 at it again and again. http://www.seobook.com/images/sape-sape-sape.gif

The key point here is that there's a 100X or 1000X gap between 4 figures & 6 to 7 figures. The cost of burning things to a crisp is much lower than building them up. Whenever incentive structures have a 1000-fold difference in outcomes the path of least resistance becomes a popular path.

-----


It seems like you are heading down the path where black hat anti-SEO on competitors becomes an increasingly viable business model. For example, company A competes with company B. They are cutt-throat so they pay a blackhat SEO firm to generate obviously spammy links for their competitor, not in an effort to bring up their own ranking, but to tank the ranking of their competitor relative to their own. At the end of the day, you do not know the intent of a link. Discounting a low quality link is reasonable. Penalizing the site it points to is not.

-----


The victim site looks at where these newly created back links are coming from, and then reports them to the Google disavow tool and disavow the lot of them. That is the mechanism for dealing with the intent of those links.

After a few cases like this, a long list of multiple disavowed domains/networks/fingerprints, where are these negative links going to come from? This anti-SEO tactic you outline can only be used a handful of times before it becomes useless.

-----


So now black hat SEO people have a tool where they can buy links and disavow the ones where they get caught?

-----


So in this scenario, my competitors can spam links for my website and put me into trouble. Google will notice that my website has some spammy links (which my competitor created), and then penalize me for it. Then I have to contact every website that links to me to ask them to remove links to my website... else their website would get penalized too. I get such emails every week.

Rather than these fear mongering tactics, why can't Google just ignore spammy links instead of penalizing the whole websites and all the websites that link to them?

-----


I didn't really like matts answers, some of them are not answers at all.

But! You may have hit the nail on the head. if i had to guess i would say that storing exceptions means that suddenly the algorithm had to check part of the corpus for these exceptions. at the scale at which google operates this could probably lead to rather big size and speed implications.

But maybe i'm just fantasizing.

-----


When monopolies distort markets, the impact of those distortions have outsized impacts on the rest of the ecosystem.

Ask Google engineers how they feel about centralization of power in telecom or finance or politics & they will tell you it is the worst thing ever http://www.youtube.com/watch?v=Ik1AK56FtVc

Yet that same sort of centralization is fine when it is Google.

Those same engineers will admit that any algorithm has some level of false positives & false negatives. They may try to minimize these, but they can't make them zero.

The other thing which gets very little coverage, but is crucially important is that in spite of already having a monopoly market position Google keeps buying search marketshare with: their secret Android partner contracts, Firefox default search placement, iOS default search placement, Flash embedded into Chrome (so security updates happen in the background without sending you to a download site where trash can be bundled in with the update) & Flash security updates that hit all other web browsers coming bundled with Chrome bundleware which sets Chrome as your default browser, etc.

If Google decides they don't like you (for any reason), then you need searchers to use a different search engine in a browser which isn't paid off by Google for default placement. And you need to hope that users are savvy enough to repeatedly say "no thank you" to the automatically bundled Chrome install with their frequent Flash updates.

Google mentions how anything is only a click away or similar, but most people tend to use defaults.

When someone changes those defaults on Chrome users, that's a horrible user experience & the "hijacked" settings must be reset. http://chrome.blogspot.com/2014/01/clean-up-your-hijacked-se... But when those defaults are paid for by Google or changed through Google bundleware it is "a great user experience."

If Google believes their marketshare is a reflection of their superior search offering, they are welcome to stop buying default search placement in other web browsers & stop bundling Chrome installs on Flash security updates. But currently they are spending north of a billion Dollars a year on these activities, which indicates they clearly feel there is significant value to them.

-----


Google's algorithm already analyze if a link is useful or spam. They can simply ignore the links that are spam and count only good links. But, Google decides to punish the websites caught in the fire. If your website has a link Google does not like, Google can penalize your website along with all the websites that link to you or you link to them.

-----


> why can't Google just ignore spammy links instead of penalizing the whole websites and all the websites that link to them

You are suggesting an approach where there are no downsides to link spam. Obviously, players of this game would then ramp up their link spam bots, and run them non-stop. Because it causes no damage to the site they want to rank.

Link spam isn't just a search engine indexing problem, it is also a detriment to the general Web user too. It makes the Web worse, not just search.

-----


That would be a Joe Job, right?

-----


Your comments on this post are a great demonstration of the attitude that is driving me off Google properties, as quickly as I can find viable replacements.

-----


Best of luck staying off of Google properties. It gets more challenging by the day. http://mako.cc/copyrighteous/google-has-most-of-my-email-bec...

Don't like YouTube's copyright claim issues on video game playthroughs? Use Twitch. Oh wait, what's that? Google is trying to buy Twitch. http://kotaku.com/report-youtube-is-buying-twitch-for-1-bill... Oooops.

-----


If manually disavowing a link truly has the same effect as getting it removed by contacting the third-party site's webmaster, then why would people still ask for links to be removed? Is disavowal somehow penalized, i.e. it reduces but does not eliminate the penalty induced if no action had been taken? Or is Google's line that the penalty would be eliminated, but people don't trust that Google's algorithms work the way it says they do, and still ask for the takedown to be safe?

-----


Links which are nofollowed are still seen by users & can still impact that user's experience with the website. Yet those are fine, even if they point at spam, because they don't have a significant impact on Google.

So then what's the difference between disavows and having the links removed? Getting the links removed costs far more time & money. And it gives Google a data mining stream of feedback they can leverage to dish out further penalties.

Google requires some of the links to be removed not because it improves the web, but rather their goals & interests are aligned with punishment. They want to add cost & uncertainty to SEO in order to discourage investment in SEO from entities not formally connected to Google.

-----


> So then what's the difference between disavows and having the links removed? Getting the links removed costs far more time & money. And it gives Google a data mining stream of feedback they can leverage to dish out further penalties.

That's a good thing right? Pushing sources of bad links down through the basement, effectively nuking the source/network from the link graph.

You know Google's ranking system is based on information, and adding more information leads to better results for the Web user.

-----


>That's a good thing right?<

That really depends on your perspective.

If it were impossible for others to throw trashy links at your site in bulk for cheap it could perhaps be a good thing.

If Google didn't change policies overnight & then retroactively penalize you for things which were fine in the past http://searchengineland.com/guest-post-google-penalty-187707 then it could perhaps be a good thing.

Unfortunately, in reality, both of those ifs are hypothetical & untrue. Which means it is absolutely not a good thing.

>You know Google's ranking system is based on information, and adding more information leads to better results for the Web user.<

Quality of information matters as much as the volume of information. Look at Demand Media's current stock price for an example of this.

But in terms of links, you have to think through the impacts here...

Yes if Google freezes some activities they dislike then perhaps that is a net positive for relevancy, however the more they fear-monger about links the less natural linking goes on. Most of the major social sites put nofollow on almost all external links. And with the sea of fear approach to relevancy, in some cases magazines or newspapers will profile a person & not link to the source in spite of the entire article being about that source. I've spent many hours being cited by journalists where in spite of being good enough to be the primary source for their content, there was no link citation.

paraphrasing David Naylor's excellent recent video http://youtu.be/UdflGrCJxYc?t=1m50s "More and more high profile websites are having a no linking policy. Which really is kind of weird isn't it, because if that is a journalistic website and they are writing about your website surely those are the kinds of links Google would want to see. ... It seems kind of weird that the links Google actually wants you to get are the first links to dry up on the web." - David Naylor

The other important factor is the quality of the information created by those who are trying to do disavows and link removals. When people are irrationally driven by fear while in a harmed state, they are NOT acting rationally. Business owners who are selling off assets, firing employees, stalling with creditors, aligning predatory lending to try to keep things afloat, etc. are stressed out & are likely to make many poor decisions in that rushed & panicked state.

Thus some of that more information which is created is junk misinformation.

Consider this email: http://www.seobook.com/images/link-unremoval.png They asked the website to remove the link claiming it was unnatural, and then they emailed again to ask the website to ignore the removal request.

They don't know which links are "unnatural" and so they are using automation to try to sort through it all. Are their automated quick guesses (which are often driven by tools) more useful than all of Google's internal reviews & ratings data which has been built up for over a decade? Color me skeptical.

There's another factor with the disavow data as well. Look at this removal request http://www.seobook.com/images/trampolines.png Our site has about 20,000 unique linking domains referencing it. However over the years we have had well over a million registered user profiles. If only 2% of the registered user profiles were ignorant spammers who spammed our profile pages and then later added our site to a disavow file, we would have more people voting against our site than we have voting for it. And those profile pages were already not indexed & the links were nofollowed anyhow. Those pages are effectively outside the search game, yet those ignorant spammers can still create negative votes against our site based on the fear-mongering.

And as bad as that sounds, there's no reward for actually removing the bad links either. At some SEO conferences numerous SEO experts have gave the advice to "just disavow them anyhow" even if the links are removed. Thus this deluge of email spam offers publishers no value whatsoever, just sunk cost & wasted time - time which could have been spent creating useful information.

-----


If my website is getting spammy links, which I did not create. Additionally, it is out of my control to create / edit / remove those links.

In such scenario, how does Google determines if I created the links? Does Google have precognitive abilities or it can spy on my communication channels (Gmail, Google Voice, Hangouts etc.) to figure out if I am innocent?

-----


That's where the Google disavow links tool comes in: https://www.google.com/webmasters/tools/disavow-links-main?p...

A model is only as good as the information in it. In this case providing details of these spammy links helps you, and every other site targeted through the same sites/networks/fingerprints.

Consider it a reverse no-follow mechanism.

-----


There is no spam, you algorithm is wrong. Wrong because it's implying the link isn't mean to be there.

It appears after years and millions spent it is impossible to get the perfect search algorithm.

-----


What Matt Cutts is saying, is that as collectively smart as his employer Google is, they aren't able to always figure out if a link is natural and when it's "paid for" as part of linkspam. If they could, they would obviously put that in their algorithm.

Matt Cutts' job at Google involves him making tons of videos explaining how the Google algorithm works. It isn't omniscient, and the guidelines show how to work within its limitations. If you don't care about optimizing your Google presence (as we don't) then you shouldn't care about Google's guidelines either. But if you do comply with some guidelines, then this will probably help your rank in other search engines which also take similar measures to TRY and detect linkspam.

Google's customers are its users. It tries hard to have the most relevant results (or provide value some other way, or just maintain a brand image so people keep coming back). The advertisers come because the users find them and spend real money. And the users want relevant search results, not spam. That's why Google spends a ton of resources on combating the spam.

How do you propose they magically "know" when a link is natural?

-----


> Google's customers are its users.

No, Google's customers are primarily its advertisers. Optimizing advertising revenue has some overlap with pleasing users, as you note, but if a decision needed to be made between pleasing users and increasing revenue, there's no doubt about which way it would go.

-----


They would choose pleasing the users.

I suppose any company that works out how to increase revenue while upsetting and alienating it's users could possibly implement said ideas. But if you're smart enough to do that, I would guess you would also be pretty successful at the traditional approach of increasing revenue by pleasing your users?

-----


I often heard that google's customers are its advertisers, and I want to ask why the users aren't considered end-customers since they and their interaction with google provides the majority of the MONEY that feeds the ecosystem that feeds google?

-----


> Google's customers are its users.

No, Google's customers are its advertisers. They're the ones who pay money to Google.

I agree with your general point though, the algorithm isn't perfect and sometimes it goofs. What I think people are having a negative reaction to is Matt Cutts' seemingly obtuse responses, as though he doesn't even understand the issue at hand (Google's actions distorting natural behavior), so he just keeps explaining the anti-spam policies in general terms. It's not a conversation, it's people talking past one another.

-----


The icing on the cake is that Matt Cutts is obviously a very bright and very perceptive person so it's easy to assume that his not understanding is intentional. That would be right out of the PR/Politics "avoiding awkward questions 101" textbook but it's a scummy tactic.

-----


I would bet Matt's IQ is easily far north of 140. He's one of the most perceptive people I've ever met.

Think of some of the stereotypes of engineers. Many can code but struggle to communicate. Matt can code, has his name on many patents, leads a team, makes hundreds or thousands of videos, regularly keynotes at conferences & interacts with thousands of people, was GoogleGuy for years, participates in the comments here, was an expert in some of the past lawsuits against Google ... he has basically had a near infinite number of opportunities to put his foot in his mouth & yet how many times has he ever done it? Almost never, if ever.

About the only times I think he has potentially missed at all was the "breaking their spirits" bit on TWIG, and then two minor bits in a few YouTube videos

http://www.youtube.com/watch?v=muZuX9OaMLo&feature=youtu.be&... "We got less spam and so it looks like people don't like the new algorithms as much."

https://www.youtube.com/watch?v=2jap11IIxfw "the gap between what you can get away with and what google says you can get away with is getting smaller all the time"

---

Who do you know who has as much media exposure as he has had who hasn't repeatedly put their foot in their mouth? For me, that answer is nobody.

---

What makes Matt's performance even more impressive is how scummy some of Google's policies have been, even as he came off smelling like roses. There are endless debates on how reasonable it is for Google to infer intent on links & so on, yet at the same time some of the past Google executive emails have quotes in them like:

http://www.washingtonpost.com/politics/google-faces-new-pres... "As with all of our policies, we do not verify what these sites actually do, only what they claim to do." (for AdWords advertisers)

http://www.theverge.com/2013/1/23/3906310/the-no-hire-paper-... "I would prefer that Omid do it verbally since I don't want to create a paper trail over which we can be sued later? Not sure about this. thanks Eric"

-----


Google's users are the product, the customers are the advertisers.

-----


"You should never know in advance a link is coming, or where it’s coming from. If you do, that’s the wrong path."

That is an absolutely ridiculous rule of thumb to follow. What exactly is wrong about doing something like:

1) Having a good article on your site about a certain topic 2) Reaching out to other site owners who have similar content and asking them to link to you as an additional resource 3) Getting that link.

Can you honestly, with a straight face, tell me that's BAD!?

-----


Can I, the potential reader, chime in? I hate this tactic because it results in lower quality content getting further up the search rankings. If I'm a gardener, I want the blogs of well-respected gardeners who have real-life community standing and mutual respect to link to each other. What I don't want is two amateur gardeners with crappy writing skills who have made side-deals to link to each other getting on the first page of a Google search.

The situation you mentioned can start out as innocuous but quickly become downright shady. Hmm, you're asking them to link to you, but what do they get in return? Oh, here's a thought, you tell them you'll link to them as well if they link to you, that way you'll both get a ranking bump! Totally fair and win-win for everyone except for the people who are trying to get quality information.

-----


As long as we are concerned about the organic structure of markets...

How do you feel about Google sponsoring educational seminars for market regulators where they do not disclose their sponsorship, actively conceal it, and request the entity putting on the sponsorship do the same thing?

http://www.washingtonpost.com/politics/how-google-is-transfo...

It is also worth mentioning that when Google was lobbying against regulation they indeed DID control & manipulate the placement of favorable media masquerading as regular content:

"the staff and professors at GMU’s law center were in regular contact with Google executives, who supplied them with the company’s arguments against antitrust action and helped them get favorable op-ed pieces published, according to the documents obtained by The Post."

---

Another thing with the line of thinking based on pureness...should companies which have repeatedly been caught rigging actual physical markets (e.g. municipal bonds, California energy, LIBOR, interest rate swaps, forex, etc.) in the real world get a penalty by Google for it?

Or is it reasonable that they take their outsized gains from their market rigging behavior & invest some small portion of it into buying out smaller competitors, lobbying & writing regulations to harm smaller competitors, buy some feel good brand ads & do various charity donations to paint a picture of themselves?

---

I also want to quote the following past statement from a Google executive about the AdWords ads Google puts directly in the search results:

"As with all of our policies, we do not verify what these sites actually do, only what they claim to do."

http://www.washingtonpost.com/politics/google-faces-new-pres...

---

and, some icing on the cake, ... ;)

check out the keyword rich deep links to some of Google's affiliate offers on BeatThatQuote.com (for credit cards, car insurance, etc.)

Google bought BTQ & then put a bunch of cross-site keyword rich deep links on it. is that natural? how is that any different than buying links?

-----


If I'm a gardener, I want the blogs of well-respected gardeners who have real-life community standing and mutual respect to link to each other. What I don't want is two amateur gardeners with crappy writing skills who have made side-deals to link to each other getting on the first page of a Google search.

That might be the ideal result, but it's hard to see how anyone could achieve it with a purely automated system. To make the kinds of distinction you're describing, in general you probably need some form of manual curation by genuine experts with uncontroversial opinions, assuming such people even exist in the field of interest.

-----


dchuk, that's a direct quote from Bing, not Google: http://www.bing.com/blogs/site_blogs/b/webmaster/archive/201... It's in the "Links are all I need" section. Here's the full quote:

"While important as a vote of confidence for the content they point to, there is simply so much link spam these days that it’s tough to know where to turn. Obviously buying links is a dead end, and it doesn’t matter how you split this hair: sharing, encouraging, incentivizing, buying – it’s all the same. You want links to surprise you. You should never know in advance a link is coming, or where it’s coming from. If you do, that’s the wrong path. Links are part of the bigger picture. You want them, but you want them to be natural. If an engine sees you growing tem naturally, you’re rewarded with rankings. If they see you growing them unnaturally, you’re rewarded with penalties."

If you disagree with that, you'd need to take it up with Bing's webmaster outreach team.

-----


Holy moly man, you were the one who provided that link: "It's not just Google that recommends to avoid link spamming. For example, at http://www.seroundtable.com/bing-links-knowledge-18535.html you can read where Bing said "You should never know in advance a link is coming, or where it’s coming from. If you do, that’s the wrong path."

Your statement implies both Google and Bing share the same sentiment.

So now, my question still stands: What about that scenario that I laid out is actually bad/spammy/evil? Why should anyone ever be punished for doing that?

-----


Why shouldn't I ask my customers to provide feedback on Internet? A business must encourage their customers to spread good word by various channels like social media, word of mouth, blogs, links, reviews, invite friends etc.

This practice has existing long before Internet, and internet has made it easier than ever. Google is essentially telling us not to ask our customers to evangelize for us.

Why such hypocrisy? How about the invites system Google uses for many products including Gmail where users are encouraged and incentivized to invite friends.

According to Google's terms if I link to my Dropbox referral link on my blog, Dropbox should be penalized for it because they incentivize users to do that.

The internet should not work according to Google's algorithm and policies, it should be the other way around.

-----


Great point!

Every marketing seminar I have ever been to has emphasised that as much as you can turn your clients into fans, in the end you still need to ask them for a testimonial and that you need to tell them that you love referrals.

-----


In other words, get your links by using a PR firm, not an SEO one,

-----


I'm sorry Matt, but that is a pretty silly thing to quote. After all, what's the point of building a strategic network, or making friends, or any other long, grueling white-hat tactic that takes forever to pull off?

Under that same logic, Google Adwords should never know in advance where their ad clients are coming from, and if you have someone out there trying to on-board big clients (or increase their spend) that's the wrong path.

You say "if they get caught" when sometimes, there is nothing to catch. You're dinging people who have grinded to forge relationships, in the name of trying to keep spammers out of the index.

It makes it too easy for me to hire a spam company to blast an overwhelming amount of crappy links at my competitors who may or may not be well equipped to disavow those links.

Which, if my company has the budget to hire someone to monitor & disavow, and my competition does not, well... somebody's still gaming the system...just not in the same way...

-----


They're not talking to you about link spamming.

-----


agreed, matt is talking in generalities at this point

-----


Matt is not reading what the discussion is about at this point.

-----


More specifically, I believe people feel they aren't being heard at this point. Rationalizations are the tell.

-----


But in this case they didn't spam. The op's site linked naturally to that site, and that site's admin got a message from Google that the op's link is unnatural (when it was natural).

That's what's wrong with this picture. The link was perfectly valid and yet Google recommended that site's admin to disavow the natural link, simply because Google's algorithm isn't good enough to see what's natural and what isn't. So his suggestion is: if it's not good enough, then don't recommend the wrong stuff to people.

-----


higherpurpose, nowhere in the original article did it say that Google said the link was bad. This was a request from a random site (we don't know which one, since the post dropped that detail), and the op can certainly ignore the link removal request.

-----


> Apparently Google convinced them, via their Webmaster Tools portal, that the link looked “unnatural”, and that they should use the Link Disavow Tool to discredit the link.

This is pretty typical of 2014 Google.

To be honest these days I pretty much regard Google as one of the banes of my existence, you guys hold so much power and influence so much of the internet the rest of us have to work around you, I'm currently pulling all our customers off Google Apps for Enterprise for a multitude of reasons (buggy code, broken imap implementations, impossible to get an actual human answer) as well as moving myself away from all Google services (including Android).

I remember when you guys where the good guys.

-----


Exactly. Instead Google should suggest that they investigate the link and appeal the way it was categorized if the search engine was wrong in its classification. Anything less is tampering with the natural way links should work.

-----


Yes, truly Google's rules held no sway over the SEO world until 2014.

-----


That's not what he's saying, and you know it.

-----


It's exactly what he's saying.

People have not only been complaining about this since Penguin came out (around two years now), before that they were complaining about the opposite: that Google's arbitrary rules were creating spammy links all over the internet. It's completely ignoring the past to pretend this is some new thing to fit a preconceived narrative.

Most of the SEO folks that seem to come out of the woodwork for these threads would agree with me that "This is pretty typical of 2014 Google" ignores years of them complaining about the exact same thing, except they're just happy to have more people on their side.

-----


For years we begged Google to simply stop counting links they didn't like. These included generic directory link, crappy blog links, article farm links and on and on and on. They claimed to be able to tell what was good and what was bad yet the search results rarely showed that in certain niches.

Then we stared seeing the first indications that real "negative" SEO was possible in certain circumstances and we complained even further. Stop counting links you don't like. Now we are at a point where we see old websites being told to remove or disavow links that may be as many as 10 years old.

Can Matt or Google really stand there and say that it took them 10 YEARS to figure out that not only was that a link that they didn't like, but it was so evil that now it needed to be removed from the internet? The solution has been, and always will be, stop letting things you don't like count as a plus in your algorithm. When in doubt, don't count it.

Instead, we have Google adding thousands of webmasters and site owners as a living part of the algorithm and getting them to remove all those links, plus many they can't see or understand the value of, by the use of fear and scare tactics. Remove these links are we will ruin your website rankings. It doesn't matter if you created or asked for them or not, remove them.

-----


Way to avoid the point. And to be honest, the whole oblivious act is getting ridiculous at this point. Don't post if you are going to play dumb. In case you're really that dense, a webmaster got a message FROM GOOGLE, that a perfectly legit link was spam. The provided email is their response to that.

The whole point is, Google spreads FUD to paper over its inadequate algorithm. People believe and look up to you as a source of truth. However, you take that trust, and you ask people to do your dirty work. That means more work for them, and not every small business has all sorts extra time money to spend on stuff like that. This is not to mention that there are probably a bunch of other more important things they could be doing. (and I know you know this, your friendly face is the head of FUD over at Google)

With all of your power and authority, you have a responsibility be more honest and straightforward. Not everyone knows better, and when people end up listening to your Google/self serving advice, they are ruining the open and free nature of the internet.

This wouldn't make me so salty, but the complete BS and favoritism exhibited by you and Google is obnoxious. So, sometimes, it might be better if you just shut up.

sorry if there are any typos etc, my (virtual) keyboard is spazzing out.

-----


> higherpurpose, nowhere in the original article did it say that Google said the link was bad

Yes, it most certainly did!

From the article:

> Apparently Google convinced them, via their [read: Google's] Webmaster Tools portal, that the link looked “unnatural”, and that they should use the [Google] Link Disavow Tool to discredit the link.

(emphasis added)

-----


You emphasized Google, but you should have emphasized apparently, because nowhere in the actual email is the OP's conspiracy theory about Google mentioned (or even the word Google!). It's quite likely they noticed a penalty and are just going through all their back links to find shady content.

-----


Sure, this is all hearsay and second- and third-hand information, and the article is really sparse on details.

> It's quite likely they noticed a penalty...

Yes, and where would they have noticed that penalty?

-----


> Yes, and where would they have noticed that penalty?

In a drop in incoming traffic, so with whatever they're using for analytics.

In all honesty it's probably just a new SEO consultant trying to clean up past sins and move on with whatever the latest grey hat strategy is.

IMO it's a gross overreaction to a form letter that a high percentage of webmasters have received.

-----


It's both Google trying to force web sites to help their algorithm and spammy sites trying to exploit the current algorithm that are causing these annoyances.

I suspect neither would be a major issue if Google modernized their algorithm to take into account (mostly/only) which links people actually clicked on rather than what is linked from where. They'd only have to be able to distinguish people from click-bots, which might be easier to do with some confidence than recognizing the intent behind a particular link (spam/SEO or real information for visitors) - I hope they are already able to do this, for the sake of AdWords customers ...

-----


> The model (google's algo) should fit reality. Rather than forcing reality to fit the model.

Google's algorithms should reward behaviour that's best for Web users, and penalise behaviour that is worse for Web users.

That way, for people who want to game search engine rankings, they are pushed more towards techniques that benefit web users in the course of trying to benefit themselves.

The problem with models is that they are not accurate simulations of the real world. They have flaws and simplifications, and those give rise to people taking advantages of those flaws. All models and representations have flaws.

-----


> The model (google's algo) should fit reality. Rather than forcing reality to fit the model.

It is impossible for Google to exist at its current scale and not influence the Internet.

The tiger is born out of adaptation to the jungle, but in time, the jungle too will adapt to the tiger.

-----


The Internet would naturally be a tragedy of the commons - and entropy fighting is something Google seems to be doing well.

-----




Applications are open for YC Winter 2016

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | DMCA | Apply to YC | Contact

Search: