

Google Banishes Nicholas Carr From The Index - quoderat
http://www.roughtype.com/archives/2009/06/banished.php

======
bwd
There is now a comment on this blog entry purporting to be from someone on the
Google web search team claiming that the site was removed because it had been
hacked and that Google had attempted to contact the author.

~~~
gojomo
The 'hack' must be very, very minor: I can still visit with Chrome, which
supposedly checks Google's 'Safe Browsing' blacklist.

Maybe it's just hidden spam links? In which case, index-removal seems an
extreme way to get the owner's attention.

~~~
AdamLasnik
Hi there,

I realize neither this comment nor the one on Nick's blog from my Google
colleague Greg are authenticated, so you'll just have to trust me that I'm me
:).

We do often remove entire domains from our index when there's evidence of
hacking and/or malware... not to be mean, but rather to protect both visitors
to that site and the site owner (from angry visitors). We can't be sure that
the hacked links aren't or won't soon become particularly malicious (e.g.,
phishing sites), nor can we be certain that the server hasn't been or won't
soon be more substantially compromised (with drive-by malware installs, etc.),
so we fall on the side of caution.

We do make a good faith effort to contact the webmasters affected... \-
Through e-mail addresses we believe to belong to that webmaster. \- Via our
free Webmaster Tools service, where webmasters can sign up and opt to get
e-mails from Google on just such occasions as this.

For more info, you can check out our most recent blog post on this topic:
[http://googlewebmastercentral.blogspot.com/2009/02/best-
prac...](http://googlewebmastercentral.blogspot.com/2009/02/best-practices-
against-hacking.html)

Hope this helps! Believe me, we want good content back in our index as soon as
possible, too :)

~~~
gojomo
Thanks for the response and link.

The "could get worse" standard is a slippery slope, unless the evidence
suggests a very specific serious compromise or foreign active content.
(Exiling a site because a few evil links appear in its comment threads, for
example, would put almost every site at risk of removal. Indeed, that standard
risks gaming by a site's rivals, using comment forms rather than 'hacks'.)

Also, looking at Carr's last comment, even though he's removed the bad links,
he doesn't know what compromise allowed them to appear. So his site is _still_
in the "can't be sure... won't soon become particularly malicious... [or]
substantially compromised" category that justified its initial removal.

Your deep and sensitive checks for problems are definitely a public service...
as long as the standards are clear and communications effective. Maybe
notification should occur via a site's own comment functionality in addition
to email? No details for third parties to exploit -- just a "please check
Google Webmaster Tools" note.

~~~
AdamLasnik
Hey Gojomo, appreciate the input!

I definitely hear you on the slippery-slope issue, and perhaps I could have
worded my initial note more carefully. We have reasonably conservative
heuristics in place to assess whether a site has been actually hacked vs. just
someone adding, say, a link to a nasty site in a comment. Also, in our
experience, once a site or server's been compromised, there's a good chance
that, well, where there's smoke, there's fire.

Regarding giving more information... I believe we do generally show specific
examples re: hacking and malware via our Webmaster Tools, though I can't say
for sure what we presented in Nick's case. Generally, we've seen that
webmasters are able to catch the root vulnerability (via their own unpatched
software or via their server), which prevents the same problem from recurring.
(and speaking of unpatched software... from my own spot checks, that seems to
be one of the most major culprits!)

Lastly, re: your suggestion about notification on the site's comment
functionality itself... that's a fascinating idea! I think that it'd be a bit
challenging to implement, given the diverse array of registration
requirements, captchas, etc. (our bots are smart, but not as smart as
humans!), but I know that our team is looking into more ways that we can let
webmasters know of issues more effectively. Thanks for the suggestion!

------
sfphotoarts
The comments on the article explain why, Google detected that the site had
been hacked and had email the author of the site.

~~~
jf
Nitpick: The host itself wasn't hacked. Part of his website had a vulnerable
MT installation that allowed spammers to insert spam into that part of this
website.

~~~
jonknee
That much is still unknown, he replied to say he was unsure how the links got
there.

------
olefoo
Curious, the search <http://www.google.com/search?q=roughtype.com>

Brings up all sorts of URL's that contain the string roughtype.com but are not
the site itself.

It's frightening, especially since we have no way of knowing what else is
being filtered out and for whom.

I would tend to give the big G the benefit of the doubt; it's a mistake or he
got on the wrong side of the algorithm, or his site was infected with malware.
(that last one does happen see <http://blog.oddhead.com/2009/06/22/un-hacking-
my-blog/> )

But still, it is the reputational equivalent of logging in to your brokerage
account to find out it's empty.

~~~
quoderat
No way to know for sure, but it most likely occurred because of this:
[http://www.roughtype.com/archives/2009/05/for_whom_the_go.ph...](http://www.roughtype.com/archives/2009/05/for_whom_the_go.php)

~~~
tjic
Why do you assert that it "most likely occurred" because of this?

Do you have any data?

------
charrington
Wow! This is crazy. A similar thing happened to my friend Matt Harrison a
while back. Here are the relevant links: \- <http://panela.blog-
city.com/burninated_by_google.htm> \- <http://panela.blog-
city.com/back_in_google.htm> \- <http://panela.blog-
city.com/google_hates_me.htm>

As Matt explains, he was re-added to the index with no explanation. Perhaps
Nick Carr will have a similar redemption.

~~~
anigbrowl
As indeed he has (see last comment on OP link).

