

HN Banned from Google - jayjay1010
http://linkaudit.co.uk/blog/hn-banned-from-google/

======
nbpoole
Luckily, this question has been asked and answered fairly recently!

Specifically
[https://news.ycombinator.com/item?id=5955043](https://news.ycombinator.com/item?id=5955043)
from last week, complete with a response from Matt Cutts
([https://news.ycombinator.com/item?id=5955374](https://news.ycombinator.com/item?id=5955374)):

 _> It's not that PG has a grudge against Google (or vice versa) or anything
like that. I believe that search engine bots crawl Hacker News hard enough
that PG blocks most crawling by bots. In the case of Google, he does allow us
to crawl from some IP addresses, but it's true that Google isn't able to
crawl/index every page on Hacker News._

 _> Here's a link where I answered the same question about three weeks ago:
[https://news.ycombinator.com/item?id=5837004](https://news.ycombinator.com/item?id=5837004)
, so this isn't a new issue. In fact, PG has been blocking various bots since
2011 or so;
[https://news.ycombinator.com/item?id=3277661](https://news.ycombinator.com/item?id=3277661)
is one of the original discussions about this._

 _> And to show this isn't a Google-specific issue, note that Bing's #1 result
for the search [hacker news] is a completely different site,
thehackernews.com:
[http://www.bing.com/search?q=hacker+news](http://www.bing.com/search?q=hacker+news)
_

_> In general, I think PG's priority is to have a useful, interesting site for
hackers. That takes precedence and is the reason why I believe PG blocks most
bots: so that crawling doesn't overload the site._

~~~
jayjay1010
Well I can't no longer find the site by searching for the brand anymore and
I'm sure this will dramatically change the traffic the site gets.

~~~
thaumaturgy
Good.

------
pg
I'm sure it's just automatic. Their sw thinks HN is dead because it stops
giving their crawlers pages after a while.

~~~
jayjay1010
Specifically your no longer ranking for your main brand! That's nothing to do
with your long tail search terms.

~~~
tptacek
Whoah, you're right! If you search for [Hacker News], you get a bunch of junk
results now. That's so cool!

------
cddotdotslash
I clicked the links and every link on the frontpage is to different Hacker
News pages. I see nothing wrong here.

~~~
jayjay1010
It's wrong bc it's NOT bringing up the best result for the query.

~~~
greenyoda
I'm getting the welcome page for HN as my third result on the "hacker news"
(without the quotes) search:

[http://ycombinator.com/newswelcome.html](http://ycombinator.com/newswelcome.html)

This is arguably the best result for the query, since it explains what HN is
about and provides a link to the front page. (People who have already used HN
know where it is and don't need to search for it.)

I block cookies from Google, so they're not relying on my search history to
give me relevant results.

And I'm not sure why searching for "HN" would be reasonably expected to get
someone to Hacker News. After all, "hn" is the top-level domain for Honduras,
and could stand for a million other things as well. It's probably accidental
that it ever worked at all.

------
muuck
His Hacker News query give me
[https://news.ycombinator.com/jobs](https://news.ycombinator.com/jobs) back as
first result. Just for fun I tried the same query on DDG and the 27th result
was the first ycombinator domain I got back.

~~~
lnanek2
Maybe jobs have non-session/continuation based perma-links? That would allow
them to work even for things like the Google Crawler.

