Http://www.google.com/bot.html = this page no longer available | Hacker News

Hacker News new | past | comments | ask | show | jobs | submit

login

		Http://www.google.com/bot.html = this page no longer available (google.com)
		1 point by gstar on Sept 10, 2008 \| hide \| past \| favorite \| 2 comments

gstar on Sept 10, 2008 | [–]

Crikey - I thought Google of all places wouldn't stuff up the user-agent bot webpage.

Google's user-agent is: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html

I mean, everyone has heard of Google - but it's very bad mannered not to have a page up about your crawler.

ScottWhigham on Sept 10, 2008 | [–]

Its up now

Consider applying for YC's Spring batch! Applications are open till Feb 11.
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact