Hacker News new | past | comments | ask | show | jobs | submit login
The story behind Google's crawler upgrade (anand.typepad.com)
12 points by neilc on April 13, 2008 | hide | past | favorite | 1 comment



very interesting, but didn't Matt Cutts say he suggested you didn't rely... on google crawling through your forms and indexing that information. The best way to get your site fully indexed (which is the purpose of the crawler) is to have a open site, with information that is not behind any logins or forms.




Consider applying for YC's Fall 2025 batch! Applications are open till Aug 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: