

The story behind Google's crawler upgrade - neilc
http://anand.typepad.com/datawocky/2008/04/the-story-behin.html

======
redorb
very interesting, but didn't Matt Cutts say he suggested you didn't rely... on
google crawling through your forms and indexing that information. The best way
to get your site fully indexed (which is the purpose of the crawler) is to
have a open site, with information that is not behind any logins or forms.

