On: Crawling - The Most Underrated Hack

Mechanize is excellent. If you'd like to do some scraping but prefer to work in Python, check out Scrapy[1], discussed[2] a while back and only getting better. I've been very pleased with it for some personal projects that rely on scraping.

[1]: http://scrapy.org/

[2]: http://news.ycombinator.com/item?id=411733