Hacker News new | comments | show | ask | jobs | submit login

When I've tried that on larger GeoCities sites (say >100 documents), the servers seems to shut me down after a while :-(

Try the --limit-rate and --wait command line options to rein in the voraciousness of the download request. You can also use the -U option to pretend you are a browser.

So the command becomes something like:

    wget -r --limit-rate=20K --wait=20 -U 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv: Gecko/20070802 FireFox/3.5.4' http://geocities.com/tablizer



Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | DMCA | Apply to YC | Contact