Hacker News new | comments | show | ask | jobs | submit login

> Is there a way to archive geocities pages like this one?

    wget -r http://geocities.com/tablizer



When I've tried that on larger GeoCities sites (say >100 documents), the servers seems to shut me down after a while :-(


Try the --limit-rate and --wait command line options to rein in the voraciousness of the download request. You can also use the -U option to pretend you are a browser.

So the command becomes something like:

    wget -r --limit-rate=20K --wait=20 -U 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.6) Gecko/20070802 FireFox/3.5.4' http://geocities.com/tablizer


Thanks!


Thanks




Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | DMCA | Apply to YC | Contact

Search: