

Simple web-crawler for Node.js - DamonOehlman
https://github.com/cgiffard/node-simplecrawler

======
zoowar
Show some respect for the web and add robots.txt and sitemap.xml support.

