
Ask HN: Web Scraping with Node.js - ErikAugust
Anyone out there have a cachet of good Node web scraping libs, and&#x2F;or other resources? Opinions?<p>I have the intuition that my Google searches are a bit inadequate. Thank you!
======
iends
Check out
[https://github.com/cheeriojs/cheerio](https://github.com/cheeriojs/cheerio)

------
tmpfs
I found [https://github.com/cgiffard/node-
simplecrawler](https://github.com/cgiffard/node-simplecrawler) to be good, I
used it for
[https://github.com/tmpfs/linkdown](https://github.com/tmpfs/linkdown) \- a
cli for validating web pages etc.

------
jneumann004
I haven't used this for web scraping, but have you tried out
[http://phantomjs.org/](http://phantomjs.org/). It's a great tool that lets
your process webpages; I don't think it would be that difficult to set it up
to scrape sites.

------
paulzerkel
Here is a place to start: [https://github.com/lorien/awesome-web-
scraping/blob/master/j...](https://github.com/lorien/awesome-web-
scraping/blob/master/javascript.md)

You might want to check out the Nightmare project
([https://github.com/segmentio/nightmare](https://github.com/segmentio/nightmare))
which makes automating an Electron instance pretty simple.

------
pesfandiar
Should it be necessarily with Node.js? I've tried to find a good toolset in
JS, but it seems other languages (e.g. Python and Java) are a lot more mature
in scraping.

------
Nilef
Try this out [https://github.com/IonicaBizau/scrape-
it](https://github.com/IonicaBizau/scrape-it)

