| ||Ask HN: What info do you web scrape for?|
129 points by cblock811 1166 days ago | hide | past | web | 110 comments | favorite |
|I have been web scraping for several months and am starting to teach it at Meetups. I'm lucky enough to work for a company that has a few pre-crawled copies of the web that I can query against and a distributed processing platform to speed up any scraping I do.|
So if you had these resources, what would you be looking for? I love building tools that help people so any feedback/ideas would be great!
I'm also open to hearing what you would scrape for on the live web. I find that if I'm doing broad analysis then the pre-crawled copies are best, and for specific sites/information I use the live web.
| Apply to YC