| ||Ask HN: What info do you web scrape for?|
129 points by cblock811 on Aug 11, 2014 | hide | past | favorite | 110 comments |
|I have been web scraping for several months and am starting to teach it at Meetups. I'm lucky enough to work for a company that has a few pre-crawled copies of the web that I can query against and a distributed processing platform to speed up any scraping I do.|
So if you had these resources, what would you be looking for? I love building tools that help people so any feedback/ideas would be great!
I'm also open to hearing what you would scrape for on the live web. I find that if I'm doing broad analysis then the pre-crawled copies are best, and for specific sites/information I use the live web.
Applications are open for YC Summer 2021
| Apply to YC