| ||Ask HN: How do you feel about web scraping?|
141 points by jackschultz on June 6, 2017 | hide | past | favorite | 163 comments |
|Most sites seem to have ToS saying that scraping is not allowed, but in lots of cases that shouldn't be the case. If you see that, does it change your opinion?|
Does the purpose of your scraping make a difference, if one use is just a project but another would be selling the data?
Are comments on sites like this public data or private?
What about sports statistics that sometimes are "private" by the league, rather than just open for people to use and write interesting articles about it?
I actually created a site (I edited and deleted the mention of the name here because apparently people don't believe that I don't care about people looking at the site.) that scrapes comments and posts from Reddit that link to Amazon products and shows that information. I'm thinking of adding Amazon links from HN and other sites, but just not sure about how people would feel about scraping from sites like this.
| Apply to YC