

Ask HN: Who uses web scraping? - AlexMcP

A few survey questions about collecting data via the web:<p>Do you have experience with people using web scraping commercially?<p>What field or industry was it being used in?<p>Do you think it fell in a 'grey' area, or was it explicitly disallowed or allowed?<p>Thanks for any info!
======
ankitml
Yes I had done web scrapping commercially, sometime back when I had just left
college. I had to scrap website(s) for getting data about all pharma products
available in india. So basically market research. But as soon as I realized
this was against TOS of the website, i deleted all data and communicated this
to client. I had to discard the contract and return even the previous
payments. Although I am happy that I did. :)

~~~
ihopngo
Eventbrite used scraping in the early days. They first argued it was a
publicly available website. "Public" in the sense that the site was browse-
able without restriction.

The plaintiff responded that it had anti-scraping language in its user
agreement. The court said "the Terms of Use are not displayed on the website
in any way in which a reasonable user could be expected to notice them."

------
sheraz
[1] - Yes

[2] - Lead generation, selling data, consumer + market research

[3] - I think it is always a gray area with regards to another site's TOS. But
the burden of proof is on them, not the scraper. That and masking your
scrapers is too easy.

