
Ask HN: What countries are safe for web scraping? - alexmarin
If I wanted to start a site that scrapes listing data from another site.  And provides analytics, market intelligence.  The source site has TOS that disallows scraping.  And they have shutdown another scraping site previously.  The source site is not big and I’m providing value and aligned with their mission but seems they don’t care.<p>I want to host it in a country that is outside of their jurisdiction just to be safe from cease and desist&#x2F;getting sued.<p>Anyone know favorable countries for web scraping?  Or what other methods I can take to protect?
======
dangerface
As far as I know there is no legal recourse against scrapping even if they
specifically disallow scraping, people have tried to sue google for scraping
and lost.

You using the data will break copyright in pretty much every country, it
doesn't matter how you got the data it's how you use it. They will sue you for
copyright breach and most likely they will win. There is no legal loophole
even for people with googles money they end up paying for a licence for the
data or remove it.

~~~
infinitone
Then how come companies like [https://www.airdna.co/](https://www.airdna.co/)
continue to operate?

~~~
tjkrusinski
Simply because they have that data doesn't mean they sourced it illegally.
Their value is likely the way in which they assemble disparate data sets,
obtained with permission or purchased.

------
sebst
First, it's their site. If they don't want it to be scraped (even if their
reasoning sounds silly to you), I'd advise you to obey their IP. (That's the
moral pov)

Second, IANAL, but the hosting location should not matter as much as you might
think (YMMV depending on your jurisdiction): It's you who is responsible for
the (probably unlawful) scraping. So, they will sue you in your jurisdiction,
not the machines hosted anywhere. So, your question boils down to finding a
hosting provider which basically ignores abuse complaints and protects your
anonymity. (That's [probably] the legal pov)

Third, even if you found a provider and they are unable to effectively take
your site down and/or sue you, they might just block the IPs they identified
the scraping originates from. So you'd have to use IPs from different subnets
to obfuscate what you're doing. (That's the technical pov)

So, in conclusion, why not scrape their site, build whatever product you have
in mind on top of their data and show it to them (and only to them) to arrange
a deal. More often than not, you can convince people more easily when they can
visualize what you're talking about. So, show them your prototype.

If they still reject a collaboration with you, they might have their reasons.
It's not your cup of tea to judge them for their business decisions, then.
Better find another business partner who matches your views and ideas.

tldr; Why would you build a business on a never-ending fight with someone you
assume you share a mission with?

------
100100010001
Alter the scrapping so it looks like normal activity.

------
quickthrower2
Get into a less dodgy business

------
duxup
>providing value and aligned with their mission

What does that mean exactly?

------
thedevindevops
Could you maybe do whatever-it-is-you're-planning-to-do-with-the-scraped-data
under the guise of a review site?

------
rolph
how about providing a security service [APP] that inspects the web scrape made
by the client browser, to provide awareness of the sites security/threat
properties ? Its semantic but scraping to protect users should be different
from scraping just to snoop a site. IANAL so the line between hacking, and
threat model identification should be investigated somehow.

And of course, the development of such a system is a time and cost concern.

