
Ask HN: Why developers hate scraping? - dogreborn
Over time we&#x27;ve built products like : adplexity.com (5million USD annnul revenue, 100K annual expense)<p>But why aren&#x27;t there more developers interested in scraping products?
======
onion2k
The legality is complicated, the tech challenges are hard, and, to be honest,
it just feels a bit _off_ to profit from other people's work without giving
back to them.

Question for you - why do you care what people think if you're making $4.9m
profit a year?

~~~
dogreborn
In absense of competition, life is pretty boring.

A scraping company can only pickup limited number of ads from limited
profiles.

If more companies start scraping, scam and advertisers will be caught faster
and prosecuted once vigilantes use these tools to report fraudlent ads.

------
rajacombinator
5mil/yr on 100k/costs sounds pretty interested to me ... what exactly are you
complaining of? Not enough API users? When I click your API tab, it just has a
“contact for pricing” pop up. You’d probably get more interest if you had open
documentation/some examples of use cases/upfront pricing.

------
deytempo
The fun part is knowing you are being scraped and adding tagged data values
that you can scan for elsewhere on the web to see where your scraped data is
going

------
wprapido
Says who? I particularly enjoy it and find it extremely entertaining and a
great opportunity to brush up my skills, while making some money in the
process.

------
sh87
1)Legality 2)Morality

~~~
dogreborn
2\. About morallity, ad tech companies are hyper targeting people with ads
which some evil advertisers take advantage of and bypass moderation.

Ads are targetted in such way that expert users and regulators do not see them
but the naive ones get tricked.

If we scrap ads using different visitor profiles and list all ads, along with
offensive/malware ads... wouldn't it help regulators moderate ads?

So is it still morally wrong to go against ad tech companies by scraping their
scam/tricky ads?

~~~
sh87
I was referring to scraping content, not ads. Not sure what benefit ad
scraping provides and to what end ?

~~~
dogreborn
1\. Sell data to ad regulators

2\. Sell data to competitors of the advertisers, turning online advertisement
less effective in long term and easier in short term.

3\. Sell ads data to activists etc...who will sue the company running
misleanding ads and defrauding consumers.

~~~
sh87
I did not know there was a market for this.

------
dogreborn
Also what's the latest tech stack for easy scraping?

Most new developers are not able to scrap using regex, css selectors etc... So
is there any new advancement in scraping field?

