What's ironic is that most of the sites with anti scraping protection also do sc... | Hacker News

Hacker News new | past | comments | ask | show | jobs | submit

login

ikeboy on July 10, 2018 | parent | context | favorite | on: Lessons learned scraping 100B product pages

What's ironic is that most of the sites with anti scraping protection also do scraping of their own.

E.g. Amazon and Walmart both do a lot of their own scraping.

Doctor_Fegg on July 10, 2018 [–]

Really going to call for a [citation needed] on that "most"!

ikeboy on July 10, 2018 | [–]

Maybe I should rephrase to “put the most effort into anti scraping”.

Every major ecommerce site scrapes, it would be a competitive disadvantage if they didn’t.

Join us for AI Startup School this June 16-17 in San Francisco!
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact