
Using Sitemaps in Web Scraping - webrobots
https://webrobots.io/using-sitemaps-in-web-scraping-robots/
======
akriukas
Very useful for some big eshops if they list their sitemaps. However not all
stores have sitemaps and even robots.txt site...

------
HackerDalius
Good article, useful general tips that work for many scraping tools and
platforms.

