If you have a noble intent - identify members of fascist organizations - then obviously when you ask the top online fascist sites if you may scrape them to build up your list of online fascists - they will say no.
OK less provocative, you have new algorithm to identify inaccessible websites, your automation is scary good, crawling a site you can identify many issues that most sites would have to pay for a full audit to get, but now these sites have problems - if you can identify their sites as being inaccessible then they have to fix these problems due to various accessibility standards that apply in the regions they operate in. But if they don't allow you access then they can maybe make an argument they are accessible due to audit they did last year, at any rate they don't want to be forced to spend money on accessibility issues right now which it sounds like they might have to if they let you crawl their site.
Version 2 of above, some years ago I spoke about a job with a big time magazine publisher in Denmark and said one of the things that would make me a good employee is my knowledge of accessibility and their chief of development said they didn't have anyone with disabilities that used their site - so if I ask that guy to crawl their site why say yes? They have no users that would benefit!! Stop abusing our bandwidth bleeding heart guy.
All of these seem like variations of the-ends-justify-the-means, which generally tends to cut both ways in unanticipated ways.
Bullying websites into accessibility compliance will most likely lead to them following the letter of the standard without giving a second of thought as to whether the content is in fact actually accessible. It's very difficult to get someone on board with your cause if your initial contact is an antagonistic one.
OK less provocative, you have new algorithm to identify inaccessible websites, your automation is scary good, crawling a site you can identify many issues that most sites would have to pay for a full audit to get, but now these sites have problems - if you can identify their sites as being inaccessible then they have to fix these problems due to various accessibility standards that apply in the regions they operate in. But if they don't allow you access then they can maybe make an argument they are accessible due to audit they did last year, at any rate they don't want to be forced to spend money on accessibility issues right now which it sounds like they might have to if they let you crawl their site.
Version 2 of above, some years ago I spoke about a job with a big time magazine publisher in Denmark and said one of the things that would make me a good employee is my knowledge of accessibility and their chief of development said they didn't have anyone with disabilities that used their site - so if I ask that guy to crawl their site why say yes? They have no users that would benefit!! Stop abusing our bandwidth bleeding heart guy.