On: AI crawlers need to be more respectful

[Saris]

Yeah that's an insane price for bandwidth, they need to move providers ASAP if that's the kind of fees they get.

[bastawhiz → Saris]

Hosting documentation shouldn't need that much bandwidth. It's text and zip files full of text. Without bots, that's a very small cost even if the bytes are relatively costly.