On: AI crawlers haven't learned to play nice with websites

[extraduder_ire]

Did any other crawlers hit it? Would be nice to have someone keeping track of this over time and publishing the data online somewhere.

Probably need to collect logs across multiple sites though, as some crawlers would exclude your domain once they knew.

[blueflow → extraduder_ire]

Only one other hit,

2025-03-05 13:32:09 45.89.148.57 (AS46844 SHARKTECH) "Mozilla/5.0 (Windows; U; Windows NT 6.1; pt-BR; rv:1.9.2.18) Gecko/20110614 Firefox/3.6.18 (.NET CLR 3.5.30729)"