Hacker News new | past | comments | ask | show | jobs | submit login

Only someone poking about would ever hit that url on someone else's domain, so where's the downside?

And "a lot" of false positives?? Recall, robots.txt is set to ignore this, so only malicious web scanners will hit it.




The downside is that you ban a whole ISP because of a single user misbehaving.

Personally I sometimes do a quick request to /wp-admin to check if a site is WordPress, so I guess that has a nonzero chance of affecting me. And when I mirror a website I almost always ignore robots.txt (I'm not a robot and I do it for myself). And when I randomly open robots.txt and see a weird url I often visit it. And these are just my quirks. Not a problem for a fun website, but please don't ban a whole IP - or even whole ISP - because of this.


Well you make a point, I use ipset in many circumstances, which has an expire option.

So that is a balance between a bad actor and even "stop it" blocks, and auto expire means transitory denial.


Do you own your ASN or unique IP? Do you like getting banned for the actions of others that share your ASN or IP?


what chance are we even talking of a false positive?




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: