The downside is that you ban a whole ISP because of a single user misbehaving.
Personally I sometimes do a quick request to /wp-admin to check if a site is WordPress, so I guess that has a nonzero chance of affecting me. And when I mirror a website I almost always ignore robots.txt (I'm not a robot and I do it for myself). And when I randomly open robots.txt and see a weird url I often visit it. And these are just my quirks. Not a problem for a fun website, but please don't ban a whole IP - or even whole ISP - because of this.
And "a lot" of false positives?? Recall, robots.txt is set to ignore this, so only malicious web scanners will hit it.