Hacker News new | past | comments | ask | show | jobs | submit login

I quickly checked for you, and my Cloudflare account currently has the following option: "Block AI Bots" under xxx.com > Security > Bots which I think does what you want.

I am not paying for Cloudflare and it allows me to enable this option. If you use the (other) Bot Fight Mode or captcha options, then yes, you will block crawlers. AFAIK specific bot user agents can also be blocked (or allowed) using a Page Rule, which is also a free tier feature.






Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: