
DuckDuckGo returns 500 if user agent starts with “Java” - oh_no_not_again
https://pastebin.com/yEKrcVUJ
======
oh_no_not_again
I wonder why. My guess is to prevent bots from aggressively scraping site
generating useless load, but I failed to find any other user agent strings
that lead to this behavior. Any suggestions?

~~~
duskwuff
Because it means you're a bot with a default user agent, and they've probably
had some bad experiences with those.

Well-behaved bots use descriptive user agents.

~~~
oh_no_not_again
Right. But why Java default user agent, and not Python request's, libwww-perl
or curl itself? Is there disproportionately more bad bots written in Java?
This is not-intuitive to me, so that's why I posted this :)

