A tricky problem all search engines face, regardless of scale and popularity, is dealing with bot spam, of which there is a lot [1].
It's not uncommon to face quarter of a million bot-searches per day and it's really difficult to actually combat since they're on a botnet, without some sort of privacy invasion, either via cookies or something like cloudflare.
It's not uncommon to face quarter of a million bot-searches per day and it's really difficult to actually combat since they're on a botnet, without some sort of privacy invasion, either via cookies or something like cloudflare.
[1] https://news.ycombinator.com/item?id=31395231 (I can attest to this pattern as well through my search engine)