> DuckDuckGo gets its results from over four hundred sources. These include hundreds of vertical sources delivering niche Instant Answers, DuckDuckBot (our crawler) and crowd-sourced sites (like Wikipedia, stored in our answer indexes). We also of course have more traditional links in the search results, which we also source from a variety of partners, including Oath (formerly Yahoo) and Bing.
AFAIK Oath / Yahoo has switched to using Bing under the hood since 2009: https://www.nytimes.com/2009/07/30/technology/companies/30so...
Kind of disheartening regardless. I assumed they had their own scrappy, independent tech stack.
For one thing, loads of sites load all their content via Ajax, so at a minimum you're gonna need a browser engine as the base of your crawler...
Doing all these would allow you to surface better results than using just one. But it comes at a cost.
Yandex is Russian and has pretty poor results. And you’d be naive to think that those of us concerned about privacy would ever touch something built on top of it.
In other words there aren’t paid search engines that DuckDuckGo could turn to. Unless they build their own crawler, the only game in town is Bing.
And yes you can compete with Google (for a certain target group) by paying them for their search results. Results are just a distribution channel, it is what you do with them that matters. For example Google and DuckDuckGo both choose to show you ads and affiliate links but that is hardly the only option.
Sorry, but I don't trust DuckDuckGo that much and when technology fails, I'd rather use a search engine based in a country that has rule of law ;-)
They're desperate to collect user click data because they know that's the only way they'll have any chance of success. Even anonymized, that's very valuable data.