Does DDG just generate lots of Bing searches and scrape the result? If so, why wouldn't MS block them?
Or did they strike some sort of deal? If so, how can that be in Bing's interest? They're just creating a Bing competitor, right? Given that Bing reasonably wants to be the #2 search engine, supporting a formidable competitor striving for the same spot doesn't sound like a winning strategy for Bing. Right?
What am I missing? :-)
In addition, DDG serves ads from the Microsoft ad network, so they will also make money there.
Also, worth noting that Google provides the same type of service through API. I built a search engine  where Google web results are one of many sources used.
If all the users go to DDG.com, it won't matter if Bing has a great backend or not.
I've recently lived this experience in another context. We built a great backend for a partner that was good at publicity. When they redirected users to their internal backend, we were dead in the water. It didn't matter that the partner struggled with their backend and ours was better. We hadn't invested resources in independent publicity, so users couldn't find us. It was a hard lesson to learn.
People in tech circles always say DDG has a crawler. I know they do. No one ever says how big it is. I have a hard time believing it’s crawling more than a couple percent what Bing is crawling.
One complicating matter is that it specializes in DDG's "Instant Answers"; apparently¹ the "source from other search engines" approach really only applies to the traditional results list, while DuckDuckBot is focused specifically on the Instant Answers, effectively optimizing for the absolute most relevant first result possible. So in its current state DuckDuckBot ain't exactly comparable in purpose.
Still, I don't see much that'd be stopping DDG from ramping this up and having it generate the traditional list of results, too. Yeah, obviously the results for quite a few queries will be far from ideal, but if the query already hits one of DDG's curated data sources, I suspect the results might already be better if anything.
You think they don't have some tight contractual relationship with built in renewals and extensions? This is really similar to how corporations lock up retail real estate locations. They don't sign a 1 or 2 or even 5 year lease and then hope the landlord doeesn't rake them over the coals when there is no where else to go to. (Which could often be the case..) You build in renewal options as part of the deal. Those can last a long long time. May even have (probably does) a 'right of first refusal' and so on.
So if DDG degraded in quality I'd be fine, because I don't expect perfection. Though I suppose it depends on how much, of course.
DDG used the Yahoo Boss API  originally. It was pretty good, but Yahoo shut it down and Bing now also powers their search engine.
In July 2009, Microsoft and Yahoo! announced a deal in which Bing would power Yahoo! Search. All Yahoo! Search global customers and partners made the transition by early 2012. The deal was altered in 2015, meaning Yahoo! was only required to use Bing for a "majority" of searches.
I know for sure people have said DDG is better over time. I can’t imagine it was very good in 2008 or 2009.
Note: I know Yahoo is Bing behind the scene - I'm just saying the technical implementation at the time was not tied exclusively to "bing.com".
2016 is the year Yahoo was sold to Verizon. 2 years later, Verizon writes down $4.6B of the AOL Yahoo subsidiary. A year after that they sell Tumblr in a fire sale. Verizon stopped caring about their web media empire when they did the write down/sale too. Effectively throwing in the towel as a major web advertising hub even though they are likely still top 5 in the west after Google, Facebook, Microsoft, Amazon. That’s peanuts to Verizon though.
I can’t imagine Verizon cares much about their deal with DDG. If Microsoft cared, it makes sense that they could have Verizon and Yahoo do what they want over lawsuit worries from DDG.
I’m being pedantic at this point and totally guessing with my assumptions. I admit the continuous praising of DDG on the web has gotten to me and likely making me feel and react this way.
2) MSFT has clauses in its contract to address this. If a commenter on HN thinks of this one minute after reading the article, you can be sure that their legal team has thought about it too.
Would ad revenue for 10,000 interactions actually offset this cost for DDG?
It seems runnaroo would therefore be limited to 10k searches/day?
> You can’t beat Google when it comes to online search. So we’re paying them to use their brilliant search results in order to remove all trackers and logs. - https://startpage.com
Does anyone know how one can get whatever deal Startpage has?
Does this API still return paid results near the top?
I’m talking about the majority of these people who outside of basic ad/tracker blocking and using DDG, don’t stick to caring about privacy above most other things.
A lot of what I’m saying doesn’t count for HN’s crowd as much as other bigger sites.
I don’t have any backing for this statement. Just what i believe from common sense looking around.
Bing also gives Microsoft:
- a relational knowledge graph
- a way to annoy Google
"Annoy Google" is a "commoditize the complement" play.
Bing (and the Bing powered Yahoo + DDG) create a viable alternative where otherwise there is none. Google now has to constantly spend to keep their search tech ahead, and Mozilla and Apple can now threaten to switch default search engines forcing Google to pay protection money. Altogether, Bing forces Google to burn around 15-20 billion per year. This is money Google can't use to poach Microsoft engineers, or push G-Suite against Office, or Chromebooks against Windows, or Google Cloud against Azure.
Microsoft already has open sourced many key Bing technologies, and actively pays consumers to search with Bing. The primary goal isn't revenue, it's commoditizing search.
Additionally, people buying OSX computers isn’t really a bad scenario for Microsoft. MS office is quite popular on OSX and iOS and people spend more on that than they do on operating systems.
What kind of investment do you think they need to make to be a threat with Windows? If you think they should be trying to "match" windows with things it does, it would be a bad strategy that isn't winnable. Throwing away money. At least with their current approach they are levaring strengths (verttical integration, mobile dominance, etc) to build something that could leapfrog windows one day. Especially if the user journeys that span into other products (hw accessories, phones, etc)
Also, there's speculation that Microsoft is starting to give up on getting wide adoption for Bing among consumers , and given Bing's low market share, Microsoft has probably determined that Bing is never going to get widely adopted enough to the point where having a deal with DDG would be a bad idea.
I imagine Microsoft and friends are also still getting some idea what queries people making on DDG, if those queries are being forwarded to them to get results. Not user-correlated, and hence, nowhere near as valuable as if someone searched on Bing while signed into their Microsoft account. But it's probably got some general value to them on search behavior.
DuckDuckGo also have their own indexer/crawler too.
Yahoo Search does a similar thing AFAIK.
Yeah, Yahoo used to have their own search index, and very developer friendly search API called Yahoo Boss. That was actually the main source of DDG results originally.
Yahoo has since given up on Search, shut down the Boss search API, and is now also fed by Microsoft Bing.
Bing is interested serving DDG, Qwant, Ecosia and a lot of other unknown search engines because of the aggregated reach they provide for their ad network. Ad-networks only work if the aggregated audiences are massive, otherwise advertisers do not bother putting their ads there, only the top-3/5 ad networks get to see any action. So Bing wants/needs a bigger audience just to be on the game. They can grow in 2 paths: 1) increase Bing search reach (difficult), or 2) use partners with different value propositions.
Bing charges little for 1K query, 1USD officially but it gets cheaper, to zero :-) The real thing though, is that if you display Bing ads, you get a 70%-90% rev-share of the ad-revenue, which varies from country to country, something between $5 to $20 per 1K queries.
So, DDG basically gets around 5$ to 10$ net for each 1K, and can spend all that money on distribution and marketing so that they get even more users. Bing gets the rest, money, and what's more important, their ad-network continues to be competitive.
Everybody wins, right? :-/
Search is so cheap 1$/1K queries and people makes 2/3 queries per day, so $1/year/user (average). It makes no economic sense to build an alternative. Unless of course, you are building out of "ideals".
I think they also use Bing to serve (and earn a commission on) Bing ADS too, but I don't see that documented in the Search API so it may be separate.
Fun fact: DuckDuckGo used to use Yahoo, but Yahoo just has a deal with Bing. In other words, DuckDuckGo paid Yahoo, to pay Bing, to deliver Bing search results. :)
Now they use other resources as well, and also have their own internal web search.
Yahoo search is largely powered by Bing, as well. That's kind of funny since MSFT wanted to buy Yahoo at one point.
Also, there is the concept of dependent competitors. In old days, GM sold transmissions to American Motors, for example. So American Motors was a dependent competitor of GM.
> We also of course have more traditional links in the search results, which we also source from multiple partners, though most commonly from Bing (and none from Google).
Note that their own crawler is only mentioned in the context of Instant Answers so it doesn't seem it's actually used for any regular search results.
Always end up switching back to Google because of local results (I live in the UK, DDG's results are very US-centric even when local results are turned on). Maybe it's gotten better recently. I've seen lots of billboards for DuckDuckGo recently around my hometown in Paisley, Scotland; not a very techie place so they're clearly trying to gain a reach!
I've been using Bing for the past few months and been quite happy with it, only switching to use Google for Maps results and the odd very niche query.
The bonus of Bing is that it pays you in supermarket shopping vouchers where Google doesn't. Sold!
So I went bing.com, ddg, and tried something like
nursery rhyme about plucking a goose
Both were filled with pages and pages mother goose links, none of which had anything to do with what I wanted.
Typed that into google and it had an immediate answer, because it correctly surmised that I'm a dumbass and forgot that it wasn't a goose all along.
The problem with Google is that it will always assume that you didn't mean what you asked, and when you did actually mean exactly what you said this gets old really fast.
At the top bar, it seems to state specifically the UK being the location. Searching for things like amazon seems to also return amazon.co.uk rather than amazon.com.
- Amazon.co.uk (UK)
- Amazon.com (US)
- Amazon.in (India)
- Amazon.com / prime (US Prime)
- Amazon.ca (Canada)
On Bing conversely I get:
- Amazon.co.uk (with links to sections of the site like books/today's deals etc. etc. underneath)
- Amazon.com/uk (US site but redirects to UK site)
- Amazon.co.uk Your Account Section
- Amazon Prime (UK)
- Amazon Deals (UK)
Bing's results are clearly more UK centric and therefore more relevant to me as a British citizen.
For that reason I can't really make the full-time switch to DDG as I keep !g'ing to get UK results and it becomes a pain.
Are the local results even supposed to do anything? I tried finding a store carrying a certain video game. I turned on the local filter from under the search bar and nothing. All of the results are the exact same as without. I could not find a single result that was actually local to my country even after going through the pages.
Advanced search on Google definitely brings up local results for me with the same term, so it's not that there are zero results.
- less breaking news stories (I realized this the other day as I was looking for a breaking news story that would typically have been at Google a year ago, with link to local newspapers, but I cannot say exactly when this change happened since I mostly left Google behind as default search engine a good while ago. )
- more relevant search results when I search for technical content. The last two or three months search results has actually been improving for me for the first time since probably somewhere between 2007 and 2009
Looks like DDG is "fighting back" the right for people to have privacy online.
> It's not *fighting back", it's trying to make a profit by trying to position itself in a way that's different from Google and in a way that it's hoping is compelling.
In that case, please start a Google/DDG competitor that's just as good and uses a non-profit model. I'll happily use your service.
-- Sergey Brin and Larry Page:
edit: Here is a quora answer from the CEO at DuckDuckGo posted a couple years ago talking about how the company is funded. 
Their poor experience and dubious practices likely spawned Google's internal motto: "don't be evil"
Sure, I remember the bad ol' days. But Google has (IMHO) become the thing they set out to disrupt. The quality of Google results has declined dramatically in the last 5 years or so.
It's all marketing bullshit now, just as Altavista or whomever was. Finding content I want, as opposed to content advertisers want me to find, is becoming very difficult via Google.
As Google is the market leader, every website is optimized for its ranking. Much like leading operating systems have the most malware issues.
Many people on HN complain that blogs and personal websites aren‘t prioritized in results anymore compared to bigger news sites. I imagine it‘s incredibly difficult to separate legitimate, trustworthy posts from useless blog spam and agenda/conspiracy-pushing crap in a reliable automated way.
In the early days, algorithmic search wasn't the only way you found stuff. Links had a lot more use, and people would spend time looking at, cataloguing, and grooming lists of resources.
This doesn't scale well if you want a one-second answer to any possible question, but the SEO-gamified Google world is far from perfect in that regard either.
Maybe we just need to give up on that "any answer to anything at your fingertips in one second" goal.
Personal websites are mostly agenda pushing because they are personal. It might be that this good was fun or Dad got a sunburn but some agenda is being pushed.
But identifying blogs are easy that is how google can depriorize them easily. News sites are easy to identify as well. Assigning a qualities to news sites is a problem.
What is trustworthy? What is legitimate? Who is the user using the service how do they define these?
I always read here about people complaining about how bad DDG’s localized results are. Considering they usually talk about switching back to google, I can’t comprehend that.
But one day, I experienced what it was to use Google without ublock origin, by mistake. It was a horrible mess.
Then it hit me: the main benefit of DDG is that it's clean, even if you don't know anything about ad blockers.
Facebook on the other hand, a company where I only have a minimal profile since I utterly distrust them manages to serve me not only less annoying but sometimes actual useful ads (as: I've bought things from Instagram or Facebook ads that have saved me money and improved my quality of life).
: Google I just dislike and treat with caution. They seem competent with data security and obsessed with protecting all their juicy data even if - at least to me - they seem so utterly incompetent at uskng it to match ads to profiles that I have suggested it might be some sort of scam against ad buyers ;-)
The only thing I worry about, is if DDG gets too big, what stops them from becoming Google-like?
How much people got rich from Google - by quickly accessing the relevant blogs, articles or information, finding answers to the questions, code snippets and so on?
In this way I think it's fair to say that while individuals may consider the trade-off totally worth it or not worth it, they also have grounds to complain. Just like in society in general, one economic solution does not unmake all the ethical problems. Human culture has been working hard at transcending this dichotomy and people just expect more creativity and ethical intelligence these days. The attention given to DDG is I think a testament to this kind of market voice.
Their browser extension sucks. It’s just a bunch of different privacy extensions (Privacy Badger, HTTPS Everywhere, TOSDR) which have been watered down and mashed together. And it automatically changes your default search engine to DDG, which was the real intent for developing the extension all along.
Instant answers are important. Ever search for the definition of a word (https://duckduckgo.com/?q=assonance), or want to set a timer (https://duckduckgo.com/?q=timer)? These were made possible by DuckDuckHack. Now they are stuck; they cannot be changed, and no new ones can be created.
I'd argue that's not particularly a bad thing as it means there's less censorship going on.
Even if most conspiracy theories are completely bonkers
I did the same search on Bing and Google and the scam site is not part of the search result.
DDG emphasizes on privacy. Yet, allow scamming site returned as top result.
Mozilla had enough failures already as it tried to do things that are not Firefox. A search engine is a huge endeavor, and I'd rather have them focus on Firefox. Firefox is, after all, the only real alternative to Chrome, others are just different Chromium front-ends, it is important.
Or they can do like everyone else: an somewhat anonymized front-end for Bing, but what's the point? We already have DDG and a few others.
I love DDG, but at the end of the day they’re for-profit, US-based and closed source.
Google initially was bootstrapped by crawling such curated lists. Now that people are relying on search instead of curating their own lists search quality is starting to decline. So I'm going to make a prediction. Federated knowledge bases with curated links are going to make a comeback because only humans can create semantically meaningful content for other humans.
I have the DDG search widget, but I cannot remove the Google search box that appears to be hard-coded to the home screen with no way to remove it or change the search provider (... unless someone knows otherwise?) So now I have two search widgets...
I can use a separate launcher I guess, but it would be nice if I could just configure this without resorting to that.
A lot of people assume Chrome has the monopoly based on looking at false analytics.
In the future, it may be worthwhile to download some plugins like Cookie Autodelete, uBlock Origin, Privacy Badger, etc.
> A fundamental understanding of complex data structures and algorithms, which enables you to develop and apply new solutions and confidently navigate and improve existing code (ours is mostly written with Perl).
Would love to hear more about this. That kind of surprised (frightened?) me to be honest.
I live in NYC and I'm seeing lots of billboards in the city and on highways in NJ surrounding the city as I have family that regularly travels in/out. I've seen a lot of whack billboards in my day but it really caught me by surprise seeing them try and make this push. It's exciting, and hope it'll gain some traction. However, I think the vast majority of people will still just use whatever browser and whatever search comes bundled with their machine.
It seems like a reasonable implementation to me. Maybe the number of choices can be upped from 3 to 5 to help drive prices down. But surely there has to be some upper bound of the number of options?
I tried to pretend like I cared more about privacy than results but the god's honest truth is I don't. I want the results I want and Google provides them.
Another thing is regarding k-anonymity which I suppose privacy-first guys would use. If I gather user info with a high entropy, I don't have an incentive to sell it to anybody and create a competitor, and it'd probably illegal to do that. But if I anonymize the data set into a k-anon data it's now less restrictive to sell from legal point, also I'll probably sell that data to third parties just to make ends meet.
> Weinberg: We make money the same way other search engines make money. When you type in “car,” you get a car ad. We syndicate Microsoft and Yahoo ads through the Microsoft Yahoo Search Alliance.
But insofar as we have (formal and informal) notions of information ownership, we (sensibly) assign ownership of that info based on who collects/computes that info, not the person about whom that info refers. If I collect information about how many cars go down the highway each day, the only entity who might be the owner of that data is me, not the driver's of the cars or the city through which the highway runs.
If Google collects and exploits information about you (e.g., how many web searches per month you make), there certainly may be privacy principles they are violating, but not ownership issues. If Google were taking information you had generated/collected (e.g., if they sold illegal copies of the novel you wrote on Google Docs), that would be an ownership issue, but that is usually not the sort of thing people are worried about.
“Today, DuckDuckGo combines data from more than 400 sources into its search results: Microsoft’s Bing is predominantly used to surface relevant pages, but Wikipedia, Apple Maps, TripAdvisor and DuckDuckGo’s own web crawler also contribute.”
Not sure why you'd go to the trouble of avoiding google and just give all your data away to ddg? Anyone other than yourself is liable to become/be acquired by some shady entity
If anyone is interested in exploring alternative search indices, try Yippy, Mojeek, or private.sh (a proxy to Gigablast run by PIA). Yippy is actually pretty OK. The other two are somewhat marginal but will occasionally pop up an interesting result you'd never see on another engine.
But I can't make any sense of the second part of the OP.