Hacker News new | past | comments | ask | show | jobs | submit login
DuckDuckGo browser seemingly sends domains a user visits to DDG servers (github.com/duckduckgo)
824 points by commotionfever on July 1, 2020 | hide | past | favorite | 488 comments

Hi all, Founder and CEO of DuckDuckGo here. I’m literally just waking up and reading the comments here.

I’m new to this issue and happy to commit us to move to doing this locally in the browser and will have us move on that ASAP.

That said, I want to be clear that we did not and have not collected any personal information here. As other staff have referenced, our services are encrypted and throw away PII like IP addresses by design. However, I take the point that it is nevertheless safer to do it locally and so we will do that.

Thank you, this is the response people here want to hear.

And if this gets fixed in a reasonable timeframe, this is just one of those "everyone makes a mistake one in a while"-things, no big deal.

If anything, it's much better than 'no big deal'. It's "We made this design decision, thought you would like it -- we've learnt, changed, and will avoid it later".

Can you imagine Google doing something similar? Heck, they're just about to throw the Android rooting community under a hardware-attestation DRM-filled bus.

More like we made a design decision, than someone warned us this is bad for privacy, we ignored, after 1 year it blew up on HN, now we are fixing it.

Could have been quicker but it's still far better than some smarmy non-apology and continuing as usual, which is sadly what we've come to accept. I don't think they've done too poorly.

That's exactly right. Don't know why I can't upvote this

In this day and age, bringing problems to the attention of the right people is a challenge.

They started a fire through mild negligence, denied the fire existed, and only put out the fire when the entire neighborhood started yelling.

It was a forgivable-but-negligent decision to write/approve that code in the first place. It was a sign of a bad process that a reported security vulnerability was not escalated to people security-conscious enough to immediately identify this as a major problem.

I don't agree with the outrage. Anyone who has followed DDG knows they're legit. They just need to do a bit better. They probably will.

Their main feature is privacy. They should be at least as sensitive to privacy vulnerabilities as their most aware users.

DDG should announce that they now pay out privacy-related vulnerabilities like this and send the reporter $5k. It would be good honest PR and well worth the expense.

Correct me if I’m wrong, but by default DDG uses redirects to prevent leaking your search queries through the referrer, so they already can technically see every URL you visit. Except their whole product and system is designed around protecting privacy and not storing that data. If the favicon endpoint respects the same rules (which it obviously does), it is no different.

Except the favicon thing applies not just to searches on DDG, but every page you visit if you use this browser

Every domain. Still not good, but there's a huge difference.

There's no way that this feature would've made it into Chromium.

True, it is better

Wow. That was really not very complex at all... Did they leave off some capabilities that the server side version performed, it was the complexity overstated?

> Did they leave off some capabilities that the server side version performed, it was the complexity overstated?

The former. This is a quick initial fix that will fail on many sites.

This response validates my trust in DDG as has happened so many times before. Seriously cool company you've built here.

Really, just a response, saying everything is ok, validates your trust? Bought some wirecard stocks latly?

Creating a throwaway account to disparage a particular point of view is also questionable, no?

At what point did we stop taking people's word and commitment as valid? Sure, I too want to see proof that they are doing the right thing here (because I don't understand the design decisions that led to the creation of that service in the first place), but because these changes are not immediate, this statement does at least answer some of the questions I personally had (Like "will this be fixed"? "when"?).

> At what point did we stop taking people's word and commitment as valid?

At the point that they where caught violating privacy while claiming to respect it, and standing to make a profit off of violating it.

If I trusted people as blindly as people seem to trust DuckDuckGo, I would have trusted Google to not be evil and never have switched to DDG in the first place. This breach of trust destroys the whole point of using it, so I switched my default search to Searx between reading this submission and writing this (had been procrastinating the switch for a while and this was exactly the motivation I needed).

Well for me it was back in the 90’s

I will of course be keeping an eye on whether or not they follow through, but the admission of this being a problem and committing to fix it is -far- more than we see from most companies these days. Is it everything I ever could have wanted out of the company? no. However it is a positive direction I would like to see more of so displaying my appreciation is hopefully validation for the company to continue in that direction.

Did you even read the post you replied to?

"...as has happened so many times before."

Clearly it's not just the response.

Thank you Gabriel. This was what I expected.

It's a little confusing to see 2 accounts, with very different usernames (epi0Bauqu and yegg), that appear to be posting as Gabriel Weinberg . Are these both legitimate? And if they are, what's the reason for two of them?

Yeah that's my bad. epi0Bauqu is my original account, but since no one knew who I was under that account I, some years later, made the yegg account and I try to post from there. The issue here is I logged in to post the comment, and then switched to my laptop where I was already logged in this other account.

Good to know, I just wanted to be sure that someone wasn't attempting an impersonation.

It was only a minor issue that got resolved quickly thanks to your question, but DDG's response to this whole issue of losing users' trust seems to be accurately characterised by the phrase "a succession of embarrassing own goals". :-/

Have you considered noting this in the user profile data of this HN account, then logging out of this one on your laptop.

I think it would be more clear if you come up with a statement like:

‘we never used this data, other than showing favicons’

We never used this data, other than showing favicons.

In fact, we didn't (and don't in general) collect any user-level data in the first place, per our strict privacy policy: https://duckduckgo.com/privacy

In this case, the way it works is you hit our favicon service and it returns the favicon, not using any PII in the process, and our web servers are configured not to log any PII. In other words, our system is technically designed and we are legally bound by our privacy policy to not use this data for anything other than showing favicons.

Thank you very much for confirmation.

You might want to clarify you never retained any PII, unless you can 100% confirm that that favicon request didn't include any embedded user- or installation-id in cookies, headers or the like?

Thank you for re-opening and prioritizing this.

However, this problem demonstrates gross incompotence for a browser team supposedly concerned with privacy. Will you please do a post-mortem on how this code made it through your code review process in the first place, as well as how it managed to stay in place for a full year after it was pointed out that it represented a privacy problem?

"Sends every URL you visit to the vendor's servers" is the single worst thing DuckDuckGo could have done for privacy in this web browser, and that needs to be accounted for. There was a major failure in the code review process, ticket review process, and in how you treat your community. A standard marketroid "by design" response with washy promises that "we'll take very good care of this highly sensitive personal data, just trust us" is not something I want to see in the future from this team.

[reposted from GitHub]

I’ve worked with many companies who have demonstrated “gross incompetence” when it comes to privacy and information security. This is absolutely not an example of gross incompetence.

I agree that for a company built around privacy even the appearance of impropriety needs to be avoided. DDG holds themselves to a higher standard and their users hold them to a higher standard.

This was a design flaw and a process flaw. DDG prioritized speed and efficiency over privacy (or in this case, perceived privacy) and I suspect there isn’t a soul on HN who hasn’t made that trade off at some point. They assessed the cost/benefit and risk/reward and it turned out their assessment was wrong. Now they’re fixing it. It happens. But to call this gross incompetence is really blowing it completely out of proportion.

I'm not blowing it out of proportion. This one specific "design flaw", if we're being generous, has been raised many times with many different browser vendors and add-on vendors as a very bad thing that you cannot do. There is plentiful wisdom on this issue.

The first rule of privacy is never handle the private data in the first place. An accidental leak is one thing, but deliberately designing a feature whose side effect is exfiltrating heaps of private data, then doubling down on it for a year after it's pointed out to you, then doubling down again when it's raised on HN - this is gross incompetence.

You can’t think of anything they could have done that would be worse than sending URLs to their lookup server? It’s the single worst thing?

My browser syncs URL history between my devices, and that’s a feature that I value about it. Your comments on this topic seem to suggest that all users are making the same decisions about what is acceptable usage of their data, and that’s pretty obviously not true.

If your browser is Firefox, then it encrypts your history before sending it to the vendor.

Maybe you’re taking this a bit too far? They explicitly state they will not store your data anywhere, and the main safeguard you have for that is your trust in them, not this one specific line of code somebody happened to notice which can’t even break that promise on its own.

Privacy is never built on trust. It's built on mathematical and logical facts. The only effective way to keep data private is to never handle it in the first place.

Thanks. What kind of process changes are you putting in place to prevent things like this from happening again?

Good. Thank you.

Thank you, keep up the good work!

(I always knew there was a business model for privacy and I'm glad someone is working to figure it out)


Looks like they developed the favicon service for their search engine so they could show favicons in search results.

It actually increases privacy there, DDG already know what domains your search returnedy and the alternative would be fetching favicons directly from websites, leaking information to them before even clicking.

Then they re-used the favicon service for their browser without rechecking the privacy issue and realising that browsers have different privacy needs.

Completely over-looked the 3rd - that they're human and made an honest mistake that they're attempting to rectify.

But from your attitude I'm guessing that think you've never made a mistake...

Sure humans make errors, but this service is for a privacy-concered-company like you or me walking over a red traffic light without looking to the left or right and beeing crashed by the biggest Truck one can imagine. Anyone who belives in an error is beyond help.

The issue was highlighted from open-source code from a company who proactively works to reduce privacy.

It seems rather reasonable to assume that in the goal of maximising privacy in search engines, it would be wise to add user-friendly and convenient features so that the alternative is appealing and does not sacrifice too much convenience.

> That said, I want to be clear that we did not and have not collected any personal information here.

So, you do collect information just not the kind you would classify as 'personal information'. I wonder if my personal domain with my full name qualifies?

There is no way this feature would be created in a company built on privacy considerations.

Like, wow!

Sure it could. A junior programmer aware of the values but not the repercussions could easily overlook this without any malice or disregard for privacy.

Right, let's blame the intern game.

Didn't that server endpoint need an upkeep? Didn't they wonder why it's getting all the traffic? Maybe they were DDOSed or something?

Multiple people in that company knew exactly what they were doing.

> So, you do collect information just not the kind you would classify as 'personal information'. I wonder if my personal domain with my full name qualifies?

Considering they have operations in the EU, I would imagine that would fall under what the GDPR considers personal information (https://gdpr.eu/eu-gdpr-personal-data/), or risk being in breach of it.

To be more clear, your staff, and you, have said PII ‘like IP addresses’, and have said ‘thrown away’ some places and ‘not collected’ others.

Contrary to this framing, it’s not possible to not incidentally become aware of every single browser users’ usage timing and user IP addresses if the browsers are phoning home this way — a colloquial understanding of ‘collect’, not the James Clapper NSA dodge definition of ‘collect’. Most normals think of collect as become known not as permanently store. You knowing it means others can know it if you break trust or are required to comply with authorities.

And regardless of end-to-end encryption, that this user is phoning home to your fave icon endpoint, when, and from what IP, is revealed to every ISP in the chain. You’re leaking browser usage telemetry to every single party to that traffic — the source IP address PII you mention is in unencrypted metadata.

The fact this browser connects to that endpoint reveals demographics (choice of privacy browser) and behaviors (when and how much web surfing) to e.g. ISP or nation state firewall operators who are certainly not bound by your ‘just trust us’ privacy policy.

Privacy policies are a patch for insufficient privacy engineering.

To be a strong privacy browser you could consider what it would take to be “NSL proof” such that if handed a national security letter with gag order, you cannot comply. That is not the case with this faveicon telemetry endpoint.

Just to be fair, as a matter of fact, you surfing that site is revealing you, surfing that site to your ISP and state actors, in the first place. A change, where to get the icon from (origin vs ddg), will not change this fact. It is all about ddg not getting to know, which sites you are surfing, when not searching for it on ddg. Which should, indeed, be a no-brainer.

"You surfing that site" not adding a single surveillance point for every user of a particular browser, it's not compromising your other privacy measures (e.g, VPN that prevents your ISP from knowing) by sending your site surfing data out to that surveillance point, etc.

Getting the icon from each site means surveillance would have to be at origin or every site, while telemetry going to DDG gives a single surveillance point.

I've already posted this somewhere else, but I'll copy it here again as well:

It's not immediately obvious whether it is more privacy preserving if the client automatically makes a request to each site in the search results while scrolling through the results, especially since you're already trusting DDG when performing the search.

Maybe this should be an opt-in rather than an opt-out feature?

All in all its really not as big of an issue as people here make it out to be.

This was not about search results, where their favicon webservice is in fact privacy increasing, but about the privacy browser and the favicons it displays, where it is privacy decreasing as it involved sending information about visited sites to a central authority while you are not on the DDG search engine. For example the TabRenderer will fetch the favicons from DDG instead of from the site you are actually visiting: https://github.com/duckduckgo/Android/blob/db728523240e37727...

Anyway, great decision by Gabriel.

Thanks for pointing out that the service was already in use on their search results pages. To me, this goes a long way toward explaining how this could have happened:

Scenario #1 - "We need to show favicons in our browser tabs. Lets develop an API that requires every domain be sent to us!"

Scenario #2 - "We need to show favicons in our browser tabs. Hey look, we've already got a service that provides this. We know it collects no PII and our users trust it already."

Obviously the second scenario is flawed thinking, because (of course) it's better to not send that info at all. However, I can easily see how their developer(s) may have arrived at the conclusion that this is still compliant with their privacy ethos.

The fact that the favicon service already existed (and was trusted by users) before this was implemented, makes it much easier to understand how this could have been a legitimate mistake and thus, they deserve the benefit of the doubt.

Yes, it is a totally plausible mistake. The fault was "only" ignoring it after it was reported.

Ah I guess I should have read TFA l, because search results have a similar feature (that is opt-out). For a browser I agree it makes more sense to get the icon directly from the site being visited without any privacy risks.

Yes, after consulting with staff, I understand we thought it was more privacy protecting because we know our services are already encrypted and throw away PII, and so to get the favicon you could either (a) make another request to our known anonymous service or (b) make a request (or possibly multiple) to a non-anonymous service. On the other hand it is another request to a distinct domain that traverses another path on the Internet, albeit an encrypted one.

> not as big of an issue as people here make it out to be.

Please refrain from speaking for others without being asked to.

I don’t think that line is speaking for others. It is characterizing what others are actually saying.

Exactly. Everyone should decide for themselves how big of an issue it is for them and not one person who just shrugs it off. That person made their statement sound like a universal fact.

I think the most charitable (and probably the default) reading of that clause has an implied "I believe" prepended. It certainly does not have an implied "It is a universal fact that" prepended.

By way of analogy, when you earlier said "This is ludicrous. Can we switch to the metric system already?" no one thinks that you're speaking for everyone, but rather are advancing your own belief in the "This is ludicrous." sentence.


DuckDuckGo staff here. As mentioned in the linked page, the purpose of the request is to retrieve a website's favicon so that it can be displayed in certain places within the app or on the results page. We use an internal favicon service because it can be complicated to locate a favicon for a website. They can be stored in a variety of locations and in a variety of formats. The service understands these edge cases and simplifies retrieval within our apps and our search engine.

Like our search results, the favicon service adheres to our strict privacy policy[1] in that the requests are anonymous and we do not collect or share any personal information.

[1] https://duckduckgo.com/privacy

Take the code from Firefox iOS or Android-components. We spent a lot of time on these and it is all on device.



I wonder how many lines of code from big open source applications are generic enough to be reused in other projects.

Firefox and Google Chrome probably have the equivalent of many small high quality libraries embedded in them, implementing 'business' logic or protocols, that could be reused in more places.

I guess a large scale study on github could be done, with a graph analysis to show potential "cut off" points in codebase.

This is one of the goals of our Android-components project. Take a peek. Many components are reusable across browsers/applications.

One issue is that if a bunch of code of a similar theme can be broken off, many times it will be (depending on the culture of the software team). Is looking at the source code of a big project with a hundred dependencies that were all libraries spun off to support the project different to looking at a project where people tried to keep everything monolithic (where you’d expect better re-use potential per line-of-code?)

Yeah... the gesture is nice, but good luck extracting any code from a massive project. Might as well say “Here’s some free oil; all you have to do is dig for it.” Unlike oil, this might not be worth the excavation.

It’s a bit telling that they linked to the GitHub repositories rather than specific lines of code they were talking about.

Here's their kotlin implemenation, looks fairly straightforward/self-contained: https://github.com/mozilla-mobile/android-components/tree/ma...

Android components is a foundational technology for our browser products on Android but also for many other applications. We’ve designed things in such a way that you can pull in just those things that you need. If that is not the case, file an issue and we will take a look at how to improve things.

Looking at the iOS project linked, https://github.com/mozilla-mobile/firefox-ios/blob/76faae8c6... , other than being written swift, it looks pretty self-contained/ok?

There is also a content script I think and storage for these icons and their meta data. It is probably less generic on iOS than on Android but should be fairly simply to take some big chunks for reuse.

Yeah, let's violate privacy instead.

What a world class reply. This is what makes open source valuable.

Yes, apart from Chrome's implementation there is probably no better tested and more mature implementation. Still it seems to be a non-trivial problem because Firefox sometimes shows me a favicon from another site I used.

Why don't they just take the code from their "internal favicon service"? The whole thing smells.

If you don't visit the site explicitly, just have the site somewhere in a list/bookmark/whatever, that site now has your IP address and basic header info when it needs to go retrieve it. By going through DDG, the site has a bot hit.

To me it looks to be trying to uphold your anonimity until you commit (click) through to the site/link. But certainly other ways they can approach this if it really bothers people.. I'd prefer DDG doing the lookup.. or having no fav icons.. over my computer going and downloading all my bookmark or other source icons

It's probably not written in Java.


I think you're being downvoted because you chose to piggyback your comment on a seemingly unrelated one at the top, are being vitriolic, and didn't back up your claims with respect to their intent and refusal to change this.

Well it's related because Mozilla actually cares about your privacy vs DuckDuckGo which obviously could care less from their reaction to this issue. Their refusal to change this is all the proof you need to know. I dont even use DDG I use Google I just think its funny they have a "Privacy browser" that sends all the sites you visit back to their servers

It is not relevant to this sub-thread, and should have been a new top-level comment.

It's a direct reply to a comment on the sub-thread.

This is nonsense. DDG cares deeply.

We all, including Mozilla!, have made design decisions that in hindsight could have been better.

The important thing here is dialogue and change. And both are happening.

(parent edited comment, can't delete this)

That bit was unnecessary, and pretty tone-deaf given the current socio-political climate.

So let’s call them out on that, briefly, then get on with the argument.

It's not immediately obvious whether it is more privacy preserving if the client automatically makes a request to each site in the search results while scrolling through the results, especially since you're already trusting DDG when performing the search.

Maybe this should be an opt-in rather than an opt-out feature?

Edit: as pointed out by warpspin in another comment, this is about the DDG Browser, not search results.

Scrolling through results is not a good place to ping websites for their icon. Not from a technical perspective but also not from a privacy perspective.

This is mostly a UX issue IMO.

I appreciate you answering, probably knowing you'd face some negative feedback. Saying "we should trust you, it's for a good reason" is what google and everyone else says. You'll be better off if you just end this. The loss of the fav icon is less important than keeping your credibility.

Seconded. Day in and day out we're given empty promises we can never audit -- now it's being done DuckDuckGo staff too?

thirded. an inconsequential token is not a suitable reason to engage in this kind of data collection, especially for a search tool that prides itself on 'not tracking' users..

Fourthed, and honestly it's strange that they are willing their promise to privacy on saving 2-3 queries on something as trivial as favicons? Why does no other browser need to do this?

Favicon what a hill to die on

For the want of an icon, the kingdom was lost !

This is the correct answer. We all remember "Do No Evil" and have been around long enough to see that sentiment die. Don't go down this path.

They've already started down that path, judging by all the DDG billboards I see driving my 18 wheeler around the country, and all the DDG ads I hear on NPR-related podcasts.

Not that I'm against making money. But there's a tipping point associated with some height value in a pile of cash, and once you cross that point then the pile controls you. DDG probably hasn't crossed that point yet, but self-justification is one of the steps on that path.

Are there really DDG ads on billboards and NPR? That’s cool.

There are ads on billboards here in Sweden.

I amazingly saw one in Las Vegas recently

In Spain too

> is what google and everyone else says.

Do Chrome, Firefox, or Safari do this? I would assume they do it on-device.

Chrome, according to my understanding, hardcodes a few favicon URLs for builtin search engines, and caches everything else on site visit. There's SQLite3 database named Favicons in your profile directory (e.g. on macOS it's ~/Library/Application Support/Google/Chrome/<Profile>/Favicons). Here's the schema:

  CREATE TABLE favicon_bitmaps(id INTEGER PRIMARY KEY,icon_id INTEGER NOT NULL,last_updated INTEGER DEFAULT 0,image_data BLOB,width INTEGER DEFAULT 0,height INTEGER DEFAULT 0,last_requested INTEGER DEFAULT 0);
  CREATE INDEX icon_mapping_page_url_idx ON icon_mapping(page_url);
  CREATE INDEX icon_mapping_icon_id_idx ON icon_mapping(icon_id);
  CREATE INDEX favicons_url ON favicons(url);
  CREATE INDEX favicon_bitmaps_icon_id ON favicon_bitmaps(icon_id);
Related source code: https://github.com/chromium/chromium/blob/master/components/...

I haven't looked at Firefox and Safari but I assume they do something similar.

I would hope that it can also be made to cache 404 and 410 responses to favicon requests too (or at least 410 even if not 404), so that it won't keep trying to access it.

Also, in SQLite, note that LONGVARCHAR is the same as TEXT, and that you don't need to specify both UNIQUE and PRIMARY KEY (it is redundant), and that if it is not a INTEGER PRIMARY KEY and not WITHOUT ROWID, then it isn't the real primary key but just an index (same as UNIQUE); add WITHOUT ROWID if you want to make it a real primary key, but note that the way the data is stored differs then, and WITHOUT ROWID is inefficient with tables storing large blobs.

In germany we have the words "Datensparsamkeit" (data parsimony) and "Datenvermeidung" (data prevention) [1]. Which wikipedia merely translates as "Privacy by design" [2].

DDG is unneccessaryly producing (aggregating), transmitting (and collecting?) very sensitive user data here, which is just the opposite of data protection. I can't even understand why they try to justify their actions. It's like omitting the seat-belt in a car, then telling customers that this was required to make the in-car entertainment system more usable.

[1] https://de.wikipedia.org/wiki/Datenvermeidung_und_Datenspars...

[2] https://en.wikipedia.org/wiki/Privacy_by_design

In fact I think what they do here is illegal by GDPR. It does not matter that they say they do not collect the information, it is enough it is unnecessarily sent to their servers to make the whole function illegal.

The transmission of ip address alone, which is necessary for the TCP request to happen, deanonymizes the request enough to not be considered anonymous within the GDPR framework.

GDPR Article 5 (1) c: "Personal data shall be ... adequate, relevant and limited to what is necessary in relation to the purposes for which they are processed (‘data minimisation’);" - this is the "Datensparsamkeit" you mentioned.

Exceptions from Article 7 do not apply: The user has to give wilfully give informed consent, which he cannot do as the privacy policy of the browser omits the information that all visited domains are transmitted to DDG servers.

GDPR Recital 30 "Natural persons may be associated with online identifiers provided by their devices, applications, tools and protocols, such as internet protocol addresses, cookie identifiers or other identifiers such as radio frequency identification tags."

Oh, and the fact I'm downvoted for a purely informational comment additionally does not shine a good light on DDG.

Yes it does not look very GDPR conformant. They may try to argue that the transmission of visited domain names serves a purpose (browser performance) and that the user agreed to that transmission by accepting the TOS etc. However, I think they may be in trouble as consent for data processing under GDPR needs to be given "freely" [1]:

"When assessing whether consent is freely given, utmost account shall be taken of whether, [..] the performance of a contract[..] is conditional on consent to the processing of personal data that is not necessary for the performance of that contract."

As DDG's favicon-hack is not strictly neccessary for operating the DDG-browser, DDG would need to give users the option to opt-out of the favicon-retrieval, otherwise they may have "forced" the users to consent to the data processing, thereby voiding that consent as far as the GDPR is concerned.

[1] https://gdpr.eu/gdpr-consent-requirements/

You are completely right about the not "freely given" aspect. But well, it already breaks down at the "informed" aspect.

Their appstore pages link to the generic privacy policy of the company instead of for the browser specifically. This alone will usually already NOT be accepted by the supervisory authorities at least in Germany - the Landesdatenschutzaufsichten usually require a product specific privacy policy being linked for it being valid (personal experience), or at least making clear in the general policy what applies to the product, what not. And as mentioned, the fact visited domains are sent to the DDG servers (also outside the European Union) was not mentioned at all in that policy when I looked some minutes ago.

> DDG would need to give users the option to opt-out of the favicon-retrieval,

Wouldn't they need to give users the option to opt-in, under GDPR?

I think you are being downvoted because your interpretation of GDPR is extremely broad and people disagree with that. I also don't think it's in line with the way it is being applied in practice. Nothing to do with DDG.

I guess I might be one of the few people here who actually have been regulated under the GDPR by a supervising authority.

When that happens, you have to do insanely stupid seeming stuff like explaining in your privacy policy even, why your application, which is clearly for accessing a webservice, actually needs the Android permission to access the internet at all, true story. So I am pretty sure, an app sending visited domains to a central server outside the EU without even a mention of that fact in the privacy policy will cause problems with them, should they ever check the app. Of course, other countries might handle this differently, as Ireland shows.

So whoever thinks my interpretation is overly broad should first have the decency to step forward and actually explain why, instead of hammering a button and second, talk to me again after he had a meeting with the responsible authority and listen to THEIR interpretation of the GDPR ;-)

People should not mistake my interpretation with endorsement of the overly broad text of the GDPR itself.

Wait what? a TCP request already breaks the GDPR rules? Didn't know that...

Any human readable ways of dealing with that?

That's not what the parent said. The TCP request includes info that deanonimizes the request.

Or put another way, a TCP request sent by your app from my computer can not be considered anonymous.

If it is an unnecessary request to another service, yes.

IP-adddresses are considered personally identifying information. TCP requests transmit IP addresses.

Under the strict interpretation of the GDPR, a lot of things which are common outside the EU might be illegal, like e.g. embedding Google Fonts. To be on the safe side, people usually at least list these external dependencies in their privacy policies to construct some kind of "consent", but till we have more actual court rulings, this is a huge problem area.

For the problem at hand, it is pretty clearly illegal, as it's not only an ip address transmitted, it is a combination of ip address plus visited unrelated domain. This allows the creation of profiles. It does not matter for the GDPR, if the profile is ACTUALLY created, the pure possibility of creating it any time is enough to be a problem.

I don't think this is an accurate way to analyze GDPR compliance. As the staffer points out this favicon service follows their own privacy policy, if by this policy they keep (or analyze, sell, distribute, etc.) no data on your use of the service then there is nothing of interest for the GDPR.

They might have to prove that their privacy policy is indeed GDPR conformant and that their service works as advertised, but in practice this is likely more about public trust that legality.

It's not as simple as that.

Art. 4 GDPR (1) clearly makes the (ip-address, visited domain) tuple personal data Art. 4 GDPR (2) defines "processing" data, and the pure "collecting" of data, even if immediately thrown away, is usually already considered "processing", therefore the GDPR applies.

If you are doubting this, just for a moment imagine, instead of the visited domain they would have sent all form data, including for example credit card data, you entered somewhere on a third party webpage to their central server and did not mention the fact in their privacy policy.

Do you really think then there is "nothing of interest for the GDPR" just because they do not actually permanently record that information? It would clearly be a violation. But to the GDPR, the importance of that data is equal. In fact, the domainnames might actually be more important to the law, as article 9 establishes event stricter rules for "sensitive" data about e.g. health or sex life of a person, and the domainnames might just leak that information.

> Wait what? a TCP request already breaks the GDPR rules?

If the TCP request carries personal data like the name of a visited website plus the user's IP address, then it "breaks the GDPR rules" in so far as you now have to fullfil your GDPR transparency/consent etc. duties /before/ sending that request.

Maybe not all website names look like sensitive data to you, but some website visits you surely want to be treated as sensitive, personal data (like names of hospitals, doctors, political parties, religion etc.).

You really can't use "we promise we won't misuse the information" as an argument, that's what everyone says whether it's true or not, and the whole point of using a privacy-centric browser is that as a user you can't trust those kinds of promises.

They’re primarily a search engine, so yes, you definitely have to trust that they won’t misuse information about what search terms you are querying for.

That’s pretty shaky ground. Trusting a site for X has nothing to do with being comfortable to trusting them with X + Y when Y is unnecessary.

I suppose that's valid. This is the first time I had heard that DDG has a browser too, and I was just assuming that anyone who would use the browser probably also uses the search engine, which they obviously have to trust when they send search queries to it.

AND being so reluctant to remove this pinging mechanism adds more doubts about the real purpose.

> You really can't use "we promise we won't misuse the information" as an argument

Sure they can. Doesn’t mean you have to believe them.

When people say "you can't do" something that is already being done, it means they can't justifiably do it. Context matters for what words mean.

By the way how DDG spend money on the ads like at bus stop, I already starting to NOT trust them. Spend the money on development of development of products.

Seems like time to get SearX a try now: https://searx.me

FWIW I got higher quality results with Searx.

This doesn’t make sense for a browser: just embed the service’s logic in the browser, the browser has all the same information the service could get.

We had already had created this anonymous favicon service for our private search engine. In addition, doing it this way avoids another request (and potentially multiple) to the end site.

The service is private as we do not collect any personal information (e.g. IP addresses) on any requests for this or any service and the requests are all end-to-end encrypted.

> In addition, doing it this way avoids another request (and potentially multiple) to the end site.

Potentially saving a few requests here and there is certainly not worth phoning home with that kind of data regardless of what records you keep how much you do to anonymize it. This is especially true for a company that has built its brand on promises of privacy!

Besides, favicon requests are small potatoes compared to the kind of tracking, ads, metrics, and other often-unnecessary page resources that bog down most of the modern web. And a well-designed website can mitigate the issue pretty easily.

It seems like DDG just doesn't want to do more work to make the browser do the sensible thing and instead piggy back on their existing favicon service, and then going to make the argument that it is safe with us. The domain of browser is totally different from reaching out to the website and rendering favicons server side. Do you not see this as a problem? I think it is acceptable to say "We agree this is a problem, and we are gonna look into fixing this" instead of pushing back on an obvious problem.

Surely you could fetch the favicon asynchronously and update the address bar or tabs or wherever you display it only after the other assets are loaded? It’s not like the DOM has elements that depend on it. This is a really, really weird hill to die on.

And all you lose in the process is hard-earned trust from people like me, who switched to DDG years ago for privacy concerns.

This is troubling.

I understand we’re suppose to take your word but it’s making me skeptical now about the actual intentions. If DDG truly cared about user privacy then you shouldn’t transmit user information for something as trivial as favicons.

Is this service open-source and publicly auditable ? Because we've learned not to trust one's word of "don't be evil", only facts matter.

If you say the service is anonymous and does not leak data, prove it.

I'm sorry, but this is bullshit. If the received HTML contains a <link rel="shortcut icon" ...> element, you already only need one request, so this is not an improvement. If that is present, you should NEVER use your API.

If it's not present, then you have options, and yes, using your weird API is an option (which I still don't like, but ok). But sending private information to your servers even when sites follow the standard show either that you're probably not trustworthy, or that your product team is so painfully incompetent that I'd be afraid to use their browser at all.

I think people here are more interested to know if you’re going to fix it.

> The service is private as we do not collect any personal information

This sentence is 100% meaningless. I understand you have good intentions, but these things must rely on proof, never on trust. Either you get this information or you don't; whether you say you "collect" it is inconsequential.

You may not collect it, but your upstream could collect it, and encryption has a shelf life.

It's a really bad look and you should ditch it.

> We had already had created this anonymous favicon service for our private search engine.

Which presumably means you've already created the logic for determining favicons. I'm not sure why this couldn't be implemented in the browser.

> We had already had created this anonymous favicon service for our private search engine.

I don't think here's a need for adjectives here. Why stress that it's anonymous (when that's hard to verify) or that the search engine is private, when that too is starting to come into question? Repeating these things won't will them into the reader's perception.

> In addition, doing it this way avoids another request (and potentially multiple) to the end site.

This isn't true, unless I'm missing something here? When I access a website, the HTML response I get from that website includes all the information my browser needs to, on its own, get and display the favicon. Can you clarify why you think/say this avoids one or more requests? What mechanism is this service a substitute for?

Saving a request is not a good reason to phone home, especially given what the DDG brand is built on and the main reason I would use it over Google. Please reconsider.

It's alright, folks. We got E2EE and a pinky-swear.

Please ask your higher-ups to educate you about essentials of privacy in the modern age. It's not about what you may be doing wrong, it's about what you technically could do wrong.

Complicated code can run just fine on device.

I’ve been an avid DDG user for years and it worries me that DDG staff don’t see why this is an issue. We shouldn’t have to trust your privacy policy if you minimize exposure.

This. No matter how tricky the logic is, there's literally no reason to run that code on DDG server's and throw in a remote connection. What the hell?

That's not fair. There is a very clear reason: it simplifies development for them and avoids duplicating functionality in two different codebases. You can argue about whether or not that's a good trade-off given the privacy implications (and I would agree that it's not) but you can't claim that there is "literally no reason".

Does Gabriel know about this? If not could you please clue him in and get some guidance because you are absolutely getting roasted here and are wrecking DDG's carefully built up reputation. I can easily see how this might seem to be a good idea to you and other DDG engineers but it goes 180 degrees against DDG's stated mission. In other words: you may be well outside your paygrade on this.

> you may be well outside your paygrade on this.

Not as worse as publicly denouncing an honest engineer while referencing his paygrade. I hope there is no affiliation you have with DDG to be honest, because this is much, much worse.

What do you mean? This response is fine. It’s honest feedback, and it isn’t a personal swipe to say it’s above someone’s pay grade to be responding to a PR crisis as an honest engineer.

I was once an honest engineer too, publicly. Being honest in private is enough for me now. It’s a lesson worth learning.

That said, it’s not like a single HN comment will make or break a company, so if they’re really just a rank-and-file engineer, I hope the company won’t come down on them too hard. A simple “don’t do that” would suffice.

Yes, but chastising an employee is not in the interest of good PR, even if we sadly see that quite often in the days of social media. That is especially true if the employee just honestly laid out the facts, I wouldn't even call that a mistake.

I use DDG and the possibility of getting a statement directly from an engineer conveys much more trust than a carefully crafted PR statement ever could. I would think again about using it if the company does indeed come down on employees that live the values the company writes on its flags to have honest and transparent business practices.

That said, I am careful too when I state things about my company, even if I believe there is nothing to hide. Still, people that think it isn't the place for others with knowledge to comment are often not too impressive and would have difficulties in convincing me that privacy and transparency are real goals instead of just looking decent enough.

Furthermore the naming of management of DDG creates a stark contrast to the suggestion for more professional distance. I don't like PR very much as you might have guessed, but like a good design it needs some congruence.

If people find out that you just shut up for your company, it might give people the wrong impression about their business.

It’s your duty as an employee to shut up for your company. I didn’t learn this until later in my career. Fortunately it didn’t have lasting impact.

By commenting on an ongoing PR crisis without consulting management, you are both undermining their ability to respond in an effective way — imagine how strange it would look to see a “Hey, X from <company> here” after an existing one was already posted — and you’re acting on your own rather than in a team. You’re a part of a team; how could you think it’s a good idea to act alone?

Of course, I am talking to my former self with this comment, since that’s exactly what I did at S2 when working on HoN. It was a mistake, and I gave the community the wrong impression about the company’s priorities.

You have to understand, when you’re given money to do a job, you’re not given authority to become that job. Just because your job is getting beat up on social media doesn’t mean you should just jump in and go “Hey, that’s not true!” It doesn’t matter whether it’s true. Here, let me pretend to be DDG:

“Hi, Shawn from DDG here. You’re right; this was an oversight on our part. Obviously we dropped the ball on this. To clarify, we were unintentionally gathering the data as a side effect of our favicon service. <some technical details here>. We’ll be acting immediately to reverse this, and we’ll be enacting policy changes to ensure that user privacy — our core mission — is maintained going forward.”

But that’s not what they said. And if you’re gonna tell the community the opposite of what they want to hear, you’d better be in charge of the company’s Telling The Community Things division.

Thank you for explaining this all far more eloquently than I ever could. DDG is precious and worth preserving, it wouldn't take much for the press to have a field day with a couple of careless comments. And no, I don't have a stake in DDG other than as a user.

While that last bit was slightly crass, the rest of the comment was dispensing some very sage advice, I thought.

I don't think it's fair to say this comment is 'wrecking DDG's carefully built up reputation'. If what he describes is damaging their reputation to you, that's on the business, not on the commenter. If anything, the commenter seems to have been honest and straightforward with what's going on.

Sounds like you'd prefer him to have run a message past management/public relations first?

If you speak in a public form in response to something that seems damaging to the reputation of the company in a way that is probably even more damaging to the reputation of that company you are either an official spokesperson or way out of your depth. Pointing that out is a service, not a sleight. And yes, if this is not in an official capacity he should either keep quiet or run it by the PR dept. People have lost their jobs for far less. Anyway, I've pinged @yegg, hopefully he can shine some light on this.

This is an asshole response.

No, it is the response of someone who was once upon a time CEO of a startup that fortunately did not operate in the present day news cycle where reputations can be destroyed in a couple of hours. DDG is precious, saying things that are very much against what I know to be the founding principles of the company is something that should only be done by those that have been given the proper authority. The response here is candid but ultimately not the one that DDG would and should give. Gabriels' contribution upthread pretty much confirms that. My comment was simply to ensure things would not get worse than they already were until someone in a position of authority at DDG could step in.

All you have to do is to read this comment thread to see the kind of damage that a single statement by someone affiliated with the company can do.

For me it does the opposite. Hearing engineers speaking off-message inspires trust, while pr-evasion and weasel speak makes me think they have something to hide.

That's nice. But in this particular case the engineer says things that I know for a fact are not in line with DDG's mission statement so even if he speaks off-message he's actually destroying that trust. There is nothing about @yegg's response that strikes me as weasel speak, it is the content that is different and exactly where it matters: privacy first.

> There is nothing about @yegg's response that strikes me as weasel speak

Agree, didn't mean to imply that.

> this might seem to be a good idea to you and other DDG engineers

If that's true, then I am so glad they ghosted me when I applied there.

I’m very disappointed in how you guys responded to this. As a privacy focused company I would not have expected an answer that sounds like it came from a data collecting company like Google.

I just switched to DDG browser a week ago and will now be looking for a new browser now. I hope you know this is not an appropriate response to the situation. Especially because all you guys do is preach about how much you protect your users’ privacy. Now you’re here asking us to trust you not to abuse our data and just linking us your privacy policy. I’m sad to say that my faith in the DuckDuckGo company and team is now lost.

Lost lost lost.

I'ts not like we couldn't have predicted this disaster.

One more reason to never trust a companies "word". Show me the code.

Are all these edge cases part of the html/w3c/whichever standard? If not, let the edge cases fail. I'm not going to lose sleep over an icon not showing for a site I'm visiting once.

I agree. Sometimes it feels that companies throw any possible value out of the bus in order for the perfect UX.

I'll state this in no uncertain terms: this is not acceptable, and you need to stop doing it. It makes sense on your search engine, but adding it to your web browser is very much over the line.

I have read your explanations in good faith and they don't cut it. This behavior cannot continue. Good privacy promises are not based on trust - they're based on not ever handling private data in the first place. If you don't quickly admit your mistake and roll this back, it will jepoardize your entire brand - and rightfully so. If you believe this behavior is okay, then it demonstrates incompetence; if you don't believe this behavior is okay but do it anyway, it demonstrates malice.

This is the one thing you Should Not Have Done.

I'm surprised at how you're handling this. DDG is supposed to be friendly to privacy-aware users. You're dismissing people's valid points and asking them to trust you, just like any other privacy-non-friendly service would do.

Edit: I'm speculating here. But specifically because of the way you've replied here and on Github, my actual level of trust in DDG team went down.

What a bizarre potential privacy flaw to introduce for a tiny little icon nobody cares about. I understand it, usability and UX is important - but you guys are DDG! Come on! Your customers all have tinfoil hats!

> Your customers all have tinfoil hats!

Generally speaking. Mine is shielded with lead.

At this point we're all well aware why the app phones home. Continuing to spout that like it's some ward against the fact that this is a very real vulnerability is an insult. Trust me, your target audience doesn't give a crap if favicons work; they care that DDG acknowledges the risk of a glaringly obvious vulnerability. Who do you even think you're arguing with on HN and GitHub? My children can't multiply yet but they'd be able to understand why this is bad practice.

The repeated handwaving that no one in your company is ever going to do something bad or stupid when the browser phones home for what amounts to a cute sticker is extremely suspicious.

You're repeating what's on that page, which is exactly what everyone is worried about in the first place.

Maybe I'm too old for this, but wasn't a favicon supposed to be located at "fancy.url/favicon.ico", or alternatively as a "<link rel="shortcut icon" \>"?

Curious to know why this is an issue.

There are variations of it now due to mobile devices, social sharing etc that may prefer a larger icon.

An online favicon generator will create these variations

<link rel="apple-touch-icon" sizes="57x57" href="/ico/apple-icon-57x57.png">

<link rel="apple-touch-icon" sizes="60x60" href="/ico/apple-icon-60x60.png">

<link rel="apple-touch-icon" sizes="72x72" href="/ico/apple-icon-72x72.png">

<link rel="apple-touch-icon" sizes="76x76" href="/ico/apple-icon-76x76.png">

<link rel="apple-touch-icon" sizes="114x114" href="/ico/apple-icon-114x114.png">

<link rel="apple-touch-icon" sizes="120x120" href="/ico/apple-icon-120x120.png">

<link rel="apple-touch-icon" sizes="144x144" href="/ico/apple-icon-144x144.png">

<link rel="apple-touch-icon" sizes="152x152" href="/ico/apple-icon-152x152.png">

<link rel="apple-touch-icon" sizes="180x180" href="/ico/apple-icon-180x180.png">

<link rel="icon" type="image/png" sizes="192x192" href="/ico/android-icon-192x192.png">

<link rel="icon" type="image/png" sizes="32x32" href="/ico/favicon-32x32.png">

<link rel="icon" type="image/png" sizes="96x96" href="/ico/favicon-96x96.png">

<link rel="icon" type="image/png" sizes="16x16" href="/ico/favicon-16x16.png">

<link rel="manifest" href="/ico/manifest.json">

<meta name="msapplication-TileColor" content="#ffffff">

<meta name="msapplication-TileImage" content="/ico/ms-icon-144x144.png">

Nonetheless, the browser can see this when parsing the page and choose the appropriate path.

Geez, I've been out of the webdev loop for around 10 years. That's insane.

favicons should be SVGs and done with it. This is really insane!

It's really not an issue. The rest of the browsers handle this without a web service (even Chrome apparently)

This is obviously an insufficient answer. Why risk your one selling point on such a trivial bit of code?

Doubling down is fairly ridiculous when one has to imagine the original reason for doing this was to save on time and engineering for the app by leveraging what DDG had already built, but a mindful response would be that you are aware of the downsides of this approach and you'll be working to change it.

Besides, how do you handle Intranet, VPN sites, and auth-only sites where DDG's god-tier favicon parser in the cloud couldn't fetch the URL anyway?

How can users turn this off?


Uncheck the very last option "Site Icons"

This should be the top response. Just under it should be a DDG staffer telling us they will be turning this config option off by default starting with the the next release.

No, since it's incorrect. We're talking about the browser, not the search engine results.

I'm assuming that doesn't affect the browser, which is what this issue relates to.

Maybe you have to be logged in to DDG for this to work? Which if true feels that much worse.

What advantage does this give you? DDG just gave you a list of URLs, so how can it be "tracking" the results it just presented to you?

The issue, as I understand it, is that the Android app loads the favicon service for search results you actually open in the app.

Route icons.duckduckgo.com to null.

Sidenote: The more I use pi-hole the more I realise how essential it is!

Is there a difference between using pi-hole and edinting etc host on Windows? I've heard about pi-hole but I'm not sure what it does exactly.

This is essentially the same, but pi-hole gives you a nice UI, stats etc, and of course you can point all your machine at a pi-hole instead of editing the hosts file on each.

Are you able to block youtube ads using pi-hole?

Sadly, not anymore - used to be the case quite a while ago, but not today anymore as the ads come from the same domains the videos are streamed from.

Those simpler popover-ads which can be closed clicking an X in the upper right corner still are blocked tho...

Spilling my secret tho (and YouTube execs hate me for it!): i block YouTube in my mind and only rarely go to it if i really need to watch a video (which, for me, is rarer than i ever thought it'd be).


Or mpv (for single videos, or local playlists), or mps-youtube, or youtube-dl.

It's complicated and a quickly moving target. I use pi-hole plus local browser blocking like uBlock origin.

Please choose another hill to die on. This is just not worth it. Clearly it's possible to do this on device like mozilla did it.

The reviews are in for this response and they are bad. It's concerning that given the react it got, there's no edit addressing the concerns. The HN audience has to be the power user, bread and butter of a product like this and when you see a company ignore the concerns of a key constituency like this, their future almost never looks bright.

Technical aspects aside, don't you agree it's a legitimate privacy concern from the user's point of view?

It’s amazing how tone deaf technologists can be when it comes to privacy, even when they have nothing to gain by exploiting the user’s data. DDG’s response reminds me of Mark Shuttleworth’s argument that they “have root”, so we can trust them with our life.

Dear DDG, you are getting complaints on GitHub and Hacker News. This is not the general public, it’s people who understand the issue. You should definitely reconsider whether you’re doing something wrong.

> We use an internal favicon service because it can be complicated to locate a favicon for a website

That must be the worst justification for this possible. Favicons. Complicated to locate? Who are you trying to fool, 5 year olds?

The road to hell is paved with good intentions. At the end of the day, your privacy policy is just a bunch of words with nothing to actually prevent you from abusing the data collected. Instead of us relying on DuckDuckGo to act ethically, just don’t collect the data in the first place.

Can you tell, for instance, how many of your users visited site A?

Can you tell how many visited site A and also site B?

You are just repeating what's already written in the link, it isn't very useful. Try to address users' concerns instead.

Sorry but this is not enough reason. There is a simple question you should ask to yourself.

- Would you be ok to use a third party for this with same privacy policy?

This answer and other replies from yourself in this page seem copypasted from here: https://help.duckduckgo.com/duckduckgo-help-pages/privacy/fa...


I don't know how you can misunderstand your core demographic this badly mate.

If you think the next time I hit the shitter I'm not going to be looking for a new browser, you're dead wrong.

Just do the basic checks and then fall back to a DDG logo, no one cares that much about the favicon.

> the purpose of the request...

... is completely irrelevant. Even if they were trying to save babies from a fire (which they really aren't) it wouldn't excuse the fact that they're doing something orthogonal to their stated policy and sole reason for existing.

Everyone makes mistakes, that's not the point. The point is to correct them when they're found, instead of digging one's heels in the ground and pretending it's nothing.

This is a bit concerning that for a company with a marketing so focus on privacy, you guys thought it was a good idea to have such a service.

Nobody in the company at any point thought that it could be a problem?

Your strict privacy policy mean nothing by the way, because you are a USA company and you must respect your local laws such as the patriot act, which are not very privacy friendly.

come on dude. I thought you got a better answer.

Does it have an option to disable the internal favicon service and/or an option to disable favicons entirely? (I disable the favicons on my own computer, since I don't use them.)

so, essentially you blew privacy because of favicons? favicons??

favicon has fairly enough known meta tags in case favicon.ico url is lacking

I don’t care if I get the favicon or not... can I just opt out of this functionality?

From elsewhere in this thread:


Uncheck the very last option "Site Icons"

appears to be the search engine results _not_ the browser.

How if you type a url into the browser how do you stop the browser from sending that url to ddg to get the favicon?


They need to run international ad campaigns, what's that tell you?

What is this possibly meant to imply? Nonprofits run international ad campaigns, governments run international ad campaigns, the military runs international ad campaigns, most organizations could probably find a reason if they have international domain. What is your insinuation specifically?

There's an interesting disease showing up here in the responses.

I accept DDG's statement that this is about a favicon and that they "do not collect or share any personal information", and despite that, I also agree with others that DDG should be on the safe side and just stop doing this small thing. It's just the safer and more moral thing to do (So DDG, as many are suggesting, plz stop doing it. Today is good).

But... the reaction here is "they made a mistake, let's pile on like kids in a playground" ignoring the genuinely huger issue of the amount of info and mining that google et al. do. There's no measure of proportion in the responses, someone is making a mistake then there's a wolfish, pack-like desire to get stuck in and hurt someone.

Which is why politicians rarely admit mistakes, because it's taken as a sign of weakness, not strength, to admit you were wrong. DDG isn't the big evil on the web but from reading some of these you'd think it was the 2nd google.

This isn't about DDG, just the proportionality of responses in public errors and what society you'd like to have.

(no affiliation to DDG)

Oh, please. There's a million threads on HN complaining about Google and Facebook. Doesn't even matter if they did something, any post will still have those comments. You can't consider the proportionality of the response by just one thread.

It's really quite amazing that when a company that's hitched it's brand entirely to privacy first commits a big privacy faux pas, hides it for a year, and then doubles down on it not being a problem, you have somehow managed to turn the top voted thread to a discussion on the failings of other companies instead. Bravo.

>hides it for a year

Huh? It's an open source project. Maybe you consider them closing the issue on GH to be "hiding it" but I don't. And the plenty of people talking about it in that thread still apparently don't, either.

IMHO their response made sense. You're already trusting DDG with your search history which for most people might as well be a list of 99% of the domains they visit. If you trust their privacy policy for that, I don't see the big deal here.

I do agree that it should be changed, but only because as is it seems imperfect and I seek perfection in most things. Again, it's an OSS project. AFAIK, any one of these people comprising the screaming masses are free to fix it themselves. Personally, I don't think it's worth my time.

The point is DDG has insisted that they put privacy first, & don't collect user data, then (supposedly) in the interest of a small performance increase they introduce code that — wait for it — COLLECTS USER DATA. All they would need to do is use {URL}/favicon.ico which most if not sites have. From what I understand according to posts on the github page this could be done on-device so no need for sending anything to a server. Then when the issue was opened it was closed without the issue being fixed.

It was not IMHO meant to deflect from DDG. It was a more general statement about peoples responses. I dislike trendy phrases but the one in play here is "outrage culture". Its not good and it's getting more common.

Thanks, that was exactly my point, which some people are missing.


This breaks the HN guidelines against accusations of astroturfing or shilling without evidence. If there's one thing I've learned from looking at the data for countless hours, it's that internet users are far, far too quick to jump to this interpretation of what they perceive on the internet. The overwhelming majority of such accusations are purely bogus, and they poison the community, so we don't allow them here.

Actual evidence is a different story, of course, but then you should be emailing it to hn@ycombinator.com so we can take it seriously and investigate.

Lots more explanation at https://hn.algolia.com/?sort=byDate&dateRange=all&type=comme... going back years now.


What would "actual evidence" look like? Proof that the comments in question originate from a known social media marketer? Do you think they all share a specific IP?

> It's because DDG has been shilling on HN for quite a while.

We’re they caught shilling or something, or are you just upset about just normal promotion?

I think what angered people was actually that a company saying to hold privacy high was simply refusing to change something after a mistake was pointed out and instead kept on defending it with a technical argument, which makes no sense at all.

The reaction would have been actually a lot different if someone from the company admitted the mistake and promised it will be changed.

Update: Gabriel Weinberg has promised to change it, linking it here so it does not get buried in the pile of comments: https://news.ycombinator.com/item?id=23711597

Read your comment again.

You are faulting someone for defending thier own argument. You suggest that people who do not cow and apologize to the mob deserve the anger and retribution the mob has to offer.

People have a right to think differently and express themselves without threats, bullying, or shaming.

The mob does not deserve apologies. The comment above is spot on - we've lost all sense of proportionality.

It is an indication of the modern online mob sickness that they always demand others beg for forgiveness.

What emotional void are mob participants trying to fill with the apologies of others?

i cant fully agree.

you obviously should be allowed to make a mistake and be forgiven for it. that does not mean that i personally would ever forgive any `company` that markets itself as pro-privacy after its been caught gathering data on its users.

i could forgive the people working at the company and would definitely expect future employers not to hold that against them, however.

but if a `company` does something while claiming to stand morally opposed to exactly that.... proves that it doesn't actually care about the topic. it just wants the publicity for marketability, discrediting them entirely for all future communication.

in this particular case, i wouldn't go that far however. they weren't gathering any data on their users if i understood it correctly. it was just a badly implemented feature, which will get changed


Not particularly.

I was specifically responding about your outrage how internet 'mobs' demand forgiveness from the people they've supposedly wronged. The whole comment was just me talking from the perspective of a possible self identified victim and how that person (me) would respond in such a case.

Forgiveness would've to happen for me to trust that party again, which would be a requirement for me becoming a user/paying for the service again. If that possible other party doesn't care about wherever the people they've supposedly wronged use/pay for their services, then there is obviously no need for any interaction after that point.

And as I said before: none of this applied to ddg, as they didn't spy on their user. They implemented a leaky feature, which they've committed to changing.

This is a really strange take.

Once people have gone down the avenue of earnestly reporting private information leakage, the correct answer is to investigate. DDG decided not to do this and instead dismissed the problem completely out of hand and ignored it for almost a year without taking any action.

No one asked for an apology, they asked DDG to admit fault and then to fix the problem. ie "we shouldn't have done that" not "we're sorry we did that."

You keep saying "mob" but these people didn't collect to harass, if you actually read all of the comments the vast majority are people who are (rightly!) concerned about their data privacy advocate built software leaking every visited URL.

You’re absolutely ignoring the possibility of an argument where there is no fault and this is being blown out of proportion. You just assume your opinion is correct and they owe you to fix it. That is in itself the problem OP was exposing.

Great! Can you provide that argument or am I supposed to think it up myself?

>defending it with a technical argument, which makes no sense at all.

Well hold on now. If there's a valid technical argument, and it's not a violation of privacy, why doesn't that make sense?

If people are so distrustful of DDG that they don't believe that argument, why use their browser under any circumstance?

A performance optimization around a trivial thing like a favicon is no reason to destroy everyone's security.

Let's assume DDG is a great and honest company and will collect all the info, but never use it for anything bad. Guess what, they can get still hacked and all the info leaks out.

This is a horrendous breach of trust that they WERE collecting it however, and I'm glad they got caught. It will not be tolerated. They'll have to change this or face a revolt.

Yeah people burnt witches in the past ...

If they are confident that this feature has no privacy implications they are right to defend the point, despite what people say or think

People don't run DDG, the company

People can use another search engine if they want

I'll keep using DDG anyway

DDG can do what they want and people can as a result publicly state their opinions on it. Freedom goes both ways.

You’ve described the race to the bottom we live in. Can we please try and break the cycle?

Those witches have a right to be on fire.

Where's my pitchfork?

If you claim to fight for privacy and rise to popularity by shaming your competitors' evil anti-privacy (or shady) practices and do something exactly what the rest do, you deserve every bit of criticism. Why should you receive a free pass and the competitors shouldn't? Simply because of your marketing stance??

And for the record, collecting your browser history just to display a stupid favicon is the most ridiculous excuse I've heard in a long while. And I am not going to blindly believe them because they said that's what they use it for.

I agree with this.

As a privacy-first company, for them to make any decision that is for performance but adds additional privacy risk shows they make the wrong decisions.

They also left this for over a year after being told about it and part of the only reason it was caught was that their browser app is open-source. What about all their closed-source services, like their favicon service their browser apparently relies on.

We shouldn't blindly trust any company and a privacy-first company should be willing to assume we don't trust them and therefore it is up to them to prove everything they do.

I get a few of the responses on their Github page are now non-helpful, but the big point here is that by DuckDuckGo taking this series of actions (implementing the browser like this, ignoring a valid bug report about it, and only reopening the issue after massive push from users) it shows that DuckDuckGo isn't as privacy-focused as I thought, nor many others.

I've been using DuckDuckGo as my sole search provider for several years now, on all my devices, but I'm concerned about what else they may have done in the sake of "performance" over privacy.

"collecting your browser history just to display a stupid favicon is the most ridiculous excuse I've heard in a long while"

Though I agree the the implementation could be better, they should just check the head and the root for the icon and if not found that's that. But the possibility of something be used for malicious ends does not entail confirmation of it being used that way. Just because we have knifes in our kitchen does not make us automatically guilty of stabbing people. I find it easier to believe that this was actually, if misguided, an attempt to solve a problem rather than a nefarious plots to track users across the web.

> But... the reaction here is "they made a mistake, let's pile on like kids in a playground"

It's not one mistake, but several. Other then the initial mistake there is also the sloopy reaction and the fact they just closed the issue without bothering to fix it. And this was 1 year(!) ago. Nothing changed in the meanwhile. Now someone pushed it to public and after just some hours they reopen the issue and promise to fix it.

This is the reason why people react loud, because it works. And often it's even the only way that works.

Part of the point is that likely this has zero privacy implications (except potentially disclosing to someone monitoring your traffic that you are using their browser).

The mistake is rather an area of improvement where they can change something that respect privacy by policy to something that respect privacy by design.

>Part of the point is that likely this has zero privacy implications (except potentially disclosing to someone monitoring your traffic that you are using their browser).

It has zero implications if you trust DDG and good privacy is not based on blind trust. Keep in mind that you also need to trust the government under which DDG acts to not require them to disclose this data, trust the government to not put black boxes in the DDG data center, trust DDG's security apparatus against external state actors, trust any rogue DDG employee to not use this data and so on.

I agree, I still think it is relevant to treat actual privacy violations differently from engineering choices that might make privacy violations easier in the future.

My opinion is that DDG should have never made this choice in the first place, but as far as I am concerned this is at the level of an implementation detail that can be improved, not as if DDG was intentionally using user data in some non-private way.

The concern as I see it is that this issue and the initial DDG response to it shows a lack of understanding of technical privacy controls. What they are and why they matter so much. DDG's backend is not audited and so one wonders if that same lack of understanding applies to the backend as well. DDG cites privacy policies but, in my experience, the best policies are backed with strict technical controls as humans never follow policies perfectly on their own.

Google don’t sell privacy.

Anyways, what’s this got to do with Google? “Privacy browser violates basic privacy to do something useless” is actually ridiculous. And the response is even more ridiculous! How does literally every single other browser do it?

I’m disappointed because I put my reputation on the line to recommend DDG to users based on...privacy. But here we see they actually do not hold true to their stated values. And they don’t even seem to care.

> Google don’t sell privacy.

Of course they do. They sell users' privacy for advertising dollars.

I am not defending DDG here, as they are clearly in the wrong - but let's not pretend that their error is even close to what Google does.

It’s an explanation of why complaints are being directed at DuckDuckGo (privacy is their raison d’être) even when Google is worse (privacy is not their product – or, in wording that’s more ambiguous in isolation, Google don’t sell privacy).

Can you explain how they sell "privacy"? This is thrown around all the time but I don't think it means what people think it means.

Do they sell PII data ?

Because I don’t think they do.

"Google don’t sell privacy."

What does DDG 'sell'?

Which is to say, how much have you paid them?

If you're not paying for it, then you are the product, that's the inevitable trade. There is no free lunch.

> ignoring the genuinely huger issue of the amount of info and mining that google

By using DuckDuckGo you're doing the opposite of ignoring privacy issues in Google products.

And hence the reaction. Why use DDG at all, if they're not safely protecting your private data 100%?

Because some protection is better than no protection

Why use a condom which doesn't protect you 100%?

Or why wear a motorcycle helmet since there's still a chance you'll sustain a serious head injury and die if you crash? Like you, I find the extremity of the GP's point of view lacking in wisdom.


I don't know how the rules are here about getting personal, but aside from making other users be uncomfortable you're missing the point completely.

I don't personally feel a need for a high level of privacy, but I can understand other people who do. And if they are using DuckDuckGo for that purpose, this issue would seem like a big thing. If you served a dish for a vegan with 5 % meat in it, you wouldn't justify it by comparing to the amount of meat in non-vegan dishes.

I'm fairly certain that personal attacks arent allowed on this platform...

This isn't just a mistake. They're saying it's not a problem. Do you agree with that? If not, you're arguing with the wrong person.

This is my feeling exactly

You're okay with them having data on what you're searching for, but them potentially having data on the domains you're visiting from those search pages is where you draw the line?

I mean, I agree with the principle of storing as little data as possible but this isn't a huge deal in my eyes.

I disagree, or maybe we read different responses, but the ones I read where more critical of how (a) ddg (employee) handled this.

Didn't see anyone claim that this was on a google-level of bad, more like pointing out that google started out as a small company wanting to "do no evil", but slowly turned into what it is today.

Is it really that weird that people are worried that this might be the first of many small steps down the slippery slope?

I didn't have to go through the posted link for long to find people reacting as though this was more than a slippery slope issue:

> But instead to react professionally and contritely you made it worse to stamp on the shards to make sure no useful piece of trust will survive.

> I've just de-installed the Duckduckgo app and also won't be using their search engine anymore. Trust is lost. Their CEO can put his statement where the sun doesn't shine.

> Didn't see anyone claim that this was on a google-level of bad

The point is that people are reacting as if it was.

> But... the reaction here is "they made a mistake, let's pile on like kids in a playground" ignoring the genuinely huger issue of the amount of info and mining that google et al. do. There's no measure of proportion in the responses, someone is making a mistake then there's a wolfish, pack-like desire to get stuck in and hurt someone.

Sadly people simply derive satisfaction from piling on like this. The pop psychology explanation is that it's because people are dissatisfied with their own lives and are lashing out at anything that allows them to vent that underlying frustration. It sounds plausible, but it also sounds like it might be an over-generalisation.

I think it's certainly fair to say - as depressing as it is - that it may be somewhat in our nature to behave in this way, that most or all of us may possess this characteristic to a greater or lesser extent, and that the current political, cultural, economic, and media climate is only serving to amplify that tendency.

Also with the current social media platforms, it's really easy to find and participate in such pile up.

The current social media climate favors disagreement. This is directly in correlation with 'engagement' around which platforms are designed. Call it banking on outrage/hacktivism/quarrelling or simply engaging in peaceful debates. Companies and their algorithms have come to a funny conclusion that this is what increases said engagement (I'm not questioning their business model). User retention is crucial in this era, so is the intention of getting them to check back in ever so often.

Thus, dark patterns emerge. Point is, once you get used to 'that' online-aura, you unwillingly carry it with you wherever you go. Human brain does not have buttons to switch between modes. It's comically easy to get into this mindset and hard to shun it.

My guess is that DDG is being held to a higher standard because it's the liferaft competitor in a world of Google data harvesting, targeted manipulation and selling customer data.

Most complaining or being heavily critical about DDG are probably already upset to the point of abandonment with other services and they don't want the same trend to happen to this competitor (DDG). This sort of reaction is, IMHO, due to poor diversity of viable competitors.

In our societal structure, competitive options are the only things that keep power in check. I'm personally not entirely convinced you can have a reasonable amount of diverse competition in our economic system and there are some inherent equilibria that we tend to converge on over and over again in a market space (without corresponding massive social equalibria shifts).

If you do have any sort of faith left in our economic system, then you certainly want competitors like DDG to be different and be successful. Even if you don't have much faith, outside of say stringent regulation, supporting these sort of competitors is really the only practical option we have in the current state of affairs.

> There's no measure of proportion in the responses

Man. You just described 2020.

Anyway, I’m now using the DDG browser, which until today, I didn’t know existed. I think DDG will do the right thing, ultimately.

Many argue that "other things are worse". This argument is invalid in my opinion. We're talking about this special case and if people feel like they need to openly show their opinion about it this is okay. Even if the response is big in numbers.

That just shows how much also the small things matter.

If you only care about "the biggest" or "the worst" you'll never get anywhere...

You're right mr throwaway, let's be adults here and stop using DDG.

What would you suggest as a better alternative to protect your privacy?

... any other browser?

After we have been stung for the umpteenth time patience starts to wear thin.

> But... the reaction here is "they made a mistake, let's pile on like kids in a playground" ignoring the genuinely huger issue of the amount of info and mining that google et al. do.

What about genocide too? Please people, stop with these "but the other bigger unrelated issue should get more visibility".

> Which is why politicians rarely admit mistakes, because it's taken as a sign of weakness, not strength, to admit you were wrong.

Can you point me where DDG admitted they were wrong doing this in their first response? They didn't... they just explained why they did it but completely ignore the greater issue because they consider themselves "good". Just like that politician you may talk about, or Google, or whatever. They are part of that bigger issue you mentions.

This is about DDG.

Luckily that pile of kids in a playground made them realize that mistake, they would have ignored otherwise (like they did on their first respond).

> But... the reaction here is "they made a mistake, let's pile on like kids in a playground" ignoring the genuinely huger issue of the amount of info and mining that google et al. do.

What about genocide too? Please people, stop with these "but the other bigger unrelated issue should get more visibility".

> Which is why politicians rarely admit mistakes, because it's taken as a sign of weakness, not strength, to admit you were wrong.

Can you point me where DDG admitted they were wrong doing this? They didn't... they just explained why they did it but completely ignore the greater issue because they consider themselves "good". Just like that politician you may talk about, or Google, or whatever.

This is about DDG.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact