Hacker News new | past | comments | ask | show | jobs | submit login
Encrypted email is still a pain (incoherency.co.uk)
544 points by jstanley on Feb 13, 2017 | hide | past | web | favorite | 431 comments



Encrypted email is pretty much over in 2017.

The emerging consensus among experts is that it's not worth the trouble, or, worse, incapable of doing much more than generating a false sense of security. That's for a bunch of reasons:

* An enormous installed base of clients that won't do encryption, meaning that at best you're attempting to tunnel encrypted messaging over an unencrypted transport.

* A protocol that leaks metadata, including some message content, at the envelope layer.

* Hundreds of millions of users that primarily access messages through browser clients that can't meaningfully implement crypto.

* An archive-always UX that ensures that huge amounts of plaintext are scattered around the Internet by both senders and receivers.

* An unencrypted installed base that ensures encryption will be opt-in for the foreseeable future, meaning that users will routinely reveal plaintext accidentally by, for instance, quoting messages and forgetting to encrypt.

* End user demands for things like search that can only be delivered efficiently at scale by databases of plaintext (most likely at centralized servers).

All these problems are probably surmountable (with enormous, concerted effort). But: why bother? Email is just one of dozens of messaging systems available to Internet users. Better to move sensitive conversations to things like Signal, WhatsApp, or Wire --- the double ratchet construction is designed specifically to make IM-like protocols secure even when conversations are sporadic and last months.


> The emerging consensus among experts

"conseunsus"? a few blog posts about some bad user experience with GnuPG / the PGP ecosystem is, at best, just an (re)emerging topic on HN, not the end of email encryption. OpenPGP implementations may not be the easiest encryption software out there (its usability issues have been discussed for two decades now) but that's simply because PGP was not designed to be used by the laity, which, doesn't necessarily mean that you need to be an expert to use it: my friends and I learnt how to encrypt, decrypt and sign emails when we were still in high-school. And while you probably need to read a manual (something that may sound draconian by the current standards of the software industry) to learn how to use gpg, there is no other cryptography tool as powerful, versatile and secure as this.

Why bother? because e-mail is federated, Signal, WhatsApp, and Wire are not. Because you don't need to put all your trust on a device like a mobile phone, or a company like Facebook. Because you don't even need e-mail nor Internet to send encrypted mails to some journalist, for instance. But please don't get me wrong: I really appreciate the effort of messaging apps like Signal for casual conversations, but if my own life was at stake I wouldn't even touch my phone (except for taking out its battery).


You know, I've been having this conversation ever since PGP first came into existence. And much as I love the idea of encryption, and despite having invested lots of time in arguing for the right to encrypt and to share encryption algorithms etc. etc. I've always had to admit that if you're not a geek who loves computing for its own sake then encrypting all your email is a massive pain in the ass, whose costs substantially exceed the benefits for most people. Given that this argument has been going on for 20 years and that hardly anyone encrypts their emails on a daily basis, I'd say that the empirical evidence is that your solution, while clever, is Not Good because it doesn't meet people's actual needs.

Stop telling me why you like it and build something that's easy for other people to use. In your pursuit of technical excellence you are completely ignoring the importance of network effects on adoption and the disutility of standing out from the crowd by your use of super-solid encryption. Nobody wants to maintain a collection of public keys for every single person they know. No matter how bad things get politically there is not going to be a sudden mass awakening that will cause everyone to start using public key encryption for email, or we'd have already seen it take off like wildfire in politically repressive jurisdictions.

It's. Not. Going. To. Happen.


These are good points but I'll nitpick on one of them: we already maintain a collection of emails, phone numbers, etc. for everyone we know, and a public key is just one more data point in the contact list.


Totally agree; it's just been my experience that crypto specialists don't care about UX any more than UX people care about crypto.


Whoah. Hang on there. Crypto specialists aren't the people lobbying loudly for PGP-encrypted email in 2017. They're the ones who made the world's most popular messaging application double-ratchet deniably encrypted by default without the userbase even noticing.


I'm sorry, I undermined my own point in pursuit of a witty syllogism and I don't want to give the wrong impression that the crypto in apps like Signal (which I prefer) is somehow second-rate.


Wait: which one is that?


Singal, nee TextSecure, I'd assume.

Edit: Crap, I mean WhatsApp, which _uses_ Signal's protocol now.


Does WhatsApp have reproducible builds yet? Do we know that the app isn't storing the keys somewhere else?


> Does WhatsApp have reproducible builds yet?

WhatsApp isn't even open source, so no, it doesn't have reproducible builds. (Signal is open source and does have reproducible builds).

> Do we know that the app isn't storing the keys somewhere else?

No, but WhatsApp is still strictly better than either email or SMS. Even if you don't trust Facebook, that still only leaves three parties that can compromise the integrity of messages (you, the recipient, and Facebook), as opposed to the uncountable number of parties that can passively observe email or SMS in transit.


Gotchya. Re-reading Thomas's comment, one of that set at any rate.


That's true.

Seems like a great use case for a password management company to add address book management and/or develop an email client.


I'm hoping Apple will lead the way. They have enough devices in the wild that could force others to adopt over time. They did it with floppy drives and optical drives. I don't see why they couldn't do it with encrypted e-mails. With their stance on no backdoors, they could raise their privacy profile for consumers and increase sales of iPhones at the same time.


The fact that Osama Bin Laden didn't use PGP should be the final nail in its coffin.


That the NSA said GPG was gsme over for mass collection in the Snowden lesks should be a reason for everyone to try to improve its UI.


thats a pretty terrible argument. And how it follows for your own logic is beyond me.


That the NSA cracked, backdoored, or intercepted most providers people trusted but couldnt beat GPG isnt an argument for GPG being secure? I think it's quite an endorsement for GPG given most people's adversaries will be weaker than NSA.


1) I don't take the snowden leaks as gospel, sorry. 2) Even if i did, "most peoples adversary" is a meaningless phrase at this point, and also used quite often as a rhetorical feint to take down someones arg. And given the profound unification of the security state across seemingly all lines, its also dead wrong. Technically everyones adversary is the NSA, as long as data is shared surreptiously and , more and more, openly and legally between TLA's, state, and local LEAs. 3) GPG may be an excellent tool, the first time you use it, but if you transmit anything encrypted you are automatically targeted, another point directly from the snowden docs, no? And since virtually no one is going to use one time devices and farraday cages unless your model of communication is "I just have to get this one message out, then I'm good" its worse than useless, given that it will only make you more of a target.


"1) I don't take the snowden leaks as gospel, sorry. "

Don't take them as gospel. That's faith. Review the evidence they're true from U.S. governments' reaction to them to what similar malware was found by third parties. Once evidence is in, then you have reason to believe them and then in stuff such as GPG by extension. And the leaks didn't say anything about faraday cages. Just that they had to rely on the extremely-limited resources of TAO... such as targeted attacks on specific sites/configurations/endpoints... if the target used something strong.


the idea that "evidence" comes from the govts reaction is just weak, its just a terrible argument. Extrapolating from that is also largely a mistake. Any number of possible interpretations of the docs themselves and the responses by the state could and have been made, each of which could point to mutually exclusive conclusions for toosl mentioned in the docs. This isn't a particularly novel argument (my arg, I mean) either. I wasn't pulling the faraday cages from the docs themselves, as well, I was positing that as an extreme example of data security.


Your comment is its own best response to itself.


except that its ....not? You don't have to treat the snowden leaks as gospel, you know that right?


Even the government stopped denying their truth, especially after the drip feed of info caught them in baldface lies on several occasions. Also, if the info wasn't genuine, why would they be so upset and calling Snowden a "traitor" etc?

Additionally, this wasn't even the first time any of us had heard about this sort of thing. Ever heard of Room 641A? The Snowden leaks aren't exactly implausible or unprecedented.

I don't know if you are genuinely unsure of their veracity or just making a rhetorical point, but it's not a point of contention in any serious debate forum I know of, even among intelligence community sympathizers.


> "consensus"? a few blog posts about some bad user experience with GnuPG / the PGP ecosystem is, at best, just an (re)emerging topic on HN, not the end of email encryption.

Well, tptacek is himself an expert. As he's the founder of a successful security consultancy who has friends among academic cryptographers, I took his comment to mean the belief among himself and his peers.

It's ultimately an appeal to authority, but it's a useful data point. Maybe there are different "circles" of security/cryptography experts, and tptacek runs in a different one from others who haven't given up on email, but I suspect he would have said that if that's the case.


>As he's the founder of a successful security consultancy who has friends among academic cryptographers, I took his comment to mean the belief among himself and his peers.

Security consultants are who you should trust the least when dealing with security. They aren't interested in reliable, easy-to-use and widespread security. They are interested in difficult, interesting security that breaks and needs consultants.


And all developers purposefully write spaghetti code while sysadmins hide passwords and secret configurations, all for job security.

Do you truly believe everyone is so cynical? Of course there are some bad actors, in all jobs and all domains. But not everyone—I'd argue the majority of people—are just out to screw everyone else.


Security consultants don't need to engage in sabotage to keep job security. User incompetence is adequate.


Let’s say that what you wrote holds true. Who should we trust then, when it comes to security advice?


Why use PGP anymore when you can use Keybase and the next generation of key management?

Instead of having one master key for your identity, the paradigm is changed:

Identity is a set of claims "X on domain A is Y on domain B". That's it.

"Domain" can refer to a server-based service such as reddit, or a client app on a device.

Such proofs are easy:

1) For public identity on sites which don't support this scheme, X simply signs something with their private key for A, and posts it on A at a URL only they control (such as an About page or a Facebook post). Presumably, X and A had to cooperate to do that. Now, to prove "you are X on A", you simply claim your your identity and the url to check, and sign it with the same private key. The verification proceeds by checking the https url for the actual claim. You hold that key. If A wants to revoke your identity proof, they just delete it from eg your profile. Each site A has to be supported individually by the client lib though, because "urls you control" are different on each site.

2) For private identity on sites that cooperate, there is a huge variety of things you could do. Everything from oAuth 2, to the signature scheme above, except the user id X on A can be different for each relying party B.

So the claim "X[B] on A is Y on B" is a private claim - others can verify it only if they identify themselves as B (eg X must be logged into A and using https on A and B and doing cross-domain communication using iframes).

They can't satisfy these conditions unless they spoof B's SSL certificate and trick X into visiting their site via a poisoned DNS. And in that case they can just find out that X is X[B] on A. Very limited privacy leak. They still can't decrypt or sign anything as X on A.

This is the scheme anyone can implement and use. It should be standardized IMHO somewhere.


There is an RFC extension for OpenPGP that does just that: https://tools.ietf.org/html/draft-vb-openpgp-linked-ids-00

It's implemented in Android OpenKeychain.


Planned to say this as well. The Keybase model is working well for me, and there's nothing special about that network: similarly implemented alternatives could probably strengthen one another.


I would love to be able to use Keybase to send encrypted email. Sadly currently Keybase does have a email as a main property on the profile and fetching it from the GPG is not very cool.

I really want to be able to type Twitter, Hacker News or Github names into emails.


You can just use Keybase to distribute PGP keys though, right?


I've been using email since 1992 in a huge variety of work, study and personal contexts. I installed PGP now and then in the '90's out of curiosity but never sent a single encrypted email as I never came across any recipients with it installed. I have never had anyone request email encryption. I don't know what the expert consensus is. But the user consensus is that PGP is invisible, and will never be used outside of a few tiny niches.


The hardest problem, IMHO, has been key management. How do you get+trust the other's key?

I think a combination of keybase + a useful client can help, but the reasons listed in parent are pretty convincing.


If you care about the physical identity of someone: web of trust.

At some point, you'll have to ideally meet at least one person in the flesh to exchange keys and verify their identity. After that point, it's possible that others you are trying to communicate with might be within your web of trust. If not, you'll have to go through your keysigning procedure again.

  https://www.gnupg.org/gph/en/manual/x334.html
Some organizations facilitate keysigning:

  https://wiki.debian.org/Keysigning/Coordination
But that might not be necessary for you. For example, I don't necessarily care about the physical identity for some of the people I communicate with online. If I see in e-mail archives that person is using the same key to sign their mail for the past N years, I'll use that key to encrypt to them. Similarly with commit signing and such. In that case, I just care that my message is reaching the intended recipient.

I communicate with a number of GNU hackers. Package maintainers upload their signing keys to Savannah, and sign each of their releases with that key. If I simply want to know that my message is reaching that maintainer, I can get the key that way.

But if I want to know that I'm actually speaking to the person that the maintainer _claims_ to be, I'd want to use the web of trust. They could very well be an imposter!


> After that point, it's possible that others you are trying to communicate with might be within your web of trust.

The problem with the web of trust is that it simply doesn't work: the fact that I know you means nothing about whether I trust you to vouch for others. The fact that I trust you to vouch for employees of Acme Widgets means nothing about whether I trust you to vouch for members of the a political party.

PGP's usable despite the fact that the Web of Trust is kinda a misfeature.


If you don't trust that individual to vouch for others then they are treated differently in your web of trust: set their trust level to "none".

If you refuse to place trust in anyone, then no, the web of trust will not work for you. But it works for many others; it doesn't make it broken. The purpose of key signing is to verify that a person is legitimately who they claim to be---_that_ is what you are trusting in your web of trust: that someone has verified their identity in a means consistent with accepted protocols.

If enough people say "this person is who they say they are" by signing that person's they, then you decrease the odds that the person is a fraud.


> If you don't trust that individual to vouch for others then they are treated differently in your web of trust: set their trust level to "none".

Whom in the world do you actually trust to vouch for everyone else in the world? For me, at least, the answer is 'no-one,' — which is why neither XPKI nor the Web of Trust work for me.

> But it works for many others; it doesn't make it broken.

I suspect that no-one (older than, say, four years old) trusts any other human being or organisation to vouch for every other human being and organisation, and thus that the WoT is in fact broken for everyone — but that most folks just try to ignore that.


I have confidence in certain people that they will follow a given protocol to the best of their ability. They're not vouching for someone: they're indicating the successful completion of a keysigning protocol.

But again: you don't have to trust a single person. As more people sign Alice's key, it's increasingly unlikely that Alice fooled every one of those people.


>The problem with the web of trust is that it simply doesn't work: the fact that I know you means nothing about whether I trust you to vouch for others.

Actually it means a lot. That's how trust works in the real world as well.


You don't know any deadbeats you trust less than a random person selected from the population at large?


The "web of trust" is not about trusting everybody you happen to merely know.

It's, and the name is kind of a hint, about knowing those you trust -- it's a web in that there's higher level trust (people you personally know and trust yourself), secondary trust (people trusted by those you trust), etc.

And in cryptography it's even more specific: https://en.wikipedia.org/wiki/Web_of_trust

It's not in any way about trusting someone just because you know them.


Point, but I'll refer you to my following sentence: 'The fact that I trust you to vouch for employees of Acme Widgets means nothing about whether I trust you to vouch for members of a political party.'

The Web of Trust assumes that I trust anyone to vouch for everyone (interesting, TLS — itself also a product of 1990s crypto-thinking — makes the same assumption). But I simply don't. I don't trust my nearest & dearest family & friends to vouch for every identity I care about. But I do trust some of them for some identities.

I trust myself to validate possession of any key. I trust my employer to validate possession of keys related to its work, but not keys related to, e.g., my family or my blog. I may trust one of my brothers to validate keys related to his immediate family, and maybe I trust two of my brothers to jointly validate keys related to our family, but I don't trust them for work, or my blog. I may trust my blogging co-admins to validate keys for roles in the blog, but that doesn't mean they get to validate keys for identities at my employer, or validate keys on behalf of my parents or children.

I could use different email addresses for each identity (I-the-employee, I-the-son, I-the-blogger), and have each identity trust only those who are pertinent to it, but that makes PGP more, not less, difficult. And it's certainly not the model that PGP advocates.


TOFU works within a number of contexts. Though not all.

Generally, out-of-band or reference-based (e.g., third-party vouching) of identity.

Web-of-trust is useful but IMO ultimately a tool of limited use, and presents numerous issues with data disclosure, as it is an independent and public validation of social networks. Often who knows who is more critical than who says what to whom.

PGP-over-email unfortunately leaks massive amounts of metadata.


>How do you get+trust the other's key?

Snail mail + several other out of band methods. Or you can exchange a one time pad, physically.


This also raises the question: With a walled garden like Signal/Wire/etc, how do you get+trust the other's key?

Their convenience comes at a cost.



Yup, and I've done these. But you can do similar things with PGP public keys, too.


I don't understand. You've verified a Signal key but still felt the need to ask the question With a walled garden like Signal/Wire/etc, how do you get+trust the other's key?

What cost were you talking about then?


Sorry if my earlier comment came off as rude. I was just trying to say that the walled gardens aren't really that much better than plain old PGP, and in practice they tend to lull people into a false sense of security.

I think too many people are way too trusting of shiny new apps with a pretty UI. If you don't do the extra work of verifying the key, you're effectively letting the service provider act as your one and only CA.

If you can verify a Signal key fingerprint, you can verify a PGP public key fingerprint.


I didn't think it was rude, it just contradicted your other comment.

I still don't see what cost you were speaking about. Mistakes with PGP are at least as likely as mistakes with the shiny easy to use GUI.


One idea: send them a signal (hi, this you?) then immediately dial their number and confirm.


Ideally dual channel. Send them a message on their mobile phone, call on the home phone (if they have one).


There are federated options for messengers, the fact that the current darlings aren't is not a mark against the option itself. Riot exists.

Can you find a security expert RECOMMENDING email? That would be a better example of how it's not a consensus, like you claim.


As far as I can tell, encrypted email is still how you reach out to CERT, security teams at distros such as Ree Hat, Debian and so on?

These people might not be crypto experts, but hopefully many of them are security experts.

Gpg is also how most mailing list communication is (clear-)signed - and that coupled with public archives does give a way to verify that the person that controls the key that signed release notes for this package these last five years, is the person that will be able to read this zero-rated report that is critical. (It says very little about said owners real identity, or his or her legal name)

What does FreeBSD, OpenBSD or Oracle recommend for sending sensitive information to security@<company|project>?

That said, modern email suck.

Mutt (possibly with "not much") might "suck less" - but we need many more, better (graphical) email clients. I think Fastmail's work on a json-based client protocol is interesting (not because it's json, but because every blank staring IMAP-client writing developer keep saying that there are terrible horrors laying in ambush for the unwary).

Opera had a nice, new-ish, fast mail client. Other than that I'm unaware of any serious effort to make a new, modern, easy to use IMAP client. Let alone an open source one. Or one that doesn't beg to expose library bugs by rendering html, images etc in-line - or ignore user privacy by loading external resources that enable user tracking.

I've been contemplating writing one for quite some time.


> every blank staring IMAP-client writing developer keep saying that there are terrible horrors laying in ambush for the unwary

Oh, my, yes. Lasciate ogni speranza...


For those(like me) who are looking for the famous reference Implementation Vector, it got a rename[0] to Riot[1] lately.

[0]Rename: https://medium.com/@RiotChat/lets-riot-f5b0aa99dc8e#.3toozs7...

Homepage: https://matrix.org/docs/projects/client/riot.html


Riot might be a great platform for doing business, but it's pretty useless for any other kind of activity. If you're a political activist having an app called 'Riot' on your phone or computer is not going to look good to anyone in law enforcement.


"having an app called riot" is the absolute least of an actual activists concern when selecting something to help them communicate. One of the most well known leftist platforms for email is riseup.


I'm aware of that, and I think flagging yourself by choosing such a branded platform is the height of stupidity. Maybe my social circles are just more privacy-oriented.


well, it's FOSS; someone could maintain a fork with a less controversial name (I hear 'vector' is up for grabs O:-)


Whilst you focus on "consensus" as the key word, I focused on "emerging" as the key word. Perhaps "a few blog posts" is where this consensus is emerging.


> But: why bother? Email is just one of dozens of messaging systems available to Internet users.

No, it's not. It's the only widely available, decentralized system, with which you can send to anyone, if you know the address. None of the big ones is this open.

XMPP tried to address this and failed; now Matrix is trying again.


WhatsApp has over a billion users. There are big places where its market share exceeds that of SMS --- another big centralized service that has a userbase comparable to that of email. My conclusion is that the people who care about "decentralized" systems are a rounding error. I care about non-technologists managing to send asynchronous messages to each other that are well-encrypted by default. That's a solved problem.


People who care about encryption are a rounding error too. I care about non-technologists managing to send asynchronous messages to each other that are not controlled by a centralized entity (especially not one based in a country who's interests are often adverse to my own). That's a solved problem that you seem to be trying very hard to unsolve.


That's exactly the point. Take a step back and think about what you just said. It's true: most people don't care about crypto. But here's are two other true statements:

* In modern messaging protocols, they don't have to care about encryption. The protocols are designed to reliably encrypt messages without user intervention, and security isn't "opt-in".

* The people who most need encryption are not the ones who are most aware of the need. In fact, the Venn diagram of "need" and "want" for crypto has very little overlap.


> In modern messaging protocols, they don't have to care about encryption. The protocols are designed to reliably encrypt messages without user intervention, and security isn't "opt-in".

Sounds good. Doesn't sound worth giving up decentralisation for. Doesn't even seem like something we'd need to give up OpenPGP to get - if client design were equal (and it isn't at the moment, but I see no reason it can't be) I'd far rather have that client experience but with the more established/tested protocol.

> The people who most need encryption are not the ones who are most aware of the need. In fact, the Venn diagram of "need" and "want" for crypto has very little overlap.

I think that's even more true for decentralization than it is for encryption.


What's the benefit of decentralization? Not being snarky, I just don't really see it. What does a decentralized PGP email have that I don't have with my Signal Messenger?

Also, given how PGP works I fail to see how you can claim that you can achieve comparable client design/ease of use/UX to Signal.

At the very least it appears evident to me that the problem is much much harder than Signal (and it should be, Signal was designed from the ground up with UX in mind, and made several important trade offs for it).


> What's the benefit of decentralization? Not being snarky, I just don't really see it. What does a decentralized PGP email have that I don't have with my Signal Messenger?

It's a lot harder to block. You can have anyone run a mail server on any port (SSLed if necessary), which means you can use it for secure communications inside any "great firewall" (like that of China or Kazakhstan), or even in a country/region that's been cut off from the Internet (which we've seen happen for various amounts of time in Syria, Crimea, Turkey...). WhatsApp/Signal have a very limited version of this by getting makers of popular HTTPS services to work with them, but those services are subject to their own commercial pressures (all the big names have been known to collaborate with e.g. the Chinese authorities in the past) and attackers always have the option of just blocking those services outright.

It rules out a whole class of attacks involving compromising the central servers and tricking the client into redoing an initial exchange. If the Signal client is implemented correctly this shouldn't be an issue, but it's an extra attack surface that simply doesn't exist for OpenPGP.

Also compared to Signal et al it's more practical (though still difficult) to use pseudonymously, because a pseudonymous email address is eaiser than a pseudonymous phone number. Signal advocates talk about having deniability because your messages aren't provably signed by you, but I don't think that advantage exists in any reasonable threat model - if we're worried about a state adversary hunting down everyone who corresponded with dissident x, they'll find it easier to track down a phone number than an email address, and once they've tracked down one of x's correspondents (i.e. someone who has a phone with a number that exchanged messages with x) they're not going to be bothered by the fact that they don't have signed mathematical proof that this is the same person who corresponded with x.

> Signal was designed from the ground up with UX in mind, and made several important trade offs for it

Such as? At the protocol level I mean.


https://whispersystems.org/blog/the-ecosystem-is-moving/

One of the explicit protocol level trade offs is federation:

> One of the controversial things we did with Signal early on was to build it as an unfederated service. Nothing about any of the protocols we've developed requires centralization; it's entirely possible to build a federated Signal Protocol based messenger, but I no longer believe that it is possible to build a competitive federated messenger at all.


The maintainer of the "Conversations" XMPP client wrote an interesting response to that article:

https://gultsch.de/objection.html


Thanks, I wasn't aware of this. Though to be honest, I am not exactly convinced. Calling HTML federated seems a real stretch. My browser doesn't have to talk to other browsers, just servers, and those can speak many languages. The network effects are really _very_ fundamentally different.

The reality is that, no matter how much I wanted xmpp (and google wave for that matter) to succeed, Signal (and if you are willing to trust the closed source client WhatsApp) are the only ones that did.

And that's even though XMPP preceded Signal by a decade. The argument that a good federated user experience is possible in principle starts to sound a lot like talk about "sufficiently smart compilers".

Let's grant it is true that we don't have sufficient resources to update a host of different clients to use all the extensions. So then it still seems that in a resource constrained environment federation is not feasible. Open Whisper Systems managed to get encryption on a billion devices with a team of three people. The idea that it only succeeded due to a resources advantage (rather than fundamentally different trade-offs) does not seem very plausible.


> And that's even though XMPP preceded Signal by a decade. The argument that a good federated user experience is possible in principle starts to sound a lot like talk about "sufficiently smart compilers".

For how much of that decade did we even see "a couple of full-time developers" applied to XMPP though? Yet alone an actual UX designer.


Have you tried Conversations? You might want to give it a try. On the whole, the user experience is not significantly worse than Signal. With both sides using OMEMO encryption, privacy should be about the same.

Trade offs:

Onboarding is trivially more complex. You have to enter your JID and a password -- registration of a new JID adds one checkbox to that.

Contact discovery does not piggyback on phone numbers, so you will have to add JIDs to your address book if you want Conversations to pick them up.

Another new XMPP-based app, Zom (https://zom.im/) makes some of this easier by letting you automatically register on their hosted XMPP server, locking you into sane default settings, etc. Android app seems still a little buggy, though.


All due respect it not exactly mind-blowing that to compete with some of the most successful businesses in the world you have to do things they can't. The author often likes to use catchy quotes so let's go with the classic "It is difficult to get a man to understand something, when his salary depends on his not understanding it". They make their money selling licenses and consulting for centralized messaging services. It's not in the interest of either party to have disagreements on this issue.


Signal making choices for ease of evangelizing it doesn't condemn the entire idea any more than poor implementations of secure email means we should totally give up on it


Also, is it really true that a state actor could not effectively block email? Or for that matter all encrypted email? They are, after all, blocking web pages. It seems to me (as a lay man observer) that at the state actor level the Internet is relying on centralized resources already, maybe that's why decentralization seems intuitively less important to me. This is not to disagree with your points.


It's more that a private actor can exclude you from any network. Maybe WhatsApp blacklist you for "abuse". Maybe they're right. But even if you kill someone you're allowed to use the telephone network, and send or receive letters.

As long as the message silos aren't regulated as utilities, decentralised systems give us more of the freedoms.

It's pretty easy to run a separate dns system - you can even blend your own private "authorities" dns for new tlds with fall back to the centralised root servers.


That's a fair point actually. Does the fact that signal is open source and based on phone numbers change that though? Seems that as long as you have access to the phone network you have access to the signal network...


No, because the servers are still centralised and they can blacklist you at that level.


They can block any traffic in/out of their network. But e.g. it's possible to use email on a LAN (or a wireless mesh network) entirely disconnected from the Internet.


Hm?

if /BEGIN PGP SIGNED MESSAGE/.test(message) { greatFirewall.block(message); }


Sure, but that only catches messages that cross the wall. They could disconnect China from the Internet entirely, but email would still work within China.


> What's the benefit of decentralization?

No single point at which to apply judicial- and/or rubber-hose cryptography is the big one.

I do agree that the problem-space of attempting to make a reasonable UI for GPG has been explored for a long time with no useful results. I'd love someone to prove me wrong, but it seems like that's a hopeless endeavor.

It is worth asking why, though. I'm not a UI person, so apply appropriate weighting to my opinion. I think this is one area - application of the Unix philosophy to task-driven security software - results in software that only the really invested will use. A combination email encryption/key distribution system with most nonessential options stripped out, targeting one mail client (at least at first) might be simple enough to achieve something closer to Signal-level usability. And the rounding-errors can continue to use the full GPG for our weird open-source rituals.

But who knows. Maybe the entire model is just too complicated for mere mortals.


> the problem-space of attempting to make a reasonable UI for GPG has been explored for a long time with no useful results

Has it? Have we ever had even e.g. 2 reasonably skilled design professionals spend a year trying? That would, I suspect, be much less effort than has gone into Signal et al. But still beyond the resources of volunteer-only FOSS, unfortunately.


I don't think the real problem with OpenPGP is the UI issues. OpenKeychain and K9-Mail together provide a not terrible UI for OpenPGP, and if someone wanted to more tightly integrate them with each other,the UI could probably be improved further.

But PGP-over-SMTP would still leak important metadata, and you would still have problems with forward secrecy and key revocation.

Matrix looks like a much better decentralized solution to build a new email infrastructure on. But there are still metadata leakage issues with federation, and there need to be some standards and an example implementation for email-over-matrix.


> But PGP-over-SMTP would still leak important metadata, and you would still have problems with forward secrecy and key revocation.

I don't think a well-integrated PGP-over-SMTP client would leak any more metadata than the likes of Signal does? Build in a good subkey rotation config and you'd solve most of the forward secrecy issues, and good defaults for how to treat revocation (including better expiry defaults) would resolve that issue. No?


You would still leak unencrypted headers, which in SMTP are numerous and interesting. A client could minimize the useful content of the message headers, but you're always going to have at least the envelope headers available to every intermediate mail host.

I do not know enough to be sure about your point about forward secrecy. You may be right.


Depends a bit what one considers integration:

https://en.m.wikipedia.org/wiki/Anonymous_remailer


> you're always going to have at least the envelope headers available to every intermediate mail host.

Sure, so any intermediate server would see who was talking to who. But that's the case with Signal et al as well isn't it?


”But that's the case with Signal et al as well isn’t it?”

No.

”Because your phone will be connecting to Signal’s servers, your cellular carrier can determine whether or not you are using the service. However, your carrier cannot gather any information about the individuals or groups with whom you are communicating.”

Source: https://github.com/WhisperSystems/Signal-iOS/wiki/FAQ#what-a...


Right, but Signal's servers see who you're communicating with, and they're the equivalent of the intermediate mail servers here.


Currently, yes.


>What's the benefit of decentralization?

Decentralization/Federalization + Standard Protocols. To really get the full potential, one needs both, since only then the network is truly free.

Nobody can tell you how to access the Network, what software to use, who you can communicate with, etc.

If you try using an unofficial WhatsApp client (eg. when you don't have you're phone), they can delete your account, and you can't do anything against that. This is IMO just too much control the central server has over the client.


Harder to prevent access to, harder to wire-tap. As other people pointed out WhatsApp has been blocked in countries before. Not to mention that with WhatsApp having a central location all messages are routed through we really don't have any guarantee that there isn't a compromised actor in there intercepting everything. Even if WhatsApp's crytpo is as flawless in implementation as we'd like FB still has access to all that metadata.


> Even if WhatsApp's crytpo is as flawless in implementation as we'd like FB still has access to all that metadata.

Note that this argument is even more problematic for OpenPGP-encrypted email, as such email sends all metadata and some message data in plaintext.


Note that this argument is even more problematic for OpenPGP-encrypted email, as such email sends all metadata and some message data in plaintext.

I ususally respect tptacek a lot but when it comes to Whatsapp I actively avoid it if at all possible, preferring Telegram even if the crypto is more than questionable. Same goes for mail: I prefer it - even unencrypted - over Whatsapp.

For some of us our treat model is more concerned about Facebook and less about major TLAs.

With Whatsapp I have to expect that all metadata about me - and my friends - are fed into Facebook and datamined from here to eternity and back again or until the end of the world as we know it.

I have little to hide but given the catastrophically bad ad targeting of Facebook (yes, I am still a happily married father, a Java developer, a Norwegian. I don't need ANY more ads for dating websites until I specifically change my profile to let you know, THANK YOU. I would be happy to learn about useful developer tools or underrated fast food restaurants though. I also appreciated the uber ad with bundled coupon in google maps a few weeks ago and tested uber for the first time.)

Given the same catastrophically bad targeting and given that it is the same owner who as far as I know hasn't yet apologised for his remarks about how trusting him was stupid it wouldn't surprise me if is more a WHEN than a IF that Facebook is going to sell everyones data to insurance companies, support scam call centers etc.


I agree with not using WhatsApp due to it's ties to FB and metadata issues, but Telegram is arguably even worse. They've been (rightfully) panned for implementing their own crypto and doing it poorly. You should be using Signal on a phone if you're trying to use a secure messenger.


You should be using Signal on a phone if you're trying to use a secure messenger.

I am not trying to use a secure messenger. I am trying to use a good one without ratting on my friends to the worst (as in size x badness) company I am aware of. Ohh, and I don't want to be be part of their network effect either.)


All reasonable views, but I don't think that's a Telegram given their track record. At this point I'd put more trust in a transport-encrypted-without-end-to-end messenger than Telegram, and there are plenty of good options in that space (e.g. Discord, or hell, AIM). And if you want genuine security there are good decentralised options: XMPP (Conversations et al), Riot/Matrix, possibly Wire.


I'm not advocating for WhatsApp or PGP encrypted email, I'm pointing out that people who make the line in the sand at "decentralized vs centralized" are boiling the problem too far down.


Decentralization is good to help protect the user against abuses from the central service provider.

It also encourages competition, since the user can switch to a different service, and not be penalized by network effects preventing them from communicating with other people.

I use MX records in my domain so that I can switch to any mail provider I want, and people will still be able to contact me with the same address.

How is this possible in a centralized system? If I switch from WhatsApp to Signal, I lose all my contacts. If I want to keep communicating with them, then I have to convince them to switch to a different app, or just never stop using whatsapp. The network effects dynamic here makes it very difficult to switch from one centralized encrypted chat provider to another. If you try to leave, you lose all your contacts.

The other thing I dislike is that they all seem to require a valid phone number. This puts you at the mercy of the phone company. If you change your phone number, then you have to get all your contacts to change their contact info for you. This is a huge step back, compared to email with own domain and MX records!

The other thing I dislike about these centralized encrypted chat providers is the lack of client choice. 1 company can only support so many platforms, fair enough. But that will likely mean I'll never see the company develop a Linux desktop client, or a terminal based client for their network. And because of centralization, no-one else will be allowed to develop one either. In contrast, there are many different tools I can use for sending and receiving email on the Linux desktop, cli based backup tools, etc.

And as far as the spam problem goes, I'm not sure how Signal/WhatsApp are better in this regard? If you have to give your signal address to out for people to communicate with you, or to do business with a company, then I don't see why companies can't just spam you. Signal can centrally filter all messages you receive to make sure they don't have spam in them and block spammers messages from going through (but this is no different than what many mail providers also do). You can block individual contacts, but that doesn't help if there are a very large number of different accounts sending spam. You can whitelist your contacts, but then noone you don't know that wants to talk to you can contact you.

It also doesn't stop known contacts from sending spam to you either because

- They are forwarding spam messages to you, like chain letters.

- They got a virus or some malware which sends spam to all their contacts

- It's a "legit" company you need to receive communication with, but sends spam mixed with vital communication: marketing lists you get auto-opted into (imagine if all the "give us your email to read the article" pop overs got replaced with "add us on signal to read the article"), amazon sale ads, facebook constantly trying to entice you to go back on the site, etc.

If signal replaced email, I don't see how it could remain spam free.


We give up decentralization for diversity and redundancy. We don't limit ourselves to one platform for communication. Decentralization worries me as much if not more than centralized systems of communication. Suddenly there are n number of actors I need to worry about vs one.


There are also people that use OpenOffice on Desktop Linux, and believe in their bones that everyone else should too.


They're wrong, obviously. LibreOffice is much better.


I sometimes preferred it even on Windows and even when I have a valid office subscription.

Some people really manage to mess up styles and Open or LibreOffice used to make it much easier to clean up those docs. (I'd switch back to get page numbers right though :-/)


I've also preferred it on Windows, having a valid Office subscription, because I've needed to open obscure/obsolete file formats that Word/Excel don't support.


> My conclusion is that the people who care about "decentralized" systems are a rounding error. I care about non-technologists managing to send asynchronous messages to each other that are well-encrypted by default. That's a solved problem.

You don't understand the problem: if you rely on a service like this, two things can - and by the laws of probability, will - happen:

a, a centralized service goes down and suddenly no one can talk to nobody b, you're unable to send a message to someone and never realize it

Imagine if there was _one_ ISP. One single mobile provider. If it goes down, it goes down everywhere, for everyone. Not fun.

As for b, I can't remember, but there's a term when a provider let's you post but it won't get show to others. They can also alter messages - see the Whatapp signal implementation issues that certificates can be silently re-issued.

You are, unfortunately right though: very few of us realizes how important this is. Thankfully the smarter elders of the internet seem to do. https://www.decentralizedweb.net/


No, I fully understand the problem. If Google Mail vanished tomorrow, a pretty large number of people would probably stop emailing altogether. The number of people for whom that's true increases every year.

If you find that unthinkable, consider the bubble you might be living in. I appreciate that there are people that require a decentralized service for messaging and understand where they're coming from. I don't deny their existence. I simply think their numbers are much smaller than they appear on nerd message boards.


> No, I fully understand the problem. If Google Mail vanished tomorrow, a pretty large number of people would probably stop emailing altogether. The number of people for whom that's true increases every year.

I highly doubt that's true. Email is pretty essential to the functionality of the internet, from signing up accounts to getting notifications, to just plain discussions with professionals. It's pretty much the only thing that does what it does.

To use non-nerd examples people in my life have done recently via email: contacting the school registrar's office, updating insurance information, discussing minor problems with a recent surgery with their doctor. Especially for people with anxiety issues who have problems on the phone, email is a life saver.

I lived off email when buying a house through a builder last year. Everyone went through email and nothing else was even offered in many cases. The builder, bankers, lawyers, electricians, everything was email.


Wechat is what's doing this is China, and it's working fairly well for them. It's obviously impossible to do the same in the West (companies won't be trusted by people in Europe, nation-level apps won't be trusted in US) but it's not impossible to replace email. Note: mobile is gigantic compared to desktop in China, so this might also be a reason.

I still believe email will outlast all the current solutions though, but the crushing presence of a few big providers is not doing any good for the protocol.


While WeChat is somewhat more advanced then its rivals, the reason it works is because Chinese businesses are less 'sophisticated' than western ones and much more human based. WeChat is to a large extent a phone call replacement, which is especially useful in a country of multiple languages and dialects but a common written one. Western messaging services are mainly replacing things like text messages and other instant messengers not e-mail. (e-mail is probably still the de facto most insecure protocol on the internet and should be replaced).


Electronic conversations didn't even replace paper mail.


Replace? That's a strong word but it has more or less deprecated paper mail. Everything from insurance cards to my recent W2s are delivered electronically via e-mail now. I recently bought a car and all the paperwork was completed online. The bank uses electronic signatures for everything. The amount of first class mail delivered by USPS has halved over the last decade. Is paper mail dead? No. Is it on it's deathbed? Probably.


Out of curiosity, I plotted first class mail delivery vs. population:

https://i.imgur.com/Fp2LLCg.png

Annual per-capita mail delivery is down 50% in the US since Y2K.


So 20 years of internet, the rise of mobile and hi-speed connectivity, the multiplication of communications means including emails, chats, text and social network and the paper mail is still here.

I still receive all my most important communications through the mail box, including anything related to administration, voting, my landlord, invitations to major life events, bank details, etc.

Now if you hope to kill email, you gonna have to remember that.


Well, I think some of this can attributed to personal preference. Each of the use cases you described can be accomplished via e-mail or other electronic communications. Other than a wedding invite or two every year, I receive no other personal mail. This didn't require any special technical skills, many organizations actually encourage you to setup paperless accounting when you sign up for a new account.


Three responses:

* Email remains important for middle-class Americans because it's used for business. But that is a small subset of the whole population, including very large numbers of Americans.

* For almost all those users, email might as well be a Google, Yahoo, or Microsoft product.

* Every year, the number of people and businesses that rely on email gets smaller --- in the last 5 years or so, by something like 15%.

There's no trend we're looking at that suggests that email is becoming more relevant.

We should be happy about this, not freaking out. Even if you think the future is decentralized messaging protocols (I like decentralized too; I just think we should figure the nuts and bolts out before we decentralize), email is a boat anchor holding us back.


> * Every year, the number of people and businesses that rely on email gets smaller --- in the last 5 years or so, by something like 15%.

If that's true then where's that stat from and how are these businesses getting contacted online?

There's no decent replacement for email in that department at all to my knowledge.


Can't answer the source of the stat, but where they're getting contacted - FB messenger or Twitter, for starters.

I've done most of my non-B2B communication with businesses over the last year or so via FB messenger, Twitter or phone.


>I just think we should figure the nuts and bolts out before we decentralize

Bolting on decentralization works about as well as bolting on security.


Can you qualify that third point?

What's considered reliance? What's the name of some place that stopped using email? I can see a drop in businesses that are running their own email, and that outsourcing maybe represents a big shift in terms of how they see it as "not critical and must be kept in house" but I don't know that they rely upon it much less.

How are these businesses messaging and communicating?


There's a big dilemma with using centralized systems for sensitive communication.

* You have to use one that is not economically or legally dependent on a jurisdiction hostile to you.

* There can never be many different centralized messaging systems that are economically viable because that requires network effects.

As a result, there will always be a large number of people who will not be able to find a centralized system that protects them reliably.


> * Email remains important for middle-class Americans because it's used for business. But that is a small subset of the whole population, including very large numbers of Americans.

Middle class Westerners (not only Americans by any mean) that are doing business are also pretty much the only ones that are willing to spend money on written communication. E-mail might not be on the rise, but I'd guess it grosses way more money than all IM platforms together.


re @tptacek

> Every year, the number of people and businesses that rely on email gets smaller --- in the last 5 years or so, by something like 15%.

Are you sure that's not just the spam decreasing?


Yes.


A bold statement like that without a source just screams bullshit.


I think tptacek thinks in a way where he's giving you charity in an uneven discussion, so he's not obligated to go further; perhaps harsh, but his reputation warrants a little pause before simply saying "bullshit".


His reputation is specifically why I asked for a source. While I don't doubt he's an intelligent individual, I've noticed that he also likes to "shoot from the hip" without providing evidence of many claims he makes on both here and Twitter.


No, his reputation doesn't "warrant" anything. Internet discussions don't work this way.

If he claims something, he should be able to back it up with something else than his so-called "authority".


Do you really understand the problem as you claim? The amount of people who require a decentralized system is irrelevant to the issue. The issue is in what users will do when their trusted system goes down, be it centralized or decentralized.

Say Gmail goes down (forever), will people use another of their emails? Or register for a new one somewhere else? Or start using another medium of communication alltogether? I assume all of the above will occur to some extent, but if that system was centralized, only the third option would be available. And if that centralized system was so good that it had killed all competition, then suddenly no one would be able to communicate.

It's not about the amount of people who scream for decentralized solutions. It's about the alternatives available in case something does go down, and how easy those alternatives are for users to adopt.


If you're actually in a "great firewall" type situation, though, what are the chances that an average user is going to correctly run your own server securely? Configuring an email server securely is even more complicated than setting up PGP.


How many times has WhatsApp been blocked in Brazil? What's your advice for those 100 million users when it stops working? Switch to another centralized messenger, and convince their friends and family members around the world to do the same so they can continue communicating?

What would happen if e-mail didn't exist, and a site like HN listed their WhatsApp contact information instead of an e-mail address? Suddenly, the Brazilian user base here would be unable to contact HN. They might be completely out of luck, or if the user base was large enough, HN could add an alternative contact method for users in restricted regions. "Contact us on WhatsApp. If you're in Brazil, you'll need to contact us on ABC messenger instead. If you're in China, and unable to access both, you'll need to send us a message on XYZ messenger."


Isn't it also true that the people who care about "secure" systems are a rounding error?

Average Joe is perfectly fine with just "100% secure" label. Add some "military grade hurr durr" nonsense (okay, maybe it's a bit outdated buzzword) and Joe's even willing to pay for it. No need for any actual security.


Again: the Venn diagram between people who want encryption and people who need encryption has very little overlap. And, thankfully, modern secure messaging systems work for both populations.


Which also applies to the distributed systems - email, in context of this particular discussion. Same logic here: email works for both.

And doesn't apply to IM systems, because it's just not possible for Whatsapp user to contact Signal user and invite Wire user in a group. IM app fatigue is a real problem. Or maybe it's just that nearly everyone in my bubble has load of apps just to contact all their peers.


> Which also applies to the distributed systems - email, in context of this particular discussion. Same logic here: email works for both.

Email yes, encrypted email, no. That's the whole point. A decentralized system, as nice as it is in theory requires participation to enable widespread change. You cannot just say "now we all encrypt email" and know that everyone does. But you can change the transport mechanism of a centralized system to something encrypted and know that it works for all participants, even for those that don't care or don't know how to do it themselves.

It's a trade-off. Again.


> There are big places where its market share exceeds that of SMS

The "market share" of SMS is the entire world. SMS works with anything from a $15 Shenzen dumb-phone to a $1k iPhone 7+. Not to mention that unlike WhatsApp, which is forbidden to do business with countries like Iran (due to US export regulations), SMSing works with any country in the world except North Korea.


There are in fact a billion users of a closed-source product owned by a surveillance company with history of subversive activity. They chose it over free or low-cost IM with privacy focus that were usable. They're opting against private messaging for various reasons.

Not an argument for or against any privacy solution if you're citing people who don't care or make sacrifices. They're not the demand side for private messaging. The solution for such people is security professionals getting employed with or consulting those groups to embed strong security into their products with continuous monitoring for subversion at source or repo level. They mostly arent doing that.

So, the WhatsApp point is broken twice: its users dont care enough to use something better; best security people talking WhatsApp security arent there improving or maintaining it.


How does WhatsApp compare to email? I mean, how easy is it for me to contact MS CEO via email vs WhatsApp? Or my doctor. Or my mom. Or my friend I've not seen in a while. Or????


It's not that easy to compare the two. With WhatsApp, if you know the phone number of the person you want to contact, then you can send them a message or call them via WhatsApp. Both users need to have WhatsApp installed, but this is not a big problem in most countries outside of the US, since the install base is over 1 billion.


With email, you can usually do a simple search and find an email. Super simple.


And we go to great lengths trying to avoid this because of email harvesters for spam.

However I would not like it to transform into phone number harvesters for spam.


SMS is federated, like everything else telecom. Just not as openly as SMTP.


I feel like Matrix may be the way forward; it also looks like it's possible to build an email-like interface on top of it. But currently, federated Matrix leaks metadata, so that's a problem.


Agree about Matrix. You get addresses such as @username:example.com , which work somewhat like email addresses. The example.com part is the homeserver, analogous to gmail.com or yahoo.com in emails. Users can communicate across homeservers.

It's also fast to setup. Took me about 30 minutes to set up a homeserver and host a customized riot client to use it.

It's not completely decentralized yet, and you can only use from a fixed list of identity servers. Although you can set up your own synapse node, you can't yet use it along with the centralized identity servers.

The reason they give for not decentralizing the identity servers yet is to avoid spam. But they plan to make them decentralized in the future, so at that point it would completely be like email.


Just to clarify: identity servers are strictly optional and are used just for mapping 3rd party IDs (3PIDs) such as phone numbers and email addresses to matrix IDs so you can be discovered or discover other users by 3PID.

They are only centralised atm because we haven't really started to attack the problem of decentralising them. What we really need is a decentralised equivalent of Keybase, but nobody has really built a suitable one yet. Blockstack is close, as are a few others, but we'd rather wait until we found something which nailed the problem and stuck with the jury-rig until then. I spoke about this at FOSDEM the other weekend: https://fosdem.org/2017/schedule/event/matrix_future/

There's also renewed community interest in fixing this over the last few days: #matrix-identity:matrix.org is a thing.

I would argue that Matrix itself is fully decentralised; it's just that some of the solutions to the optional identity discovery problem happen to be centralised. Just as some folks publish their Matrix E2E pubkeys on keybase etc.

In terms of Matrix revealing metadata: yes, it does so to the servers participating in the convo. This is really not very unusual; the problem of protecting metadata for realtime comms is very much still an open area of research (see Alpenhorn etc from CSAIL)! Hopefully it's not really a reason to turn anyone off Matrix though. I also briefly spoke about this at FOSDEM: https://fosdem.org/2017/schedule/event/encrypting_matrix/


Re: metadata: I really like Riot/Matrix and have convinced a bunch of formerly non-IRC people to start using IRC through Matrix (the great free IRC bouncer in the sky!). However, a couple of things that bother me:

- The fact that it appears to be impossible to disable typing notifications ("X is typing...") and read receipts. This can change the nature of a conversation and really should be optional. Coming from IRC, it just feels plain creepy. A friend flat-out refused to use Riot because of this.

- The fact that device information is leaked to everyone who happens to be in the same channel. So I can join #matrix and click on any of the >5k users and see information about all the devices a particular user has used. Here's from one random person: "https://riot.im/app/ via Chrome on Mac OS", "https://riot.im/app/ via Chrome on Windows", E5823 (ah, J. Doe is using a Sony Xperia Z5 Compact!), etc. Sometimes things like "Joe's iPhone" is exposed -- and I don't think Joe had any idea of that. This is bad.


These are https://github.com/vector-im/riot-web/issue/3220 and https://github.com/vector-im/riot-web/issue/2295 respectively. Agreed that the info currently leaked in device info sucks - please vote on the issues to get them bumped up the todo list!



doh, thanks


Forgive my ignorance, but what caused XMPP to fail? Simply the lack of uptake or is there some other reason?


Google's embrace-extend-extinguish destroyed XMPP. They made their chat system XMPP compatible for a short time which caused many people to swap to their solution. When they ended support, most users simply stopped using XMPP.


Did you also notice that Google's messaging system pretty much died out around that point in favor of Skype and similar?

I used to see people mention GTalk all the time, but I haven't seen anything similar in years. No one has mentioned G+ or Allo. That decision by Google may have been the thing that killed its user base.


Hangouts (which I think GTalk turned into) still has some usage - certainly as a quick video conf call solution, and somewhat for text messaging. It definitely doesn't seem to have much of a future though since Google released a bunch of other competing chat applications.


It's unfortunate because Hangouts is still really awesome. It works really well for video chat (including group chat), but more importantly it works great with multiple devices. I can get a message on my phone, laptop or PC, and reply anywhere, with the conversation and notifications synchronized. I nearly always have my phone with me, but it's really nice to be able to just send someone a link from my PC, for example.

It looks like Telegram has multi-device support, but then this kind of thing scares me:

> Q: How are you going to make money out of this?

> We believe in fast and secure messaging that is also 100% free.

> Pavel Durov, who shares our vision, supplied Telegram with a generous donation, so we have quite enough money for the time being. If Telegram runs out, we will introduce non-essential paid options to support the infrastructure and finance developer salaries. But making profits will never be a goal for Telegram.

That's not very compelling in terms of investing my effort to switch.


No, it turned into Hangouts, which is alive and kicking, specially for companies using Google Apps.


But really only internally, at least in my experience


The rumor[0] was that Google killed XMPP because Microsoft was doing 1-way federation (Microsoft would not broadcast availability status or typing indicators).

[0] http://www.ucstrategies.com/unified-communications-strategie...


Even worse, Google never federated with encryption, then Hangouts killed group chats which were XMPP compatible. A few friends and I resisted with several XMPP accounts until they moved to Hangouts. Not to mention all the JEPs that solved the same problems as Hangouts that Google plain ignored for years.


I've been running my email server for a decade without serious glitches. Setting up my own federated XMPP instance is much more problematic and people are already complaining about how hard email is.

I'd love to see an up-to-date tutorial that opposes my statement, eg. setting up prosody (or something lightweight) on debian (or similar) with multiple domains for multiple accounts, sending and receiving test messages from another XMPP hub, so if you know one, please link it.


> Setting up my own federated XMPP instance is much more problematic

Actually, I believe it's the contrary.

With email you have to obtain a valid TLS certificate, set up SPF, DKIM, and keep eye on the DNSRBLs so you're in good standing. And if Big Company's mail service suddenly decides they don't like you, it won't help. Oh, and spamassasin/rspamd/milter/greylisting/etc stuff so your email server doesn't get thousands of emails per day few weeks after some bot detects your host is listening on tcp/25.

XMPP has less adoption, spam exists but is much rare, so there's less stuff to do. Install a package, get a TLS cert, publish an SRV record (IIRC that's not even strictly required), and that's it. A bit fewer steps.

Not to say email is MTA+MDA combo (+some fancier LDA, +Sieve if you want more that a simple system) while XMPP is a single piece of software (unless you want transports or external extra services).


> XMPP has less adoption, spam exists but is much rare, so there's less stuff to do. Install a package, get a TLS cert, publish an SRV record (IIRC that's not even strictly required), and that's it. A bit fewer steps.

That's what I thought as well. Please link tutorial on this.


Personally, I'm using ejabberd, but I wouldn't recommend it. Don't have any good tutorial links at hand, sorry.

I'd agree with /u/problems suggestion to try Prosody. http://prosody.im/doc/install + http://prosody.im/doc/configure + http://prosody.im/doc/dns#srv_records + http://prosody.im/doc/certificates are probably all you have to do to get it up and running.


Just install prosody and open up the config file. It includes lots of comments and I believe there's even a web admin interface you can enable.

XMPP SRV record documentation can be found here, if you need to use it:

https://wiki.xmpp.org/web/SRV_Records

If you're finding the documentation to be insufficient, let me know, I might write something more detailed up.


This one's a bit minimal since it's designed to fit on a flyer, but it does contain what you need to know; install server of choice, get a TLS cert from somewhere, set DNS records if you want them, done. https://xmpp.org/images/promo/xmpp_server_guide_2017.pdf



Google, Apple, Facebook, WhatsApp.

Everyone[0] is using one of these.

None are XMPP or compatible with XMPP.

[0] I don't technically mean the entire planet.


Curious why you tossed Apple in there. In worldwide usage it's a fraction compared to the other three. (I feel compelled to state I use an iPhone in order to avoid this comment being interpreted incorrectly).


> XMPP tried to address this and failed

XMPP aint't dead yet ;)


> No, it's not. It's the only widely available, decentralized system, with which you can send to anyone, if you know the address. None of the big ones is this open.

SS7 is internet connected and federated and it's arguably as big as email.

> XMPP tried to address this and failed; now Matrix is trying again.

Oh, agreed on XMPP. Encryption was a very much after the fact addition to it though.

I haven't heard about Matrix and plugging in 'matrix encrypted email' into search gets 0 relevant results on DDG and is bottom of the page for Google. Maybe the ranking sucks cause the page takes 20s to load the JS shit and it has zero content aside from a bouncing fucking shitball above the fold.


http://matrix.org/ is a promising decentralized communications protocol.

The popular https://riot.im is built on top.


> The popular https://riot.im is built on top.

Thanks, so it's basically a opensource and federated slack?

How does this solve the email encryption use case? (longer form text not requiring presence)


As is, it doesn't. It should be possible to build an app with the email nature on top of the Matrix protocols though, and I'm led to understand there are people working on it.


What is SS7?



I was afraid you were going to say that


> No, it's not. It's the only widely available, decentralized system, with which you can send to anyone, if you know the address.

They have to be email users to have an address. Granted, it's so widespread that even signing up for one of the other messengers often requires an email address, so even the non-email users have email addresses.

But I still think email is going to go the way of usenet, non-internet phone calls and the dodo. Perhaps it's too hard fixing email, compared to just moving to something else. Email is obviously hard to solve since it hasn't been solved already.


I'm more interested in seeing something that maintains the desirable properties of email while dropping backward compatibility with SMTP. But the 'encrypted messenger' community isn't interested in working on that because they think (apparently?) that email does not have any desirable properties compared to siloed IM systems.


There is a lower barrier to entry for a new user to download a messaging app vs installing, configuring, and understanding PGP/GPG.

This is a real problem that hasn't been solved, and because it hasn't the world has moved on.


As a happy user of matrix (via riot.im), I sure hope matrix takes off!


By that logic shouldn't we try to encrypt SMS? Email as a protocol has huge security issues, and email as it is currently used honestly is NOT federated or decentralized.


For personal use email is rarely self-hosted, but corporations, governments, and organisations often run their own email infrastructure. That effectively makes email a decentralized federated system.

You might argue how a lot of email is either send to or send from Google, Apple, or Microsoft services, and that is thus somewhat centralized, but isn't that stretching the definition?


Is talking about government organizations hosting their own email servers really a point in email's favor in 2017?


Why would it be a point against it? I generally think my government being in control of its communication infrastructure is a good thing.


It means email federation is alive and well.


> why bother? Email is just one of dozens of messaging systems available to Internet users. Better to move sensitive conversations to things like Signal, WhatsApp, or Wire --- the double ratchet construction is designed specifically to make IM-like protocols secure even when conversations are sporadic and last months.

This makes me sad. There's something profoundly democratising about the idea of running your own email or XMPP server (or any kind of server, for that matter). It's something you're in control of, that's configured the way you want it. Using someone else's server for everything brings with it a kind of tyranny (even if the people in charge promise not to use it for anything tyrannical (like that's ever happened)). If the response to that is "yes but it's hard/impossible to build a secure, distributed messaging system" then I think that's overly cynical.


It is very hard to build a secure, distributed messaging system. It's even harder to run a secure one in a world where bad actors are known to exist in distributed, federated systems.

I understand why some might feel this is an overly cynical position to take. With that in mind, I would remind people that spam is a real problem in a real-life distributed, federated, open system.


This is all true, but IM is not email.

What I want out of email that IM does not (generally) deliver: 1) UI optimized for longer form messages. 2) Decentralized system where you can send to anyone if you know their address, which may be long-lasting and not tied to the fortunes of a single private company.

I don't demand compatibility or interoperability with SMTP, but I also don't want to throw the baby out with the bathwater. It looks like an an email application layer is buildable on top of Matrix, so that's something I'm following and looking forward to.


The modern messaging services agree with you. Nobody has completely nailed the UX for long-term long-form conversations, but services like Signal are designed with those kinds of conversations in mind.

At the same time: if you had to compare the UX of running a long-term secret conversation over Signal versus the UX of trying to reliably encrypt messages over email, no normal user would ever choose the latter.

Secure messenger UX will get better long before email will be reliably encrypted (that's cheating, because email will probably never be reliably encrypted, but even if it wasn't, messenger protocols are going to win this race).


> Nobody has completely nailed the UX for long-term long-form conversations, but services like Signal are designed with those kinds of conversations in mind.

No, they're actually not designed with any kind of "long termness" in mind! Case in point - one cannot move to a new device and still have access to conversations that happened on the older device with Signal. All old conversations just have to die with the old device, and the new device starts as if Signal was installed for the very first time - including starting conversations with people, getting into groups, etc. Backing up the old device and restoring that into the new one wouldn't help for this. There is no message migration feature available (separately) when one changes devices, and what's worse, in my limited observation, the Signal team just closes feature requests and issues filed on this front on Github. The ugliest part is that this is not even explained when someone installs the app or activates Signal on a new device.

So Signal is good only for ephemeral messaging where one doesn't care about past messages and their availability. Wire has the exact same problem too. One might as well treat apps like Signal and Wire like voice call apps rather than as text/photo messaging apps.

It seems like these apps make some naïve assumptions about the target user either not changing devices for many years (self-defeating in a way with smartphones, where security updates stop after 4-5 years, maximum) and/or not caring about having access to older messages whenever the users gets a new device.


Also, no way to have multiple separate conversations with the same person, or threaded conversations.

These may just be app problems -- there may be protocol support for future applications to provide these (this is where Matrix stands today). But the lack of interest in these things doesn't look good.


tbh there's significant interest in decentralised comms, at least from the geek community - the matrix.org server alone is pushing out around 1500 msg/s atm and we're having to panic around finding faster hosting hardware as a quick win whilst working on longer term homeserver scalability work. obviously this needs to transplant into mainstream interest, but we're hoping that clients like Riot will get the UX sufficiently good to bring decentralisation to the mainstream folks otherwise stuck with WhatsApp and friends.


I got a group of about 30 non-technies using Signal, not because they knew what what perfect forward secrecy is, but because they grew vaguely suspicious of the motivations of governments and Facebook, and people/Google-results they trusted were positive.

The details don't matter to ordinary people, but our approval matters much more than you think. I have very high hopes for Matrix.org.


I think your group of people would be highly disappointed whenever each person moves to a new device and finds that they have essentially "lost" all old messages and have to re-join groups afresh.

> The details don't matter to ordinary people, but our approval matters much more than you think. I have very high hopes for Matrix.org.

I agree on this part completely, and I try to introduce better tools to others too. But not having messages restored on a new device (even when a user backs up and restores all data) and not providing a way for that when changing devices is unacceptable for me. At this point I'm back to Telegram, which is a compromise, but provides a much better experience as far as usability and meeting the expectations of non-tech users go.


matrix e2e is still beta, and we just haven't implemented e2e session sharing yet. you can now export/import keys when migrating between devices as a workaround but proper sharing is coming soon.


I had a brief look at Matrix recently, and I'm really looking forward to it becoming mature. A decentralized solution with emphasis on security and privacy is what I've been praying for not just for myself, but for the sake of all humans!


You're right, losing messages is a bad experience for most people.


> at best you're attempting to tunnel encrypted messaging over an unencrypted transport.

That covers pretty much all communication encryption: ultimately, encrypted data goes out over an unencrypted link.

> A protocol that leaks metadata, including some message content, at the envelope layer.

That is indeed the major problem I have with it.

> Hundreds of millions of users that primarily access messages through browser clients that can't meaningfully implement crypto.

The solution IMNSHO is to get off of the browser. Browsers are great for what they were designed for, but they're a terrible general-purpose computing platform.

> End user demands for things like search that can only be delivered efficiently at scale by databases of plaintext (most likely at centralized servers).

I think that one's home computers can probably handle the load of searching all of one's own data. If that's not the case, encrypted search protocols may be of use.

In general, though, your last point argues against any encryption or other user-privacy measure: if users are right when they demand centralised search, then encryption is a bad thing. I don't agree that encryption is a bad thing, thus I refuse to believe that users are right to demand centralised search.

> But: why bother? Email is just one of dozens of messaging systems available to Internet users.

It's the only decentralised one atop which a trustworthy system could be built. Signal, WhatsApp & Wire all have critical flaws which prevent their widespread use (Signal is centralised, tied to a phone number, leaks contact information to OWS; WhatsApp is centralised, owned by Facebook & default insecure; Wire looks appealing, but it too is centralised and controlled by a single company).

I tend to agree with your conclusion that email encryption is not worth the trouble; I disagree that it cannot be used securely.


>I think that one's home computers can probably handle the load of searching all of one's own data.

You seem to be implying that I have one computer at home. I mostly use the computer in my pocket, but sometimes I use "my" computer in "My office" at home. Sometimes I use "my wife's computer". Many people I know have a computer attached to their tv. I'm looking into putting a computer in my garage. Someday I'm likely to get a laptop or tablet computer for travel (I've had these in the past).

We need to get off the mindset of one computer per person - it was never really true, but for the average person today it is less true.


Ideally we might have one server per person. There's no reason, really, why I shouldn't be able to run my webmail, media store, calendar, lightly-trafficked blog, etc. from an RPi sitting at home and just use them wherever I like.

I used to do this with Owncloud, and Sandstorm has done some cool work in this field, though ISP's of course make it challenging. Also, it does raise the risk of everyone contributing to massive botnets.


Related : Cloudron.io has made some huge progress in making it trivial to set up a mail server.


You're disagreeing with an argument I didn't make. "Cannot" isn't the word I would choose.


> * Hundreds of millions of users that primarily access messages through browser clients that can't meaningfully implement crypto.

Isn't protonmail pretty much a direct counter-example of this?

They even have an extension you can use if you don't want their server to be trusted at all: https://chrome.google.com/webstore/detail/protonmail-checker...

> * End user demands for things like search that can only be delivered efficiently at scale by databases of plaintext (most likely at centralized servers).

Thunderbird and Outlook at least seem to do search quite fast without any such thing. This could probably be done on something like protonmail by using an encrypted index too.


No? Virtually nobody uses Protonmail. If the only thing between encrypted email and no encrypted email was a single provider that implemented PGP, we'd have had universal encrypted email in 1999.

(I'm stipulating that anything Protonmail does actually, you know, works. I have no idea if it does. Why bother? Encrypted email isn't going anywhere. The track record on things like this is quite bad.)


In 99, webmail providers were nearly all using squirrelmail or some self-invented thing and people were using offline mail clients, where GPG/PGG could have been an option. The terminals were behind, cloud computing ahead, and everyone knew your stuff is gone if your are careless and reinstall your computer, since POP3 was still a common thing. (Unlike IMAP, POP is usually used in a mode where no copies are left on the server, so the mails only exist where you pulled them down to.)

The big difference, however, that the world wasn't yet spying on every drop of text you made, and that is why email encryption didn't take off: there was no real need.

Unfortunately, the cloud and web "revolution" happened before people realized they should encrypt _before_ moving back to centralized services and now it seems to be a lost cause.


No. I don't know if it's a generational thing, but people on message boards today seem pretty convinced that Snowden invented concern over dragnet encryption. Get a copy of Applied Cryptography and thumb through it; it's shot through with the mindset that NSA (specifically: NSA) is reading all your mail. The 1990s were the decade of the cipherpunks, the Clipper Chip, the crypto wars, Echelon, and the hacker crackdown. More people encrypted their email then than do now.


NSA, of course.

But NSA - I really hope - doesn't sell your data for profit to insurance and medical companies, so there's a difference. If NSA picks up something serious about you, you're in deep trouble, but the current data sniffer companies can make your life really hard without you even realizing it.


I'm not advocating plaintext email. I'm advocating for no email. But since it'll be 10-15 years before email takes its place alongside ICQ among protocols normal people never use, in the meantime, the right solution is to keep email banal, and keep sensitive stuff on secure messaging protocols.


So what, it's all or nothing for you? Even if there is an easy to use solution that you can set someone up with in about 10 minutes, that's absolutely valueless because it means that less than 100% of messages will use it?


What do you want to hear from me? Part of this is my own bias against email, but that's not all it is: I'm not making up the "emerging consensus" bit.


I'm a little concerned that the emerging consensus in question may be led by you, as you are a famous information security expert and people take your concerns and views very seriously. When you mention your view about this here or on Twitter, lots of people quarrel with you about it (some of them engaging with it seriously). A number of people have been persuaded by your arguments but a number of people keep strongly disagreeing.

But it's possible that more people agree with you and fewer disagree in contexts where people have more information security expertise or where more of them work on it professionally.


There is a presentation[1] by grugq on how to communicate securely. Using gpg with email is hard. Anecdotally, people reply in cleartext quoting the encrypted message. If you can train and test your co-conspirators before sending them incriminating evidence about yourself, maybe it can be done. But mistakes happen so often, people who have secrets that can get them killed or jailed for a long time just switched to Signal. That's what tptacek means.

1 - https://grugq.github.io/presentations/COMSEC%20beyond%20encr...


> Virtually nobody uses Protonmail.

Please, cite a source. Or, show us with data.


This sounds like a "drop your pants and surrender already" post.

You could add at least some constructive advice or strategy. And using...

a) closed platforms (silos) that

b) run on closed-source code that's

c) produced by companies that are PRISM partners

...is not in the interest of anybody who values real privacy and requires a solid basis for freedom of speech and democracy.


Matrix will likely be on his list of suggestions soon, once end-to-end encryption support is out of beta and/or more mature. That is an open system.


There is still very much a set of users in the incident response community that relies on PGP. These are professional teams that need to communicate with each other. They talk about upcoming disclosures, current abuse, upcoming operations or patches, et cetera.

This stuff is almost all short to mid-term secret. Most of this will become public in a month or so. Leaking meta-data is an assumed risk (or too much hassle to avoid, take your pick).


I use PGP pretty regularly, too. But what does that have to do with the comment I wrote?


Encrypted email for this user group is certainly not over, and there is still no realistic alternative for it. So PGP is not going to go away completely.


Until (I assume?) not that long ago, people were also still using icb to work on OpenBSD together. That doesn't mean it's important for us to discuss future directions in icb.


True, but let us acknowledge that those are edge cases. For the average person encrypted email has never been a thing.

I hope PGP remains around for people who do need it. There are people who have a real need for secure communication, and they are probably able to invent a cover story to explain your meta-data leaks. (everyone knows I buy widgets and from him: it is no surprise that we don't want details about our negotiations public)


> Better to move sensitive conversations to things like Signal, WhatsApp, or Wire

Really? Come on!

WhatsApp is owned by a company whose business is done by retrieving all the information it can from users and by tracking their behaviors.


Funny how it wasn't until after they were acquired by that company that they began the project to adopt the protocol that would make reading their messages cryptographically hard.

It's almost as if you can't start from some tiny set of first principles about the world and use it to reason through any problem in a message board comment.


> Funny how it wasn't until after they were acquired by that company that they began the project to adopt the protocol that would make reading their messages cryptographically hard.

Perhaps you don't remember this:

https://www.theguardian.com/technology/2016/oct/28/whatsapp-...

https://www.theguardian.com/technology/2016/nov/17/facebook-...

Can't parse your second sentence, sorry.


We never trusted Microsoft with their practice of Embrace-Extend-Extinguish. Why should we trust Facebook here?

If Facebook were collaborating with the NSA I see no reason they'd operate differently than we see from the outside now.


WhatsApp doesn't just support¹ Signal's end-to-end encryption, it enables it by default for everyone with no option to turn it off. It successfully prevented my government² from decrypting user messages even with court orders.

It's a major improvement over whatever bolted-on security email has.

¹ https://whispersystems.org/blog/whatsapp-complete/

² https://www.forbes.com/sites/parmyolson/2016/05/03/whatsapp-...


Despite "the emerging consensus among experts", I'm still happy using OpenPGP-encrypted emails (with mutt) with several of my friends and colleagues for various purposes. Of course the UX of GnuPG is terrible. Of course the security is not bullet-proof. Of course it's not for everyone. Yet I think it is a reasonable way to afford very strong security when we communicate, and it is extremely useful to us.

I don't think recent IM apps are a suitable alternative. Essentially all of these apps have young and unaudited code (assuming they are open source, which some aren't), and they are designed to run on phones with Google Play Services installed (i.e., a proprietary Google blob with complete access to the system), along with untrusted proprietary radio firmware from the phone manufacturer, all on a device with direct access to an untrusted cellular network. (Oh, and there is the issue that many things are inconvenient to do on phones, e.g., writing long emails. This wouldn't matter if these apps weren't essentially imposing a specific user interface on me, the way GnuPG doesn't. Oh, and many of these apps require a phone number, account creation on a specific provider, that I don't need for email, and that people can legitimately object to.)

By contrast GnuPG is ancient and well-tested code, running on computers, which mostly use free software, and which I trust much more than I trust my phone. I can do meaningful things on these computers, like writing long emails, coding, preparing confidential documents, that I essentially cannot do with a phone plus IM software.

Encrypted email has lots of drawbacks, sure, but it doesn't look to me like it is over. There are still lots of use cases for which there are no alternatives. And it's not because something isn't being used by everyone, or is less popular than other solutions, that it is not useful for some people, or that it won't last.


Is WhatsApp secure? It's a totally closed protocol owned by a company (Facebook) known for rampant issues with privacy.


Yes with no control over what happens to your key and you don't know if your message has been encrypted after it is sent and by default you aren't even told if the key of your recipient changes.


By making the discredited argument that WhatsApp's key-change behavior is a fatal flaw, you're disagreeing with:

* The EFF

* Moxie Marlinspike

* Matthew Green

* Bruce Schneier

* Isis Lovecruft from Tor

* the grugq

* Matt Blaze

* Avi Rubin

* Steve Bellovin

* Joseph Lorenzo Hall

* Bart Preneel

* Peter Honeyman

* Jon Callas (who cofounded PGP Corp)

* Paulo Barreto

... and about 50 more experts equally respected in the field if less known to the typical HN reader.


> By making the discredited argument that WhatsApp's key-change behavior is a fatal flaw, you're disagreeing with... and about 50 more experts equally respected in the field if less known to the typical HN reader.

No, the vulnerability was confirmed and the argument that it represents a fatal flaw for those needing fully secure communications is sound. No one competent (and intellectually honest) has disputed this, or would even try to do so. The open letter itself acknowledges it, and I know every open letter signer I followed did so as well.

What the open letter did was take issue with the language used by The Guardian, point out the potential for such language to scare some people into less secure solutions, and argue that the vulnerability is a reasonable trade-off for convenience that can benefit some users too.


No, no he's not.

GP was questioning the implementation and OP is merely identifying a clear weakness (regardless of who you says, it is a trade-off). Not quite an appeal to authority, but it's pretty damn close.

What exactly about the post is a discredited argument? I'm genuinely curious. WhatsApp can be doing anything behind the scenes without clients knowing about it, how is questioning this a bad thing?


"you're disagreeing with"

This is the problem with the security community, always with authoritative arguments. Building a religion around security is not going to end well. Some of those people already push government agendas and people eat it, because they were told not to question "experts".


It's not just security. Nicholas Nassim Taleb even invented a word for this kind of thing. He describes his complaint, in typical pugnacious Taleb style, here: https://medium.com/incerto/the-intellectual-yet-idiot-13211e...


Do you have sources for that? Because just looking at the first two you named the EFF (who marked WhatsApp down for being closed source).

And the owner of the Signal protocol (which is what WhatsApp uses). Obviously he's not going to argue against it.


EFF has criticized WhatsApp for being closed source, but not for this particular aspect of the key exchange functionality.

Because of the history around how WhatsApp was criticism over this and some of the apparent results of that criticism, tptacek particularly doesn't want people to conflate "there is something bad, unfortunate, or inadequate about WhatsApp" with "WhatsApp has a 'backdoor' in its key exchange" (and I understand that!).


> EFF has criticized WhatsApp for being closed source, but not for this particular aspect of the key exchange functionality.

The articles I've seen appeared carefully worded so as to achieve some balance, but did express some criticism and concern.

"Nevertheless, this is certainly a vulnerability of WhatsApp, and they should give users the choice to opt into more restrictive Signal-like defaults." from:

https://www.eff.org/deeplinks/2017/01/google-launches-key-tr...

Key change notification concerns paragraph from:

https://www.eff.org/deeplinks/2016/10/where-whatsapp-went-wr...


Thanks, that's a better way to put it. I should have phrased the distinction I was drawing more carefully.



The way to do it is to have it be a standard feature of the email client. Your client, if it supports it, generates the public/private key automatically. Every time it sends an email, it includes the public key. Every time it receives an email, it stores the received public key if there is one. Every time an email is sent, it encrypts it automatically if the address book has the public key.

Over time, it will work its way into most peoples' email, whether they are aware of it or not.

I know the metadata will still be plaintext, but if I am sending my tax return to my accountant, I don't really care if someone discovers that I am sending my tax return, but I do care that the tax return is encrypted. And I would demand that my accountant upgrade his email client.


Missing - key backup/escrow, verifiable trust (you can't just trust the first key sent to you for a specific email address), key revocation, and portability.

Solve too many of those problems and you've invented PKI...


Key backup - store it unencrypted on your own hard disk. The encryption/decryption is only for transport.

Verifiable trust - if it's a recipient you care about, you can phone them and ask if the key is correct.

Key revocation - easily done using email headers.

Portability - I proposed a portable system. Make it part of the email standard, just like the "Subject:" metadata.


There was a company that offered a product for Windows 95/98 users around 2000 that did that, except it worked with existing email clients instead of requiring new ones. It worked reasonably well, as long as the user understood the limitations such as the metadata issue.

There was no common extension interface for the major email clients (and many didn't have any extension support), so their biggest technical challenge was finding a reasonable way to hook in between the user's email client and the network.

Their first approach was to have their program run as a daemon that implemented SMTP and POP3, and the email client's proxy settings were changed to use the daemon as a proxy. Their setup program recognized the top email clients and know how to change their proxy settings. For other clients the user might have to manually change their proxy settings.

Later they switched to using the LSP interface to hook into the network stack to intercept and modify email regardless of what email client was being used.


> Better to move sensitive conversations to things like Signal, WhatsApp, or Wire

Just download an app that requires you to register with your phone number and then hand out your phone number to anyone you want to chat to. I mean, all the NSA-proof crypto is fantastic, but they didn't exactly test it with anyone who has had to worry about less theoretical non-state actors like, oh, jealous exes, abusive partners, creepy Tinder dates and so on.

Why do I need to give someone my phone number to use an IP-based chat service?


Yes, I find this a worrisome feature.

I understand why they make this tradeoff -- it makes onboarding trivial. And it lets them piggyback contact discovery off of people's existing address books. But the downsides are very significant and rarely acknowledged.


I'm not sure it does make onboarding trivial. How do I join WhatsApp/Signal/Telegram with an iPad or iPod Touch? A laptop? A handheld gaming console like a PlayStation Vita? I can't sign up for any of these services without a phone and a phone number. Sorry, my phone number is a legacy communication layer that should just be replaced by the Internet.

When I went to Brazil last year, I switched the SIM in my phone from a UK one to a Brazilian pay-as-you-go SIM. WhatsApp said "oh, you've changed SIM, do you still want to use your existing (UK) number?" I guess that's okay behaviour. Does Signal do this? Does Telegram do this? I don't know. I have to keep track of what goes on when I change my SIM. That's a layer violation. The network I use to access the Internet shouldn't affect the behaviour of my messaging app. If I'm chatting to my friends, it shouldn't matter whether I have a UK SIM, a Brazilian SIM or no SIM at all.

What if I move from the UK to Brazil permanently and get a long-term mobile contract? Shall I still use my UK number for WhatsApp/Signal/Telegram or should I use a Brazilian one? Will I lose my contacts? Will they stop being able to talk to me because my number has changed? I don't know. That's pointless administrative overhead.

What if I let my pay-as-you-go number go stale? If you don't make a call or text for six months or so, the mobile network may disconnect your number. At some point, they'll reassign that number. Can that person now access my WhatsApp/Signal/Telegram? Okay, well, maybe we put a secondary password on it like Telegram to stop the account getting hacked if I lose access to the mobile number. Now the person who inherits my phone number can't set up their own Telegram account because I'm already "squatting" the old number.

This is ridiculous to me. It's like an Internet without DNS, where we have to manually keep track of each other's IP addresses. Nasty legacy phone network shit. Don't need it, don't want to deal with it.


Yeah, I agree with you. What I should say is that it makes onboarding trivial for the most common case, and they explicitly don't care about any less common cases (even if they are not all that uncommon).


But is the problem email, PGP, encryption, the idea of secure online communications, or what?

The installed base of clients could be pinned on the failure of vendors to support encryption on email: Microsoft, Google, Apple, Yahoo, and others.

The protocol issues are indeed serious, and are a prime reason I argue for SMTP itself to be retired, as it should have been 20 years ago. The problem has been in coming up with an alternative protocol.

Browser clients can all be addressed through the browser (Google, Microsoft, Apple, and Mozilla being ~98%) and apps (Facebook, and a handful of other sites).

You don't mention mobile, but that's the platform of the future, with 3 - 7 billion users, and two vendors (Apple and Google) could address the situation there for 99%+ of all users.

Message archival will continue to be an issue regardless, though there may be ways around this.

The problem with the email alternatives is that none seems to be both open and a sufficiently flexible tool that it can address a sufficiently broad set of emails uses.

I'd really like to see discussion of specific properties any replacement protocol ought have.

I'll start:

1. Encryption-default-enabled.

2. Open standards, open source, free software. (BSD/MIT license as a standards-fostering implementation.)

3. Metadata protection.

4. Federated at client, server, and directory (e.g., addressing) layers.


> Encrypted email is pretty much over in 2017.

One could make the argument that encrypted email was never a thing outside a tiny, miniscule, group of folks.

Is it possible those same people probably bought more scrutiny than others simply because they used encryption?


As a messaging system Signal, Whatsapp can be considered as a replacement for email. But email is also used as your identity online. All auth systems use email for signup, login, password reset. How will this use case of email work with messengers?

In Signal, Whatsapp, messages are end to end encrypted, but one way where this encryption is undone in some ways is the backup process. Whatsapp allows users to backup messages and media to Google drive and it's unencrypted. So these messengers still don't solve the problem of having e2e communication + encrypted archives of data.


@tptacek is spot on actually: "The emerging consensus among experts is that it's not worth the trouble, or, worse, incapable of doing much more than generating a false sense of security."

I don't know what the consensus among experts is because I'm definitely not one of them. However, the Podesta email scandal/hack/case take-away for me was that... if the most expensive, modern digitally-aware campaign ever made did not bother to use GnuPG for emails, it means that it is pretty much a dead-in-the-water tech.


>"* An enormous installed base of clients that won't do encryption, meaning that at best you're attempting to tunnel encrypted messaging over an unencrypted transport."

These reasons seems to suggest that encrypted email is "all or nothing" and I'm not sure why that is. I am not interested in using encrypted email with all of my conversations. For non personal emails I am perfectly OK with sending email in clear test. For people who I communicate with only via encrypted email, I simply add them to my "encrypted email" contact list any my email client encrypts those emails. The reverse of this would be true on this recipients end. So the idea that its all or nothing is no different than HTTP traffic, example - I don't care if some trivial blog about cats isn't HTTPs.

> * A protocol that leaks metadata, including some message content, at the envelope layer.

This is true of some IPSEC modes as well example Tunnel Mode vs Transport Mode. If I am OK with just payload being encrypted then so what? Again why does this have to be all or nothing with regard to email?

>"Hundreds of millions of users that primarily access messages through browser clients that can't meaningfully implement crypto."

And encrypted email in this context would simply be "unavailable." This is not that much different from down-level versions of browsers or Java not being able to use newer versions of TLS.

>"* An unencrypted installed base that ensures encryption will be opt-in for the foreseeable future, meaning that users will routinely reveal plaintext accidentally by, for instance, quoting messages and forgetting to encrypt"

What is wrong with opt-in? Again why is it all or nothing? I either choose to add someone as an encrypted contact or not. People already leak all of kinds of information via forwarding email. Why is this an issue? I can also take a screen shot of an iMessage that was sent to me encrypted on my phone and paste it to someone else.

>"End user demands for things like search that can only be delivered efficiently at scale by databases of plaintext (most likely at centralized servers)"

Clients can maintain a local copy of an encrypted index. I don't think this is a non-starter.


You're absolutely right! If you're willing to be disciplined and communicate solely with similarly disciplined people, encrypted email can be very secure.

The basic problem with email is that the protocol does not adapt well to encrypted content. Too much usable data is in the envelope. On top of this, there are decades of UX expectations and conventions that become unsafe in encrypted contexts. As a result, most users will be highly unsafe by default, and any given person is likely to be significantly less safe than they believe.

But again, you're completely right! It's very possible to use it in a smaller group of people who understand and accept all the tradeoffs. It's just marginally less than ideal for mass adoption, which is where much of the thought is going.


> Better to move sensitive conversations to things like Signal, WhatsApp, or Wire

Will corporations adopt these platforms? You cannot share files easily, create threaded conversations, etc. More-over, one must register a phone number, and rely on the centralization of the provider of the service.

For some uses, emails serve as a record of communication, and to establish a public history of conversation (e.g., open-source development mailing lists). However, for these cases, encryption isn't used.


>emerging consensus among experts

Citation needed.

>at best you're attempting to tunnel encrypted messaging over an unencrypted transport

Like every encryption scheme ever.

>A protocol that leaks metadata

They all do? At best you get to choose which provider sees it, and even then timing attacks usually make short work.

>browser clients that can't meaningfully implement crypto

>search that can only be delivered efficiently at scale by databases of plaintext (most likely at centralized servers).

Technically, nobody uses browser clients; browsers can't talk SMTP. They use browser interfaces to cloud clients, over encrypted connections. The centralization is the main problem - you cede both security and control. Plaintext search is a weird example. That can trivially be done on a mobile phone.

Why bother? Because email isn't going away. Any encryption is a strict improvement. And it's federated! No network, federated or no, comes close to email's ubiquity.

Again, the main problem is the centralization to large providers. It makes it very hard to add encryption support as a protocol addon - you have to convince every provider that it's worth their while. The One True Solution is for people to start running their own mailboxes - which seems daunting, but plenty of once-sophisticated tech setups have been productized.

By analogy, I use SilenceIM for encrypted text messages. This is a very similar scenario to email - federated, plethora of unencrypted clients, no handshaking. The difference is the message gets delivered straight to me instead of a server somewhere, and I am free to install a new program that knows how to handle it. Mobile phones have actually provoked a once-unexpected shift back towards native clients for things (there was a time when people were predicting that web would completely eat desktop - thankfully they've piped down now).


This is incomprehensible to me. "Timing attacks" don't make short work of the equivalent of email metadata. Different systems reveal different amounts of metadata.


Every time I'm thinking of random side projects to work on, "make GPG easier to use" is always one of them. There are good things that can be done there, especially with things like keybase. Even a simple web-based (but completely offline) client would make it very easy for me to share encrypted text with colleagues and family, since they don't need to install something just to talk with me the 1% of the time I want to use encryption.

Each time I realize that Signal/Whatsapp/Wire are already doing much more here, and much better than I could do on my own. Sure, they're not email (sadface), but they've effectively solved this problem, by capitalizing on the paradigm shifts brought by phones -- there finally is a device on which to store keys locally (as opposed to on a server) that everyone has.


I realized that there was going to be no hope for mass adoption when I started using PGP. There couldn't be something more arcane and difficult to use, while at the same time adding nothing obvious to the experience except more work. You end up with something ranging from interest, but the hurdle of adoption is too high, or disinterest because "I have nothing to hide."

As you say, it's a problem that's already been solved in the minds of most people, with encrypted messaging apps. Most of the people using that want to hide from a spouse, or their weed dealer from their parents... they don't give a shit if the NSA is snooping.

Until/Unless people get seriously burned, the current trend will continue. A world that is gobbling up the IoT is not ready to learn about encryption.


Maybe I'm paranoid, but I'm suspicious of how, just as crypto is having a cultural moment, this expert consensus is appearing against the one cryptosystem that has demonstrated itself to be genuinely effective even against the most powerful of adversaries (Snowden used it and, perhaps more significantly, leaked internal NSA complaints that they're unable to break it when used correctly). And we're being pushed towards these centralized (and centralized in the US) systems instead.

Suppose the NSA have accepted that they can't hold back the tide: users want something that sounds like end-to-end crypto. At the same time they want to retain as much ability as they can to read users' messages. What would they do? If they had subverted some security experts, how would the world look different? How can we proceed robustly assuming that there are adversaries among us?

> An enormous installed base of clients that won't do encryption, meaning that at best you're attempting to tunnel encrypted messaging over an unencrypted transport.

IP is unencrypted, so that's always going to be the case on some level?

> A protocol that leaks metadata, including some message content, at the envelope layer.

True. Worrying. But certainly not fixable by switching to a centralised system. Doubly-certainly not fixable by switching to a system that uses phone number as ID.

> Hundreds of millions of users that primarily access messages through browser clients that can't meaningfully implement crypto. > An archive-always UX that ensures that huge amounts of plaintext are scattered around the Internet by both senders and receivers. > An unencrypted installed base that ensures encryption will be opt-in for the foreseeable future, meaning that users will routinely reveal plaintext accidentally by, for instance, quoting messages and forgetting to encrypt.

Client-specific, and avoidable. Better clients are to be encouraged.

> End user demands for things like search that can only be delivered efficiently at scale by databases of plaintext (most likely at centralized servers).

How do any alternatives avoid this?

> Better to move sensitive conversations to things like Signal, WhatsApp, or Wire --- the double ratchet construction is designed specifically to make IM-like protocols secure even when conversations are sporadic and last months.

There's a place for IM-like conversations, sure. I think there's a place for email-like conversations too. Certainly I want to keep what is by a long way the most battle-tested cryptosystem we have, OpenPGP (even if used over Jabber or the like rather than over email). And while I'm probably willing to use a double-ratched construction implemented by someone I trust (i.e. Matrix/Riot and possibly Conversations), I would certainly not switch to any centralized system or anything based on phone numbers. That would be a serious downgrade from what I have with OpenPGP.


What on earth could you possibly be talking about? No cryptosystem has proven itself less capable of standing up to state-level adversaries than email, and nothing demonstrates that more clearly than the progenitor of this "cultural moment", the saga of Edward Snowden.


Plaintext email doesn't resist state-level adversaries. Email with OpenPGP does.


If you are an active target of a tier 1 state, your endpoint will be compromised, your decrypted communication will be read, and no cryptosystem will prevent this.


Furthermore, if you're using PGP to evade a state-level adversary, the odds are overwhelming that you've own-goaled yourself many times over:

* You keep plaintext archives and drafts of your messages, because that's a fundamental feature of email clients going back 3 decades.

* You use a server-mediated PGP provider like Protonmail that has your security one surreptitious Javascript injection on an XHR call away from complete collapse.

* Your peer accidentally forgot to encrypt a response and quoted your own plaintext back to you.

These things aren't intrinsic problems of encrypted messaging systems, which is something I feel like HN is not doing a good job of grokking. They're intrinsic problems of email.


You keep plaintext archives and drafts of your messages, because that's a fundamental feature of email clients going back 3 decades.

To my knowledge, mutt doesn't store decrypted archives. Drafts are stored in /tmp which can be a filesystem stored in RAM. I think using Mutt also takes care of your second point.

Your third point is the biggest problem with any system where security is bolted on (e.g. SMTP, POTS, etc.) - your end may be secure but your interlocutor is liable to compromise you one way or another. Though, as you say, this isn't a fundamental property of all encrypted messaging systems.


"You keep plaintext archives and drafts of your messages, because that's a fundamental feature of email clients going back 3 decades."

No it's not, you don't have to keep it at all or in plaintext and it's irrelevant for new clients supporting encryption, their UX have to be redesigned anyway. So it's a UX issue at most.

"Your peer accidentally forgot to encrypt a response and quoted your own plaintext back to you."

Again, just a UX issue. Although all messaging apps actually have incentives to provide a UX that lets them spy on most people's communications or be open to add that possibility some time in the future.

"that has your security one surreptitious Javascript injection on an XHR call away from complete collapse"

This is a problem of centralization that you are trying to ignore and none of those messaging apps can solve it. Any centralized system is one tiny change away from a complete collapse. It can also be shut down by the state just to force people to use plaintext or backdoored alternatives. The problem is even bigger than it looks, even if you make a completely decentralized protocol there will still be incentives to centralize as much of it as possible to make money and still leave a strategic possibility to spy on everyone also for money. Makes sense?


> You keep plaintext archives and drafts of your messages, because that's a fundamental feature of email clients going back 3 decades.

Name one PGP-enabled MUA that actually does this? None of the popular ones (mutt, enigmail, claws) do it. They didn't do it in the 90s, because that would have been stupid, and they don't do it now.

You could have said indexing and search if you wanted to point out actual usability problems with PGP-enabled MUAs.


I disagree that these aren't intrinsict to any messaging system. Those are intrinsic problems of any electronic data storage and transfer system. Data tends to deleted or public, as Quinn Norton says.

I've been thinking this problem through for a while, and I'm starting to think we simply need to have a different mindset for thinking of electronic data as for hardcopy. I'm leaning toward "data physics", in the sense that there are different "rules of physics" which apply -- a metaphor, though close enough to the truth:

* Data have effectivley no inertia. They can move anywhere at the speed of light.

* Data violate the principle of location. Information can be in two (or more) places at the same time.

* Data can be exfiltrated without awareness of the subject. Most especially when held on third-party systems.

* Encryption isn't a safe. If you lose a safe key (or combination), you can drill it out. If you lose an encryption key, an entire corpus is no longer accessible. This has absolutely massive implications from a user-support standpoint, as such key loss will be an everyday (or, at global scale, every second) occurrance. Which means some sort of reliable, useful, but still sufficiently safe, and cheap key recovery system.

Essentially: you can be compromised at any time, by any number of actors, without notice. There may be some ways to address this, but throwing more crypto at the problem may not be it. "Canaries" or fictitious entries and sentries (URLs, emails or phone numbers which should never be contacted, but which if they are, you know you've been had), might be part of that.

Another realisation I had a while back was that as much as this hits the ordinary citizen, it's as much a concern for those in or near power (finance, politics, military, journalism, business, etc.) as well. See open speculation that the White House has been compromised, quite possibly by multiple intelligence operators from multiple nation-state, and possibly other, actors. Including those of the United States itself.

Which is to say: media change the societies in which they operate, and always have. Elizabeth Eisenstein made hay with this in her 1979 book The Printing Press as an Agent of Change, though I'm finding her 1968 paper prefacing that work a more concise and sufficient summary of the principles: "Some Conjectures about the Impact of Printing on Western Society and Thought: A Preliminary Report"

http://www.journals.uchicago.edu/doi/pdfplus/10.1086/240164


There are a large number of other things we need to fix to make individuals able to resist state attackers, sure. We need much better software security in general, more transparency around firmware.... Still, good crypto makes attacking more costly, even for states. My sense is that the cost of attacking someone using email and OpenPGP will be higher than for any existing alternative (including WhatsApp/Signal), at least for an adversary with the US security services onside.


And if you were aware that a state-level actor is targeting you, you'd be using a programmable, self-contained HSM for all sensitive computation. This is a topic I'm working on actually.


That sounds interesting. Do you have anything you could share on the topic ?


It's still in the early stages honestly. I will probably share more after the first paper is published.


You mis-spelled "iphone."



You have no idea what you're talking about mate. As a state-level actor, I could obtain/develop a nice Safari root exploit and get the target to visit a malicious site. Heck, if I were the NSA, I could probably get Facebook to insert the exploit payload once the target logs in. Game over.


I agree. Sure, PGP is somewhat outdated but it is still solid as long as keys aren't compromised. There is a good argument for new form of communication that is encrypted by default, but email has such a huge headstart that I'm not sure that's practical.

WhatsApp on the other hand is a proprietary platform restrained by commercial interests, closed source, and lack of verifiability as well as opaque key management.


I think, the points about search and archival must apply to any communications medium.

If one isn't searching through their IM history but does so for emails, I guess, it should mean they just don't send anything non-ephemeral via IM.

(May be this is because IM has pool usability for any large bodies of text, while email handles those really well.)


> meaning that at best you're attempting to tunnel encrypted messaging over an unencrypted transport

Can you elaborate on what you mean by this? If the message is encrypted (for example: PGP/GPG) why do I care whether the transport is secure or not?


It means that the end user is responsible for ensuring all data is encrypted, where it's easy to make a mistake and accidentally let something through unencrypted. Combined with this other bullet point:

> An unencrypted installed base that ensures encryption will be opt-in for the foreseeable future, meaning that users will routinely reveal plaintext accidentally by, for instance, quoting messages and forgetting to encrypt.

That spells disaster. This is in contrast to something like TLS, where a user with absolutely no technical understanding can visit a website with HTTPS without having to think about it at all (mixed content aside).


I agree with everything you are stating except that IMs can't replace email anymore than email can be 100% secure. Apples and oranges.


It is an absolute violation of my expectations for you to say that a browser cannot meaningfully implement crypto.


I would appreciate if you would stop recommending Whatsapp so uncritically at this point.


>* An enormous installed base of clients that won't do encryption, meaning that at best you're attempting to tunnel encrypted messaging over an unencrypted transport.

All encrypted communication on the internet is 'attempting to tunnel encrypted messaging over an unencrypted transport.' That's literally the entire point. I don't care that my messages are broken down into unencrypted IP packets, because my messages are secured by PGP.

>* A protocol that leaks metadata, including some message content, at the envelope layer.

Could you elaborate more on this? I don't really understand what you mean.

>* Hundreds of millions of users that primarily access messages through browser clients that can't meaningfully implement crypto.

They don't. They could, but they don't. There's no lack of capability.

>* An archive-always UX that ensures that huge amounts of plaintext are scattered around the Internet by both senders and receivers.

That's nonsensical. Nothing about archiving data requires that it be stored in plaintext. Again, that's basically the whole point of encryption.

>* An unencrypted installed base that ensures encryption will be opt-in for the foreseeable future, meaning that users will routinely reveal plaintext accidentally by, for instance, quoting messages and forgetting to encrypt.

That's just bad UI design. At the end of the day, your intended recipient has to be able to actually read the plaintext, and they will always be able to reveal its contents, if they want to. No matter what you do, they can always just take a screenshot or show someone their device's physical screen.

Making it easy or difficult for them to accidentally reveal it is entirely about UI design. If you quote encrypted mail when composing an unencrypted mail, make the screen go red!

>* End user demands for things like search that can only be delivered efficiently at scale by databases of plaintext (most likely at centralized servers).

Rubbish. GMail is clearly what you're mainly referring to, and the idea that GMail requires big plaintext databases for spam prevention or searching is bullshit. It doesn't.

>All these problems are probably surmountable (with enormous, concerted effort).

Most of them aren't real problems, and those that are, are quite easily fixable.

>But: why bother? Email is just one of dozens of messaging systems available to Internet users. Better to move sensitive conversations to things like Signal, WhatsApp, or Wire --- the double ratchet construction is designed specifically to make IM-like protocols secure even when conversations are sporadic and last months.

Sorry, but I have no interest in proprietary messaging protocols. I want to use email. I like email. Everyone understands email. Git understands email.


    * A protocol that leaks metadata, including some message content, at the envelope layer.
  Could you elaborate more on this? I don't really understand what you mean.
He means:

  EHLO your.mx.example.tld
  MAIL FROM:<bob@example.tld>
  MAIL TO:<alice@recipient.tld>
  DATA
  From: Definitely Not Bob <bob@example.tld>
  To: Probably Not Alice <alice@recipient.tld>
  Date: Mon, 12 June 2017 16:02:43 -0500
  Subject: Super Secret EMAIL! Don't let Eve see!
  
  -----BEGIN PGP MESSAGE-----
  ...
  -----END PGP MESSAGE-----
  .
  QUIT
Thereby leaking at minimum the sender and recipient, as well as any intermediate hosts that relayed the email. Depending on how your MTA handles email, you can also leak subject and other metadata.


Why would I take the time to answer any of your questions? You just wrote upthread that, because I have a background in security consulting, everything I write must be designed to cause security problems so I can line my pockets.

No thanks.


All of those are controlled by a single company. A single point of failure. Untill there is good enough protocol anyone can run a server of and it is - pgp stays the least worst option.

All of those messingers are terrible - in email you cannhave multiple conversations with the same person by design. In thoae you can't. There is no messinger with decent history search.


I'm having the impression your friendship with moxie is altering your judgement.

Signal and Whatsapp are not a solution for everything and will certainly not replace email.


Moxie and I are barely acquaintances.


Yes! Thank you.

Email is not and will never be secure. Sorry people but it just fucking sucks when it comes to encryption.

We have many other great communication protocols that were designed from the ground up to be secure as they can be.


Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: