
Secure Messaging Scorecard - sinak
https://www.eff.org/secure-messaging-scorecard
======
Mandatum
CryptoCat, the very same client that had an abysmal MitM attack exposed in
which the author said "don't worry, it's on a prototype debug version" when
infact it was on the deployed real-world version?

Yeah, nah.

[https://news.ycombinator.com/item?id=7518761](https://news.ycombinator.com/item?id=7518761)

EDIT: I get that they're choosing more "user-friendly" and "freely available"
applications for featured, however this is really only going to get seen by
people who don't mind a bit of fiddling around. ChatSecure + Orbot I see as
the most reliable on there, yes you'd stick out like a sore thumb (why are you
using Tor?) however privacy-wise, you'll be just fine.

~~~
pde3
For the record, we aren't endorsing CryptoCat or any of the other tools that
got 7/7: ChatSecure, TextSecure, Signal/RedPhone, SilentText, SilentPhone, or
the ones that are close like Pidgin+OTR, Subrosa, Surespot, Telegram, Threema
or iMessage.

Getting those scores is a sign that those projects are taking the right
approach. Lots of codebases have horrific bugs, including OpenSSL, older
versions of the SSL and TLS protocol itself. We believe that focusing the
community on the task of moving the best projects forward is more constructive
than

Testing the tools that are scoring highest for usability, and doing deeper
examinations of their designs and codebases, is going to be a future component
of this campaign.

~~~
Mandatum
That sounds like a good start - but I still think the way you're presenting
this data is somewhat misleading. I agree that many codebases have horrific
bugs, and ultimately exploits will eventually come to light in existing
applications.. However instead of listing many different available
applications, why not just list what works for "right now"?

For instance: "Encrypted so the provider can’t read it?" on Skype's messaging
just isn't true. If a subpoena was issued to Microsoft for conversation data,
it'd be available.

I think it'd be better to stand up what does things right, and for everything
else say why you're not listing them as effective. Approach is all well and
good, but to an anonymous journalist/source being pursued a groups - approach
over execution could be the difference between freedom and imprisonment (or
worse).

~~~
pde3
Sorry, I tried to reply here yesterday but was rate-limited out of the
conversation :/

Our aim with this project is to not give advice about what works "right now",
because we aren't convinced there are any secure messaging options right now,
especially when the usability dimensions of security are taken seriously.

Instead, what we're trying to do is articulate the things that both large
companies and open source projects need to be doing to move in the right
direction.

Since this is phase 1 of a multi-part campaign, we're going to take a closer
look at the usability and further security properties of tools that are doing
well on the Scorecard in subsequent phases.

On Skype, before launch it wasn't clear clear to us whether the NSA's reported
Skype intercept capability came from breaking or having Microsoft backdoor the
crypto (which would mean they loose the second checkmark) or by having
Microsoft hand out a false public key for the other party (which is possible
due to the lack of a check mark in the third column). We have an ongoing
conversation with Microsoft about this and are reviewing Skype's ratings at
the moment.

~~~
Mandatum
Oh I see, so the intended outcome is to raise awareness with the general
public and shed light on issues with existing software they use in hopes that
the powers at be take notice?

------
justcommenting
Google Hangouts, iMessage, fb Chat "Audited"? Wish they could point readers to
those audits..

Skype "encrypted so the provider can't read it" even though MS is a known NSA
PRISM partner? Recent disclosures quite strongly suggest otherwise.

Also weird that they didn't sort these options by ranking. it took me a while
to realize that they weren't displaying everything ranked by default (or the
"winners"). to see that, choose 'all tools' from the drop-down.

~~~
AlyssaRowan
Confirm: Skype doesn't deserve that second checkbox. It is now centralised, is
a specific real-time PRISM source, including full chat logs, and does indeed
read the messages going through it.

Try posting a URL on there and watching an anti-malware scanner ping it
sometime. I don't know if they're still doing that, but they definitely were.

~~~
MichaelGG
It deserves it as much as iMessage, doesn't it?

~~~
josho
?

My understanding of iMessage is end to end encryption. Where the endpoints are
the separate certificates on each of your devices.

~~~
MichaelGG
AFAIK, iMessage encrypts the message on your device. But Apple sends the list
of keys to encrypt to. If that list is accurate and only includes the public
keys of the target user, fine. But you've no way to verify this. I don't see
how Skype is any worse.

------
lawnchair_larry
This is definitely a moment of weakness for EFF's credibility. They need
experts to review these before going public. Many of these are demonstrably
false.

~~~
pde3
We do have experts reviewing them. Which items are false?

~~~
karlhedderich
Last I knew it was strongly suspected that Skype could look at your messages.
imessage doesn't pass the mud puddle test indicating apple can look at your
messages. Facetime should probably considered suspect but I don't know of any
articles that demonstrate how the key exchange is handled. BlackBerry also
modified their messaging app to be able to give info to LE. Telegram doesn't
have open source server code and uses home rolled crypto. Was telegram even
properly audited?

I would also recommend categories for what metadata is exposed; if messages
are encrypted at rest on your device; cross platform ubiquity.

You should include bitmessage, and i2p-bote.

I am glad that this is only the first step but I do think that you shouldn't
have done it alphabetically but rather by score and usability.

~~~
pde3
It wasn't clear to us whether the NSA intercepts Skype by breaking the crypto,
or by compelling injection of false public keys in order to perform a man in
the middle attack. In the latter case it's the third checkmark (lack of
ability to verify keys) that's their users' undoing. We're talking to
Microsoft about that at the moment, and may revise that entry.

There's a weird case around iMessage and any tool that is provided by an OS
vendor. I think we need to add a note about this, but in those cases that
company could inject malware or a backdoor either in the messaging system or
somewhere else in the OS. Since we're trying to tackle one hard problem at a
time (secure messaging but not secure operating systems and software
distribution) there should be an extra caveat about offerings from OS vendors.

The only tool that gives strong metadata protection right now is Pond, and we
aren't listing unusable tools that aren't out of beta yet. We considered but
haven't yet included bitmessage for the same reason.

~~~
justcommenting
question asked in good faith: does it really matter to a given user exactly
how adversaries are successfully attacking Skype? shouldn't some of the things
that we've already seen disclosed--e.g. that NSA gained significant, at-scale
capabilities against Skype right after it was acquired by Microsoft--be enough
to invalidate essentially any crypto-related promises the company may assert,
or even those that an audit might support? if not, i think it's at least worth
making a distinction between products with known backdoors and products
without them. today's TAO attack is tomorrow's phd thesis, etc.

------
SCdF
Wow, I didn't think the EFF were this… misguided.

> 2014-11-04 : Snapchat app has audits from an internal security team.

That being enough for the "yes the app is audited" tick, plus CryptoChat
looking to any passer by of this site as being A-OK is really concerning.

------
pde3
Some extra info for commenters in this thread:

This Scorecard is phase 1 of our multi-stage campaign for Secure and Usable
Crypto. We believe these criteria are necessary for any strong security tool,
though meeting them doesn't guarantee that a system is perfect.

Subsequent phases of the project will focus on usability (which is a huge
problem for activists and journalists who try to use encryption), metadata
protection, openness and federatability of protocols, and much deeper audits
of the design and implementation security of the software that is scoring
highest.

~~~
tptacek
Who reviewed this scorecard for technical accuracy, and who reviewed the
methodology?

------
footoverhand
While verifying some of the claims of the EFF's scoreboard, I discovered a
website which scores programs by their policy decisions and security [1].

I also find it interesting that mumble wasn't mentioned in this secure
messaging scorecard, but since this is the first step of a multi-phase
project, I imagine the EFF will be updating it.

[1]. [https://openintegrity.org](https://openintegrity.org)

------
cauterize
Was there a particular reason Tox ([https://tox.im/](https://tox.im/)) wasn't
included?

~~~
FractalNerve
I second that. First thing I looked for was how they rate tox. Hopefully it
gets added :)

------
arca_vorago
I just want to mention a highly underrated/undervalued program in this arena:
Murmur/Mumble.

It's open source, easily self-hostable, and both chat and VOIP are encrypted,
vis TLS and OCB-AES respectively. The downside is that it is not end-to-end,
and a compromised server would compromise communications, which is why I like
to self host on something I've hardened myself as opposed to buying the
service from somewhere else.

Also, it can also be pushed over TOR if you want...

I highly encourage anyone who uses teamspeak, ventrillo, etc, to switch to
mumble. I use it to keep up with friends and family from all over the world.

------
patcon
They really need a wiki with talk pages for each of these applications...

EDIT: As in, I'd like to know what the details on each analysis is. And I'd
also like this to be more like a living document.

------
Istof
Skype is definitely not safe since Microsoft purchased them and replaced the
super-nodes with MS owned servers

------
api
How good these are from a security point of view seems (roughly) inversely
proportional to their popularity.

~~~
ekimekim
Which begs the question, in what direction is the causation? Are less secure
apps more popular because they're a better user experience? Are popular apps
less likely to be secure because they're owned by large companies less
inclined to give up precious access to information? A bit of both? Or
something else entirely? Answering this question is the first step to making
truly secure communication actually USED in day to day life.

~~~
api
I think it's a question of focus. The more secure apps are a product of focus
on security. The less secure ones are a product of focus on marketing and user
experience.

------
Spearchucker
Something that worries me is that while all the attributes the scorecard uses
are good, there's at least one other I care about (in addition to those
listed).

Uploading my contacts.

Even TextSecure does it. I'm still looking for something that gets a deserved
perfect score and doesn't go near my contacts.

~~~
lucian1900
TextSecure does it in a nice way: [https://whispersystems.org/blog/contact-
discovery/](https://whispersystems.org/blog/contact-discovery/)

------
gankgu
Thanks, I want to know about wechat.
[http://www.wechat.com/en/](http://www.wechat.com/en/). Can any one provide
the secure messaging score ?

------
Animats
Is Skype still encrypted end-to-end? I thought that a few months ago,
Microsoft started running all Skype calls through Microsoft servers,
decrypted.

------
t3ra
I wonder why they didn't include telegram?

~~~
diafygi
It's there if you select "All Tools" instead of "Featured Tools" in the
dropdown.

~~~
chmars
Thanks!

(Why isn't Telegram featured?)

~~~
tptacek
Telegram? They buried _PGP_.

~~~
Mandatum
A blog post on the protocol built by Telegram:
[http://unhandledexpression.com/2013/12/17/telegram-stand-
bac...](http://unhandledexpression.com/2013/12/17/telegram-stand-back-we-know-
maths/)

 _sigh_

~~~
swordswinger12
Side rant: I really hate that people conflate expertise in math with expertise
in crypto. Dozens of completely useless cryptosystems are published every year
by talented mathematicians.

------
andy318
Thanks to the EFF for doing this!

------
nichochar
The design of this page is so bad! ahaha

Anyway, helpful tip: you can chose "All Tools" instead of "Featured tools" on
the top left of the beautiful table

