With the recent dust-up around protonmail's "end-to-end encryption", it's hard for me to believe that the hammer isn't going to come down on Matrix in a similar way. Nobody has seriously turned their attention to the compromised-homeserver issue, even though that's the only threat model under which e2ee makes sense. Expecting users in hundred-person group chats to manually verify each other individual user's keys is nonsensical. And even a single unverified key means that their homeserver operator could be listening in. I feel like Matrix should move more towards emphasizing how easy it is to self-host and maintain control of your own Matrix data, making e2ee is a non-issue.
> I feel like Matrix should move more towards emphasizing how easy it is to self-host and maintain control of your own Matrix data, making e2ee is a non-issue.
Matrix is way ahead of you. There's ongoing work to build P2P clients [0]. That is, each phone/device can be its own home server. The latest update was in May [1].
That sounds very promising! Having two mobile devices with 5+ radios (2G, 3G, 4G, wifi, Bluetooth, NFC, ...) inside 5 cm from each other yet they can't easily talk directly to each other in a user friendly way is such a fail!
Like really, my Palm TX could do that via infraport!
So any effort to break this stalemate makes me very happy. :)
I have tried to understand the reason for this design limitation (no easy efficient peer to peer on phones) for a long time now.
I have come to the conclusion that the inability to communicate is not a fail but intended.
It seems like security and business interests are standing in the way, not primarily technical hurdles.
Mesh networking and service discovery are issues where technical solutions exist. But their application is slowed or blocked by network operators and phone/OS manufacturers to enforce a central authority and paying subscribers.
Privacy concerns are often used to explain these decisions, but I see those as mere excuses. Mesh node identifiers could just be randomly regenerated periodically or handled anonymously. Also firewall rules could easily ensure only authorized services can communicate.
It was interesting to observe how quickly similar P2P features were enabled in the fight against Covid-19 (contact tracing via BLE advertisement packets).
At the same time it is harder than ever to use, for example, Android's WiFi or Bluetooth in an App-controlled manner. Only the central authority, not the owner of the device nor independent App developers are apparently supposed to actually use the devices capabilities.
Quite frustrating.
I have been thinking about building a generic case/USB gadget to enable free communication, but such a solution would have many drawbacks versus using the internal radios.
Privacy concerns are real, but possibly misguided. Mesh networking will obviously reveal much more to people around you than the centralised model. But in return you get shrouded from centralised infrastructure. It's a legitimate trade-off and I imagine people preferring one way or the other. But instead of talking about a trade-off, we tend to reject ideas because one of the aspects would regress.
I also think it's intended by the people actually paying for development of the phone hardware and OS. Also iOS being fully proprietary and even the open parts of Android being effectively developed by a privileged group of Google engineers with zero community input for project direction does not help...
The involved parties want to sell you cloud services(hello Google!), to have you use up mobile data & calls/SMS (hello operators! ) and ideally to throw the phone away after a year or two (hello manufacturers).
And all they need to do is build a device OS combo that needlessly peddles data via cloud somewhere on the internet, has not data card slot and can't talk directly to similar device on the table next to it...
Matrix P2P removes the homeservers, but clients are still talking to each other through some medium that can impersonate them.
There's really no magic solution to solve this, you need either a trusted third-party (such as Certificate Authorities, who are expensive) or a Web of Trust (impractical for users).
Matrix's existing E2EE has identity verification as effectively one- or two-hop web-of-trust using QR codes or emoji comparisons. It's not impractical for users, because everyone's familiar these days with scanning a QR code as a mechanism for logging into an app (c.f. Discord, WhatsApp, Signal etc). So as long as you scan the people whose identity you care about, you're sorted, and it's much easier for normal users than the clunky old PGP signing parties of years gone by (plus it doesn't leak as much metadata, given it's just one or two hops of trust).
For me it looks very weird and defeats the whole purpose of Matrix. I mean: it was designed from the ground as a federated network, similar to E-mail. Turning it into a completely different topology will not end well. If they want a P2P client, they should design it from the scratch. Otherwise it'll end up as a bunch of hacks glued with tape.
It wasn't designed exclusively as a federated network, tbh - we've been planning for P2P since pretty much day one; https://matrix.org/~matthew/2016-12-22%20Matrix%20Balancing%... is a talk I gave in 2015 when Matrix was less than a year old on our plans to go P2P.
Personally, I think it's really nice that in P2P Matrix precisely the same client-server API is used as for normal Matrix, so all the work that goes into the Matrix client and E2EE etc is preserved without any changes at all. Instead, it just talks to a server running on localhost, which then replicates traffic around over the P2P overlay (in future using store-and-forward servers if the target server is offline). It's really not a bunch of hacks, thanks to Matrix being architected to make the replication transport & protocol pluggable from the outset.
Realistically, when would you be in a hundred-person encrypted group? Mostly this is the case when you're a member of some kind of organization, and there are ideas how to solve this case without pairwise verifying all participants (e.g. by delegating trust through a single trusted person such as the CEO, reducing the number of verifications necessary from N(N+1)/2 to N). Even without this, fully verified E2EE is still feasible and useful for smaller groups.
And even if you own the homeserver, you still want E2EE since you don't want the data to rest in plaintext server-side.
How important is it for these kind of groups to be E2E encrypted though? If you're sending a message to 100 people then you probably ought to consider it de facto public even if only the intended recipients receive it.
"You can fool some people sometimes, but you can't fool all the people all the time."
If you don't verify the keys, e2ee is basically meaningless against targeted surveillance. As long as some fraction of people verify keys, it is still effective against mass indiscriminate surveillance.
how is e2e better against mass indiscriminate surveillance than just normal TLS? The only time when e2e is meaningfully different then https is when the server you're talking to (i.e. your personal matrix homeserver) is compromised. In that case, aren't you already in the realm of targeted surveillance?
Some homeservers are larger than others (e.g. matrix.org). They don't all need to be compromised to enable mass surveillance. It also depends on where TLS is terminated. If you're running a homeserver on AWS or something behind their load balancer, there's a difference.
Generally, I'd argue that E2EE provides defense in depth against "unknown unknowns" if server infrastructure is compromised by any means. Although I do acknowledge it adds one more level of complexity, and often another 3rd party dependency (presuming you're not going to roll your own crypto), so it's not a strict positive.
> The only time when e2e is meaningfully different then https is when the server you're talking to (i.e. your personal matrix homeserver) is compromised.
Only if everyone's running their own personal homeserver, which seems pretty unlikely for regular people. You could've said the same thing about email (it's not meaningfully different unless your personal email server is compromised), but in reality the NSA ran mass surveillance on gmail and picked up a lot of data that way.
Serious question, if a surveillance organization had control of a certificate authority trusted by your client, would that allow them access to traffic whose security relied on a certificate from that authority?
By that logic the vast majority of users of whatsapp, signal and most other e2ee protocols/apps use it in a useless way, right? Most people I know who use these apps (even the security-conscious ones) never verified the key.
Signal tells you outright when someone's key has changed, though. It's usually pretty trivial to establish that the original conversation is authentic when you're just talking with people you know in real life (where an impersonation attempt would likely fail for numerous reasons), and you can assume that device is still theirs until their key changes.
There is still a risk that someone is running a MITM attack. The initial conversation would be authentic, but the key belongs to someone else who is just forwarding the messages. Your communications would no longer be private and they could switch from passive eavesdropping to impersonation at any point without changing the key.
Most people rotate their Signals keys every time they rotate their phone hardware (which is inexplicably often for some people apparently), because keys are just auto-accepted everywhere so there is no real incentive to bother moving them. In larger groups there's always someone.
It isn't helped by the fact that the backup process is a bit obscure and doesn't work cross operating systems. For the select few that cares, verifying keys is effective against attackers who aren't Signal themselves, Google or in control of the Play Store. Just make sure to keep an eye out for that key changed warning, it's easy to miss.
Right, the dust-up was "PM advertises their service as secure and end-to-end encrypted, but in practice that doesn't mean much because user access is non-anonymous and they're still a central metadata honeypot for any law enforcement agency to log accesses to" (compared to e.g. Signal which AIUI is designed as an anonymous dropbox so that Signal can't correlate between user access to their servers and individual senders)
Even if it's not currently practical for 100 person group chats, verifying each other's keys is quite practical for 1on1 or other small groups of say 10 people or so, and you get a great benefit from doing so! Two normal users could use any matrix server, including the huge matrix.org one, and communicate privately in a direct chat with peace of mind.
Of course, I think self-hosting and decentralizing the servers is very important and good work too!
>I feel like Matrix should move more towards emphasizing how easy it is to self-host and maintain control of your own Matrix data
The problem is, it isn't easy at all. Running homeserver requires gigabytes of RAM and significant CPU performance to work at usable speeds. It just isn't feasible to run it on some dirt cheap VPS, you need beefy machine.
You'll be thrilled to know this pretty much isn't true anymore, even with Synapse. My instance is running on a cheap Hetzner VPS with a not very powerful CPU and is currently using about 700M RSS and not much CPU. And I'm in a lot of rooms, some of them quite large and high in traffic.
I'm also not even using Synapse workers at all, just a monolithic instance. Splitting the setup into workers would buy me an additional speedup if things got overly slow.
Yes, unfortunately Synapse is possibly one of the worst apps i've ever sysadmin'd for, and I'm not even the primary sysadmin for my homeserver. I still use it, regularly, but we're all really anxiously anticipating the release of Dendrite. I'm not the biggest fan of the protocol either, or the UI/UX of Riot. I think Matrix is a good idea, but there's a lot of historical baggage and I think the world probably needs one more "throw everything away but learn the lessons of the past" cycle before we get something really, truly good in the chat space.
If you want to actually use the federating feature, you will probably join some big channels. And that causes pulling of some huge amount of data from other homeservers.
disclaimer: I tested it some time ago with Synapse. Now I see there is also new homeserver software, Dendrite. It is possible that it is order of magnitude less resource hungry, though I wouldn't count on that.
One of our primary goals on the Synapse last quarter was to make it possible for a new homeserver to join Matrix HQ (a large, public room) in under 1 GB of RAM. And we did it. https://matrix.org/blog/2021/06/15/synapse-1-36-0-released
It's not the slimmest beast, in part because Synapse needs to scale up to country-scale deployments, but its ability to scale down has significantly improved over the past year.
That said, Dendrite and Conduit (https://gitlab.com/famedly/conduit/) are exciting projects which will optimize for different operational contexts.
How and why is that different from a client being in the room? I'm pretty sure that neither desktop, nor especially phone clients are supposed to take gigabytes of mem to listen to chats.
I'm not sure when you last tested, but my Synapse instance (v1.42) is currently using around 180MB RSS. I'm on a few rooms with 500+ users and multiple with 50+ users.
> Expecting users in hundred-person group chats to manually verify each other individual user's keys is nonsensical.
Maybe this could be helped if Matrix clients supported OpenPGP encryption which allows for key signing/web of trust? Some XMPP clients already have OpenPGP support, it would be nice to one day be able to send encrypted messages to Matrix users.
Still haven't received the up-to-date version from F-Droid, which is still stuck at 1.2.0. They really should have some provision to expedite a release when a serious security flaw is discovered.
Not the Matrix team's fault, but still a bit worrying. Maybe they could implement their own F-Droid repo to skip the F-Droid build server?
F-Droid do the builds rather than Element (or other Matrix client vendors), so the whole thing is at the mercy of the F-Droid CI process. The Element Android build has entered the queue apparently, but not landed yet. We alerted the F-Droid team last week to prepare for the release, but beyond that there's not much we can do.
F-droid is a blessing and a curse. Many devs experience this 'F-droid catching up with releases' due to the manual signing process. The userbase is always confused about that. Reproducible builds are claimed to be the possible solution but almost nobody is doing that due to the complex setup.
Are reproducible builds the solution? Seems like the fdroid folks would still have to build ("reproduce") the apk to verify that it matches the one provided by the developer. Then they'd sign it and distribute the double-signed apk.
I was under the impression that reproducible builds guard against the "fdroid build servers got compromised" attack vector, (or "developer got compromised", as compared to dev-signed releases) and nothing else.
My understanding is that the level of trust toward the build server can change.
Currently, most things are signed by F-Droid, so distributed building is harder. But later less trusted servers could build the apps and check, leading to a consensus system.
If a project maintained reproducible builds, then they can have the best of both worlds: their own repo where they can publish builds as they like, then f-droid.org can later reproduce them and use the upstream developer's signature.
Or they could coordinate with the F-Droid packagers, like they did for other distributions. (Actually they might have, since they don't exhaustively list them)
Element did coordinate with the F-Droid packagers on behalf of both Element Android and SchildiChat.
Because F-Droid runs their own builds and requires published source code, there is no mechanism for pre-building and staging security releases for issues which are not yet publicly disclosed.
That said, the packager was extremely helpful and prepared the metadata updates in advance to ensure that both applications could enter the build queue as quickly as possible upon release:
> Because F-Droid runs their own builds and requires published source code, …
That seems… completely reasonable for distributors of open-source projects. Mandatory, even, for compliance many open-source licenses. Are you saying there are other open source repositories (e.g. Linux distros) which would not require published source code before distributing binaries?
I wouldn't go that far. What I meant is more that Element would've been willing to share our private source tree for Element Android 1.2.2 with F-Droid several days in advance of disclosure, if it meant that they could pre-build the binaries and have them ready to ship the moment that the source was made public.
(After verifying that the public source is identical to the privately shared copy, of course.)
Do you need a full source tree for this ? IIRC major Linux distros do this via preliminary disclosure to a private mailing list possibly with a patch attached.
That way distros can do a local build to verify the patch works and fixes the issue & they then apply the patch in their public infra right after the embargo runs out.
The fdroid maintainers are... Less than competent. I recommend anybody not to use the main repo at all if they care about security, as any update could take weeks to arrive.
The maintainers aren't really the problems - it's their, long and silly build process + schedule. Updates are _always_ at least three days behind, because their build schedule is set in stone and they refuse to change it.
Quote from the original documentation: "The recommended strategy is to share the keys automatically only to verified devices of the same user. Requests coming from unverified devices should prompt a dialog, allowing the user to verify the device, share the keys without verifying, or not to share them (and ignore future requests). A client should also check whether requests coming from devices of other users are legitimate." - It made me smile, because should and recommended doesn't generally mean anything to developers. Only mandatory, must and must not, sections might get some attention.
> This is not a protocol or specification bug, but an implementation bug which was then unfortunately replicated in other independent implementations.
> We will also accelerate our work on matrix-rust-sdk as a portable reference implementation of the Matrix protocol, avoiding the implicit requirement that each independent library must necessarily reimplement this logic on its own
I like the idea of providing a core Rust/C++ library that can be called in any other runtime via FFI, compiling to WebAssembly, etc. I've used the same technique to ensure consistency in usage of a custom time-series compression format across an organization. Overall this technique seems in-line with the "DRY" principle, and the tradeoffs have been favorable in my experience. I think it will be a good step forward for the Matrix ecosystem.
The current way we're approaching this is to split the reference E2EE implementation into its own rust crate (https://github.com/matrix-org/matrix-rust-sdk/tree/master/ma...) which can be used with any SDK (e.g. we're almost finished embedding it into the Kotlin matrix-android-sdk2 client)
Separately, there's also the overall matrix-rust-sdk https://github.com/matrix-org/matrix-rust-sdk for clients to use as a "full fat" Matrix client SDK - as used by Fractal Next (https://gitlab.gnome.org/GNOME/fractal/-/tree/fractal-next) etc. We might end up using this in Element too in future (especially in Element iOS, where a Swift UI + matrix-rust-sdk backend could be quite a cute next generation architecture).
So while the first generation reference Matrix SDKs (matrix-js-sdk, matrix-ios-sdk and matrix-android-sdk) were completely independent implementations, each with their own bugs and increased audit surface, we're hoping that matrix-rust-sdk will simplify this a lot in future.
Quick comment, just want to show some additional support for the shared crate approach that Matrix is pursuing, it should give people a lot more confidence to try alternate clients.
I throw messaging apps in the same category as, for example, web browsers. It's really tough to try an alternate implementation if you're worried that they might have broken a complicated security feature. So having more shared crates mitigates some of that risk.
I'm a tad worried about getting rid of key-sharing. I seem to use this every once in a while. Pretty much any time I use a client I haven't used in a while. I guess I could just get the keys from the backup, but then I need to have THAT key, which is a bit inconvenient.
This is a great example of why key-sharing exists today... and why it shouldn't. In theory, you should be able to pick up and old client and decrypt new messages and history on it absolutely fine. However, in practice, Matrix's E2EE publishes 100 one-time keys (OTKs) on your server to let other devices establish secure 1:1 channels with you if you're offline - and if you go offline and that pool of OTKs exhausts, then new sessions won't get set up and you won't receive keys for new messages... giving the misbehaviour you're seeing.
Now, you're right that key-sharing is a useful way to fudge around that failure mode.
But an even better way to fix it would be to find a way to stop the OTK pool exhausting - and that's precisely what MSC2732 is: https://github.com/uhoreg/matrix-doc/blob/fallback_keys/prop.... This provides a last-ditch key which can be used to set up 1:1 sessions even if you run out of OTKs, which is marginally inferior to using a different OTK every time, but in practice really isn't a disaster (see the MSC for details).
However, fallback keys are relatively new and aren't implemented on all clients yet (matrix-js-sdk has them, but matrix-ios-sdk is implementing this coincidentally this week)... and so until they land, we still need keyshare requests to paper over this limitation.
But in future, hopefully it will be almost unheard-of to need a keyshare request, and we can change them to be an entirely manual or out-of-hand mechanism of some kind, and avoid classes of bugs like the vuln in question here in future.
> In theory, you should be able to pick up and old client and decrypt new messages and history on it absolutely fine.
No? Assume the lost/stolen/forgotten device has outdated (or no) protection around its keys - and it should be rotated out as soon as possible?
I'd rather not have to worry about if my old phone is in my drawer at home, or in an mi6 office (or more likely, with an estranged spouse or a journalist)?
if you think the device has been lost then you should obviously kick it off your account immediately, and it'll be immediately rotated out of all the conversations it was in.
otherwise, do you really want to rely on the OTK pool exhausting as a very handwavey way to provide post-compromise security? A much better solution would be to have a firmer metric for your server to explicitly kick out your devices after N days of inactivity. Relying on some weird bug like OTK pool exhaustion for security purposes is horrible.
However, my point still stands that fallback keys are a good idea - you should kick the device out deliberately (manually or with a timer) rather than rely on a protocol quirk meaning that E2EE randomly breaks after some period of inactivity(!)
Each OTK is a Curve25519 key (or ed25519 key, i forget) - so 32 bytes. So 100 takes up 3.2KB of storage on each client for the private keys, and the same on the server for the public keys, so it's not a big concern :) We could do bigger pools, but it's just punting the problem.
100 seems like a reasonable middle-ground between "that's a lot of storage to address a rare edge case" (e.g. say this was 3.2TB) and "most people will never need 10".
Any recommendation to get started with matrix? Like use cases, featured clients, interesting public chats, or similar? To someone whose close environment does not use it
From there, what's the first place to find people to chat? I don't have a good answer there, but the 'spaces' functionality is a nice way to group communities.
I started by setting up a synapse server (with docker it is quite easy, although reading all the config options takes some time). I already had a server with docker services running there, so deploying an extra container was easy for me.
I started trying it, with the Element.io apps and convinced a techy friend to try it. Since then, he set up his own server and we have a group of friends that we mainly communicate through matrix. Also, I try to use it with my parents. They want to use it, but end up writing to me through WhatsApp out of muscle memory.
After some time I started looking at public rooms that I'd be interested into, available in matrix.org or matrix.feneas.org.
The easiest way to start, though, is to create a free account in the matrix.org public server using the webapp[0].
As for interesting public chats: Quite a lot of the free software community is on Matrix these days; check out projects you're interested in and see if they have a Matrix link.
For example, the NixOS community has a curated Matrix Space linked from https://nixos.org/community/index.html with lots of rooms to explore. GNOME, KDE, and Mozilla also have a significant presence on Matrix.
If you're looking to get started using Matrix, but have no contacts to bootstrap it with, you can look into bridges to other networks or using something like beeper.com with hosted, well-supported bridges.
What are people using for a self-hosted "slack" type messenger these days? I have looked at Matrix (which I am interested in but I think needs time to settle right now), Zulip (I really like their threading model but it has been confusing for other people I have tried it with), Rocket chat (only recently aware of this one). What else is there? Anyone have good/bad experiences with any of these?
I hear this on occasion (though more rarer nowadays than in the past)...But curious, is there something specific that you're finding in matrix that is not yet settled? By all means, the beauty of open source stuff is that you get to choose whether you (and your circle of contacts) uses zulip, mattermost, etc...So, i encourage you to use whatever open source offering gives you (and your circle of contacts) what is needed/desired. But with me being such a fanboy of matrix, i always get curious. Would you care to share specifics?
That's basically it. A lot of options but they all have some flaw. Rocket Chat and mattermost both lock features behind an enterprise paywall (although Rocket Chat is more forgiving and has optional E2EE), Zulip is nice but still a small player and some people aren't fans of the UX, and Matrix lacks the polish of slack and moderation tools are not up to speed with Slack/RC/Mattermost, which is a problem for larger groups.
Yeah, I have good experiences with Mattermost (bundled with GitLab) as chat for a hackerspace/makerspace. As long as you're OK with unencrypted push messages, building your own version of the app and keeping it up to date is horrible. (React native levels of horrible.)
> Matrix supports the concept of “key sharing”, letting a Matrix client which lacks the keys to decrypt a message request those keys from that user's other devices or the original sender's device.
I'm a bit curious why you'd ever need to request those keys from somebody else, in addition to your own devices (or your homeserver, where they'd be stored encrypted by a key known by your own devices).
The recommendation is that devices only accept requests to share (forward) keys from a) other devices belonging to that user, b) the device which originally send the message.
It's acceptable to gossip keys between your own devices (assuming you authenticate them correctly!) given... they're all your own devices.
It's also acceptable to request the sender to re-send the keys to you, given the sender knows for sure whether you were allowed to receive the key (given it made that choice in the first place).
It's not acceptable to request keys from other devices in the room, as they won't necessarily know whether you were actually allowed to receive the message key you're requesting - only the sender knows for sure whether it should have been sent to you; it was their key after all. Plus it would risk make vulnerabilities like the one in question here even worse, given anyone could try to exfiltrate messages from anyone else, rather than "just" the sender (as was the case here).
Imagine you're a "normal" person who has a phone and doesn't really use a computer. You lose your phone and want to retrieve the history of a chat you had with your friend on the new phone. You ask your friend to send the key to your new device.
> As discussed above, key sharing was originally introduced to make E2EE more reliable while we were ironing out its many edge cases and failure modes.
Based on what I understand about this feature, I've had to rely on key sharing quite often (even requiring manually requesting keys a bunch of times) for E2EE to work reliably. I worry that my experience will be severely degraded if key sharing ends up being removed from the protocol.
Having the sending party re-encrypt messages for a new device require the other party to be online. I can see that requirement not being met in several ways, e.g. if the other party dies or uninstalls the application, or when the other end is arrested (I can imagine journalists losing access to activists' messages because of this).
> I've had to rely on key sharing quite often (even requiring manually requesting keys a bunch of times) for E2EE to work reliably.
This is because it's papering over edge cases in E2EE.
> Having the sending party re-encrypt messages for a new device require the other party to be online
We never make senders re-encrypt messages like this, and we never would.
Firstly, key-sharing from the sender does require the other party to be online already.
Otherwise, if you have the keys in an existing device, you could get them onto your new device by backing them up on the server - or using "dehydrated" devices; where the 'new' device you log into is actually one which is stored encrypted on the server, and "re-hydrated" into a new device when you login... and so already has your keys. These have security trade-offs obviously (what if your online backup gets pwned? what if your dehydrated device gets pwned?) but it's not obviously worse than exfiltrating the missing keys from the sender.
> trick vulnerable clients into disclosing encryption keys for messages previously sent by that client to user accounts later compromised by an attacker.
Doesn't the fact there's some kind of mechanism or expectation of sending encryption keys willy nilly imply a flaw with the protocol perhaps?
If "send key to all members of channel x" is enabled it would be good if that would work.
You can set it to "only other clients and people I have personally verified" but that is quite cumbersome for big channels that just don't want clear text.
