Matrix isn't peer-to-peer, it's federated. Messages are stored on "homeservers", which is similar in concept to an email server except it's much easier to host your own Matrix homeserver. Only the homeservers which have involved in a chatroom ever see a copy of the message. For private chatrooms, everything is E2E encrypted by default so the homeservers don't have access to the plaintext of your messages.
