It would work like it works for email. Most users will pick the default server. Over time other large (popular) providers will emerge and people will start to recognize the choice. Clients with branding will make that choice easy for non-technical users.
We know already it works. Federation and choice is a win for everybody. For this to start though you need either a good client/server combo for people to individually adopt, or a GREAT/POPULAR client with a default server.
I'm sorry to say that Element/Riot fails flat on this. I've used Riot and then Element for a while, and it got worse in usability and features (I genuinely can't stand it). As the reference client, it doesn't support multiple server accounts, which kill the ability to have an independent server. Federation is nice, but what's nicer currently is to be able to use matrix a teams/slack replacement where tech users can drive adoption. The central matrix.org server is slow already, and even with federation you cannot tell your users to switch servers because of this, defeating the point.
The positive side is that on the client side there's already choice. On the user front I've found FluffyChat to be vastly superior. Using Jitsi for audio/video was weird at first, but it works much more reliably than signal (I gave up on signal for audio calls due to the time it takes to establish a connection and the horrible lag it randomly has - to the point I just hang up and call via GSM to finish the conversation, defeating the point).
If I was element I'd get FluffyChat and it's author onboard as _the_ default client. On desktop I can choose multiple clients, and I have none of the "Signal" phone number/pin/central server/electron BS. It works as intended, and it will only get better with time. It's great.
I keep seeing this argument, and we all know it's bs. 20 years ago there were many email providers, people simply picked one, or hired someone to set it up and run it for them.
The same principles should and could apply for decentralized one; pick conversations.im for XMPP, pick matrix.org for Matrix, pick Mastodon.social for fediverse - or something smaller. Or ask someone to do it for you. Or run your own.
It was, actually, when gtalk, facebook, and whatsapp (thought I'm not completely sure about this, just have a reasonable suspicion) was using it in the background.
Probably like email or all the other decentralized services the federated people talk about. Email didn't get mass adoption until centralized servers like Hotmail and Yahoo came online. People don't want to run their own servers and now a days it's a lot more work to do so.
As a "federated person", yes, an email sort of scenario is exactly what I'd like, or at least it would be many times better than what we currently have for im systems. The most important thing to me (and I suspect many other people in favor of decentralization/federation) is the ability to host my own service. I couldn't care less if 90% of email users are on gmail as long as I can run own email server or pay someone else to run one for me while still communicating with everyone.
