Signal is also the protocol, and there is only one official Signal client for each platform (and Moxie has been incredibly aggressive towards any alternative clients -- threatening trademark lawsuits in some cases, or just blocking the clients from their network). Matrix, on the other hand, is an open protocol which has many clients (of which Riot is the most popular).
This would be like calling all of email "GMail" or "Outlook". Sure, some people do say that -- but it's still inaccurate.
The protocol is or at least was called axolotl and wire.com also uses it, they do not require phone, had and have many more features like video chat, desktop client ... How does wire compare to matrix?
Matrix has most of the same features, but also supports groups much better than Signal-based protocols (a group-wide key is used an regenerated frequently, which amortises the cost of all the peer-to-peer messages required for Axolotl). It's also federated -- which means you can also self-host your own homeserver (very similar to email, but even easier). And Matrix has native support for bridges to other chat systems, allowing you to reduce siloing between chat platforms (you can talk to Slack, IRC, and Matrix users in a single Matrix room completely seamlessly). As far as I know, Wire doesn't do any of those things.
I also think the newest features Matrix has for key exchange (both the emoji-based verification and the new device cross-signing features) are objectively the most intuitive and easy-to-use when compared to any other E2EE chat system. Yes, even better than Signal's UX for key exchange.
Sure, but you can't communicate with someone using just a client. As I said, it would be like referring to email as "Outlook" or "GMail". To compare Signal and Matrix, you'd be comparing the entire ecosystem (both the clients and protocols).
This would be like calling all of email "GMail" or "Outlook". Sure, some people do say that -- but it's still inaccurate.