Speaking as someone who spent a long time working on Undernet trying to make IRC work. The RFC is poorly specified (eg ~ and ^ are missed from the case equivalent list). IRC scales poorly (the protocol relies heavily on global state). IRC doesn't do UTF-8, or in fact, any sane character set (see the case mapping mentioned above). Client's ignore the parsing rules (eg the : marking multiword last arguments) and kludge around them. Each network went off and did their own thing, fragmenting the protocol space, and then declared themselves as being the One True IRC Protocol. IRC puts a lot of trust in all the server admins on the network, making it difficult to federate, and so on.
IRC was a great protocol in the 1980's, it's been dead for a while, just there are no good replacements.
Yes, IRC has warts, I didn't mean to declare it as the end of all things.
The point I was trying to make is that IRC would be a more sane starting point than XMPP. Even despite all the shortcomings you mentioned and some more that you didn't. And even despite it being a strictly centralized design that would require more server-side changes than XMPP to turn it into a distributed system.
I'll lean out of the window and even claim you could make a distributed IRCd backwards compatible to existing IRC clients, as far as the core business of presence/state, chat and group-chat are concerned.