The problem is getting the standardized semantics. If XMPP taught me anything, it is that it doesn't matter how many specs you throw at a programmer, they're just going to bash on the program until it sort of works most of the time and release it. That's where your roster problems come from, it's where a not insignificant number of your transport problems come from too. (Though the transport protocol is one of the spottier bits of the protocol.) The core bits of XMPP are generally reasonably well specified and in my experience actually held up surprisingly well as I bent and spindled it a little bit. (Corporate customers don't "get" rosters, don't get that you can start with a blank roster and work your way up, so I added a module to build rosters based on grouping criteria specified by the user and driven by outside input. Well beyond the stock ejabberd shared rosters, but pretty custom to our environment. XMPP actually dealt with these semi-magical roster entries just fine, to my surprise.) Those specifications really matter and just sort of bashing some stuff out that's 90% correct most of the time isn't good enough when you're trying to communicate with so many different systems. XMPP actually managed to avoid a lot of problems that even the "professional" systems had, having learned from their experiences; AIM last I knew still had some encoding corner cases you wouldn't expect in a modern program, all of the protocols had major encoding growing pains, surprising versioning issues, all these little quirks inside them that you never noticed because you can paper over a lot when you control both the clients and the servers. Again, I know it's not perfect but in the space of "deployed IM protocols" it does not make a bad showing.
IRC clients are actually just as quirky, IRC just doesn't hang on to anywhere near as much state or you'd see it mangled, spindled, and mutilated too. (Also part of the reason it's not a replacement, real users want that state.) This is fine, too, I don't have a problem with IRC for what it is, but you can't just drop it in everywhere you see an XMPP server.
If you try to work IRC up to be a real, true XMPP replacement, you'll be complaining about how hard it sucks in no time. Too much suckage is in the problem space.
I doubt that.
The parts that actually somewhat work in XMPP would be fairly straightforward to add to IRC (mostly related to persistence and identity). From there the question is where you'd want to take it, not what idiocies XMPP fell for. I.e. the task would be to do it right, not to imitate a broken protocol.
Just compare http://www.ietf.org/rfc/rfc1459.txt to http://xmpp.org/protocols - where the latter isn't even the full story.
And then tell me with a straight face the complexity is "inherent to the problem". No. It's not.
IRC handles very similar problems to XMPP already (and then some that XMPP doesn't have) and the specification, in its entirety, is only 3643 lines long. Extending that for distributed, message-persisting operation would not bring it anywhere near the insanity of XMPP.
Naturally that's an academic exercise, nobody would actually re-shape IRC into an IM system that way. However, when cherry-picking concepts for a new protocol then IRC should be high on the list, and XMPP rather low.
I've actually written IRC bots mostly from the top of my head.
The protocol and semantics are really simple.
Type this into a console near you:
nc irc.freenode.org 6667
USER foo bar batz boo
PRIVMSG #testchannel hello world
PRIVMSG test345 hello self
I'm not sure what you mean by fractured. Like every protocol it has a few rough edges, but those are nowhere near the semantic nightmare that I witnessed when trying to dabble with XMPP (which admittedly was more than a year ago).
How long did it take you bring your client into a reliably working state? And have you tried to do the same with a XMPP client for comparison?
As said, I didn't mean to claim IRC is perfect - nothing is.
But if you think the differences that IRC networks have introduced are problematic then I invite you to try and build a most basic jabber client.
However, I think it's a big mistake to claim IRC "scales amazingly well". The biggest IRC network today has tens of thousands of users (at the moment, freenode has 64000, undernet has 58000, and EFNet is down in the 32000 range) and the IRC networks are constantly suffering from breakdowns from overcapacity. Compare this to Skype, Facebook, or Gmail, with tens of millions of concurrent users.
IRC was a great protocol in the 1980's, it's been dead for a while, just there are no good replacements.
The point I was trying to make is that IRC would be a more sane starting point than XMPP. Even despite all the shortcomings you mentioned and some more that you didn't. And even despite it being a strictly centralized design that would require more server-side changes than XMPP to turn it into a distributed system.
I'll lean out of the window and even claim you could make a distributed IRCd backwards compatible to existing IRC clients, as far as the core business of presence/state, chat and group-chat are concerned.