One of the great things about the web is the fact that I can expect anything I create today, to be relatively forwards compatible, and I can build things in a way so they're either backwards compatible or degrade gracefully.
The philosophies around entropy isn't really relevant here IMO. The web should continue to be evergreen.
Supporting legacy constructs forever can be hugely detrimental to progress though.
Something completely new would fasttrack innovation. It could leave out all the quirky workarounds and get advanced features built in or easily extensible. When adoption is high enough it could include a legacy box, to run good old HTML based content.
Have any attempts at this sort of thing ever been made?
I don't think this is really a good idea. HTTP and HTML aren't perfect, but they've been made into ubiquitous standards already used by everyone. In order to switch to a new standard, you'd need to have a pretty convincing way in which it was better. And I don't think you can ultimately outbalance what we have now. Not to mention that the only way this could probably actually happen would be through Google doing it, which would be disastrous for the open nature of the standards.
There's plenty of ways in which we could improve on the current setup. Everything could be so much easier, for one thing. Basic things like animation, drag & drop and responsive design all require libraries or extensive knowledge to get them right.
Something like websockets is not exactly accessible to a beginner, but there is a huge need for online content that supports live interaction.
You are right that it's not going to be easy to get everyone on board, but that should not be a reason to stop trying.
The philosophies around entropy isn't really relevant here IMO. The web should continue to be evergreen.