The huge underlying assumption here is that people would use XHTML as a standard instead of using HTML4.01 with browser-specific extensions, which is what actually happened. XHTML didn't add any value to page authors. It added hugely indirect value to readers. The only people XHTML helped were tooling authors and browser vendors. It's very hard to market that as a value-add.
