I see where we have our wires crossed. I'm not considering the input problem because my software (KeenQuotes) parses the source document's apostrophes into their correct semantics (99.9% of the time).
My issue is that, having discerned the correct English single quotation mark (straight, apostrophe, or closing), I have no way of encoding it into a document that retains the semantics while typesetting it using common fonts (to match double quotes). My point is that if UNICODE had a way of capturing the semantics, it would at least be technically possible to create unambiguous documents, input notwithstanding.
Matthew Butterick, a typographer, states, "I’ve never seen any LaTeX-created documentation that’s gotten this right":
I sent him a screenshot showing my software typesetting the quotation marks properly, albeit with a document that has incorrect semantics (as per our discussion):
I totally appreciate the desire for more semantic encoding. I mean, it would be a dream if every sentence was semantically delimited, if every word was annotated with which hyphenation pattern it should follow for splitting across lines (when there are multiple), and whether the capital letter at the start of a sentence should remain capital even when converted to lowercase, because it's a capitalized proper noun. I could go on.
But that's not what Unicode is for. The apostrophe situation is just one of 100 things I could think of off the top of my head. Unicode encodes characters, not semantics. And this is by design, because people don't input, or want to input, semantics -- they just want to type something that looks right. Something other people can read, not something computers can semantically parse.
So we have a bunch of heuristic and AI and manual tools we use to try to annotate things semantically, and we put that information at the level of something like XML, not Unicode. Which is infinitely more flexible, because you can define and use whatever semantics you want, not limited to whatever the Unicode body decided.
If KeenQuotes gets apostrophes right 99.9% of the time, then just use that to automatically analyze all your input text and then store and process it in some kind of XML notation, like "Peter<apos>’</apos>s" or "<possessive>Peter’s</possessive>" or "<word>Peter’s</word>" or something. Unicode is the wrong level of abstraction.
The output from KeenQuotes is used by KeenWrite. KeenWrite can generate text, HTML, XHTML, and PDF documents. Those output document formats lack correct the semantics because of UNICODE. As much as rolling my own XML notation would be fun, it won't work in practice---nobody would be able to publish their exported documents for viewing or general consumption. We'll have to agree to disagree on this one: I think UNICODE dropped the ball on English apostrophes where it didn't have to. Having one more character for curled apostrophes would have kept open the possibility of encoding unambiguous HTML documents (with respect to apostrophes/right single quotes for quotations; your point about other characters I quite appreciate).
I see where we have our wires crossed. I'm not considering the input problem because my software (KeenQuotes) parses the source document's apostrophes into their correct semantics (99.9% of the time).
My issue is that, having discerned the correct English single quotation mark (straight, apostrophe, or closing), I have no way of encoding it into a document that retains the semantics while typesetting it using common fonts (to match double quotes). My point is that if UNICODE had a way of capturing the semantics, it would at least be technically possible to create unambiguous documents, input notwithstanding.
Matthew Butterick, a typographer, states, "I’ve never seen any LaTeX-created documentation that’s gotten this right":
https://practicaltypography.com/straight-and-curly-quotes.ht...
I sent him a screenshot showing my software typesetting the quotation marks properly, albeit with a document that has incorrect semantics (as per our discussion):
https://i.ibb.co/p3TM7QM/curly-quotes.png