Hacker News new | past | comments | ask | show | jobs | submit login

Been digging into escape sequences lately and gave WezTerm a spin to test some things out

One thing that put a sour taste in my mouth for WezTerm is they document all escape sequences [0] using a deviation from standard syntax (sometimes using `:` over `;` like for RGB colors, at least I've not found any other documentation about this being commonly adopted). They do put a disclaimer at one point in the document but that only helps if you read it linearly and it feels weird to document your implementation of a standard to only work with your terminal, locking people's applications into it.

As for rendering, I had problems with what look like correct escape sequences not rendering correctly (alternative underline styles, colored underlines). I still need to dig further into this where in the stack the problem is (my application or where in WezTerm)

[0] https://wezfurlong.org/wezterm/escape-sequences.html




Colon is the correct separator there. Semicolon separates different instructions, a group of values that is a single instruction is colon separated. For example \e[38:5:196;1m specifies red and bold.

Using ";" is popular, and some parsers even require it, but it makes little sense. A terminal that doesn't know about 38 (anything old) will interpret that 5 as blink if you use ";", which is not what anyone wants.


From my experience, the colon separator for 256/rgb attributes is a fairly recent clarification to the spec. I think that's the way it was always supposed to be. But when I was initially writing the 256 color support for the Windows Console about 6 years ago, `TERM=xterm-256color` still emitted colors with just the semicolons as a separator.

The colons are more correct, but the semicolons are seemingly more broadly supported.


I don't recall where I saw it, but my understanding was that : was in the original spec but one of the early implementors (Konsole?) misread it and used ; and that erroneous form is what took off, and here we are today.


I'm guessing it may have been one or both of these `xterm`-related documents:

* <https://invisible-island.net/xterm/ctlseqs/ctlseqs.html#h4-F...>

* <https://invisible-island.net/xterm/xterm.faq.html#color_by_n...>

More details in my other comment: https://news.ycombinator.com/item?id=35157058


Part of the problem is probably that termcap uses : as a separator between capabilities. There is \C for colon, but there are implementations that don't support it. If nothing else I know the termcap[info] command in screen doesn't.

That doesn't stop terminals from supporting it of course, just from saying they do.


> Colon is the correct separator there

Where would I find this documented and how broadly is it supported? So far, the most exhaustive documentation I've found is wikipedia [0], vt100 [2], and wezterm with only wezterm talking about this. I also have not seen `:` handled in the various parsers (e.g. `vte` for alacritty) or generators (e.g. `termcolor`) I've looked at.

[0] https://en.wikipedia.org/wiki/ANSI_escape_code

[1] https://vt100.net/emu/dec_ansi_parser


Okay, it took a little bit of digging but I eventually found more about this via a link from the Wikipedia page.

* via <https://invisible-island.net/xterm/ctlseqs/ctlseqs.html#h4-F...> (search for "CSI Pm m Character Attributes (SGR)"):

    If 88- or 256-color support is compiled, the following apply:
          [snip]
          o   The 88- and 256-color support uses subparameters described
              in ISO-8613-6 for indexed color.  ISO-8613-6 also mentions
              direct color, using a similar scheme.  xterm supports
              that, too.
          o   xterm allows either colons (standard) or semicolons
              (legacy) to separate the subparameters (but after the
              first colon, colons must be used).

          The indexed- and direct-color features are summarized in the
          FAQ, which explains why semicolon is accepted as a
          subparameter delimiter:

            Can I set a color by its number?
(This links to <https://invisible-island.net/xterm/xterm.faq.html#color_by_n...>.)

The `ctlseqs` document continues:

          These ISO-8613-6 controls (marked in ECMA-48 5th edition as
          "reserved for future standardization") are supported by xterm:

            [snip example with colons ":"]

          This variation on ISO-8613-6 is supported for compatibility
          with KDE konsole:

            [snip example with semicolons ";"]
So, the confusion between use of ";" and ":" separators is because the RGB values are not actually parameters (specified as separated by ";"), rather, the RGB values are subparameters (specified as separated by ":"). [Edit #1]

This is covered in more detail by <https://invisible-island.net/xterm/xterm.faq.html#color_by_n...> (mentioned above):

    We used semicolon (like other SGR parameters) for separating the R/G/B
    values in the escape sequence, since a copy of ITU T.416 (ISO-8613-6)
    which presumably clarified the use of colon for this feature was costly.

    Using semicolon was incorrect because some applications could expect their
    parameters to be order-independent. As used for the R/G/B values, that was
    order-dependent. The relevant information, by the way, is part of ECMA-48
    (not ITU T.416, as mentioned in Why only 16 (or 256) colors?). Quoting from
    section 5.4.2 of ECMA-48, page 12, and adding emphasis (not in the
    standard):

      [snip]

    Of course you will immediately recognize that 03/10 is ASCII colon,
    and that ISO 8613-6 necessarily refers to the encoding in a parameter
    sub-string. Or perhaps you will not.
:D

      [snip]

    Later, in 2012 (patch #282), I extended the parser to accommodate the
    corrected syntax. The original remains, simply because of its widespread
    use. As before, it took a few years for other terminal developers to
    notice and start incorporating the improvement. As of March 2016, not
    all had finished noticing.

      [snip]

     * Going forward (e.g., xterm patch #357), these terminfo building blocks
       are used in ncurses:

        * xterm+256color2 is the building-block for standard terminals
        * xterm+256color is a building-block for nonstandard terminals
It's unfortunately seemingly not possible to directly link to the specific text but hopefully that's enough pointers for people to find it themselves.

Once again, seems it's amazing that anything works anywhere. :D

[Edit #1: Corrected from subparameter separator from "," (which is correct for one of the specs I read but incorrect here) to ":".]

[Edit #2: Typo fix.]


For the sake of completeness, a couple of updates[0]:

(1) With regard to the "03/10 is ASCII colon" comment: the "03/10" form is apparently known as "column-row notation" (or "column-line notation", see https://en.wikipedia.org/wiki/ISO/IEC_2022#Notation_and_nome...) and refers to the position of the character in an ASCII table such as: https://en.wikipedia.org/wiki/File:USASCII_code_chart.png

I actually found a reference (https://dicom.nema.org/medical/dicom/2019d/output/html/part0...) that explains the notation: "the [US ASCII] value can be calculated as (column * 16) + row, e.g., 01/11 corresponds to the value 27 (1BH) [i.e. 0x1B]."

It then notes "The column/row notation is used only within Section 6.1 to simplify any cross referencing with applicable ISO standards." :D

A (IMO less effective) description can also be found in ECMA-35 (6th edition; section 5.2 "Code Tables").

(2) A nuance with regard to my "the confusion between use of ';' and ':' separators is because the RGB values are not actually parameters" remark:

The ECMA-48 (5th Edition; section 5.4.2) specification (& ISO/IEC 8613-6 / ITU-T Rec. T.416) talks about "parameter string" & "parameter sub-string" rather than "sub-parameter".

(Also, the "reserved" values mentioned apparently weren't ever actually specified in a standard; and--as noted elsewhere--the format of the ANSI parameter(s) don't exactly match the ODA (ISO/IEC 8613-6 / ITU-T Rec. T.416) format anyway--see ITU-T Rec. T.416 (1993 E) page 41.)

(3) The Wikipedia talk page on ANSI escape codes makes...interesting...reading: https://en.wikipedia.org/wiki/Talk:ANSI_escape_code#Clarific...

[0] i.e. let me inflict this "knowledge" on you. :D

[n] The best explanation for how we ended up in this mess is, I think, found in "Figure 6 - Structure of 8-bit codes" of ECMA-35 (6th Edition). :D


Thanks!




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: