One small note: the post seems slightly confused about the use of OpenType features, as it calls for:
font-feature-settings: "colr", "calt";
but there's no 'colr' feature tag in the font's OpenType layout tables, so that's meaningless here.
I suppose this was intended to "activate" the color glyph table (COLR) in the font; but that isn't an OpenType layout feature that would be controlled by font-feature-settings, and doesn't need to be "turned on" like this.
(In addition, the 'calt' feature is (according to the spec[1]) supposed to be active by default, so it shouldn't be necessary to explicitly set it either. And indeed, the font works for me in both Firefox and Chrome without this rule; sadly Safari doesn't seem to handle it.)
Interestingly it also seems to work fine on 16.5. Browserstack doesn't have 17.1, perhaps it was broken temporarily? Unless there's some other difference with the version of Browserstack somehow..
This is a horrible, horrible hack and it's terrifying and I am aghast at how amazing this is. Such a wonderful abuse of OpenType contextual alternatives. Well done and thank you for sharing!!
What makes it a horrible hack, as opposed to proper use of documented capabilities at a scale slightly larger than usual?
Putting animated SVG in a font [1] is a horrible hack; this one is benign.
A somehow larger stretch would be a font with a built-in spell checker, with a wavy underline under every letter that only disappears under correctly spelled words.
"Putting animated SVG in a font [1] is a horrible hack"
Agreed, that is even more horrible. And probably just for performance reason alone should never ever get close to a real world product. Still, quite neat.
I think the author has underestimated the power of the substitution rules - it looks like it should be possible to drag a state machine along, hence recognise any regular language.
Something like the following should handle quoted strings (untested and needs every character to be listed out in the definitions of @Char, @CharQuoted and @NonQuoteQuoted):
sub Quote @Char' by @CharQuoted;
sub Backslash.quoted Backslash' by Backslash.quoted2;
sub Backslash.quoted Quote' by Quote.escaped;
sub [@NonQuoteQuoted Quote.escaped Backslash.quoted2] @Char' by @CharQuoted;
I don't know if context can be carried between different lines.
Here's a tested version that, given text containing only lower case letters, backslash and double quote, capitalizes the letters between quotes and handles escaping reasonably. (works with NotoSans-Regular.ttf)
@Char = [a-z quotedbl backslash];
@CharQuoted = [A-Z quotesingle slash];
@NonQuoteQuoted = [A-Z slash];
@Escaped = [asterisk dollar];
feature liga {
lookup xa {
sub quotedbl @Char' by @CharQuoted;
sub slash backslash' by dollar;
sub slash quotedbl' by asterisk;
sub [@NonQuoteQuoted asterisk dollar] @Char' by @CharQuoted;
} xa;
} liga;
feature liga {
lookup xb {
sub quotesingle' by quotedbl;
sub slash' by backslash;
sub dollar' by backslash;
sub asterisk' by quotedbl;
} xb;
} liga;
although that would also change "snāke" to "snāk", which is not quite what you asked for.
I don't think there's a way to do many-to-many substitution in general, although it would always be possible if you create intermediate glyphs. I believe Section 5 of http://adobe-type-tools.github.io/afdko/OpenTypeFeatureFileS... gives the full list of substitution types.
Impossible before. It can't be less invasive: _original text stays intact_ - no wrapping tags.. no JS.. just works with userContent.css.. - simply another dimension.
> I'm also not an OpenType expert, so I'm sure the substitution logics could be improved upon. I'm open to sharing the modified source file to anyone interested. If you have any ideas, suggestions or feedback, let me know. You can reach me at hlotvonen@gmail.com.
- so, how far can it be improved then ?!
- what other font editors moreover to Glyph (mac only) have good support for advanced contextual alternates ?
Thank you! (I'm the author)
I'm also very curious to know if there's some nifty way of improving the lookup logic. What I did was kind of a brute force method, but on the other hand, the CALT "language" is very limited.
For font editors, Glyphs is the industry standard, and, as far as I know, there are not many good alternatives. There's FontForge, but its contextual alternate editing seems even more confusing: https://fontforge.org/docs/ui/dialogs/contextchain.html
Just today I found out about a new browser based font editor, fontra, but it looks like editing OpenType features is still on its roadmap. Maybe something to keep an eye on though. https://fontra.xyz/
I wonder, if looking at the actual diffs of the before/after font it wouldn't be possible to write a compiler of sorts, taking a grammar, a font and a color scheme - outputting a custom font with highlighting for the grammar?
Perhaps especially if the sibling comment about embedding a state machine pans out?
I have been considering using php like that. It would solve most of my issues. The problem is that I would still like to use static hosts, like netlify or GitHub pages, and they don't support php...In addition I would have to run php on my local machine, where as I would prefer to just edit html with no setup necessary. But if anyone knows a good free host with php support, let me know!
Your solution is quite nice and probably as simple as site generators get, but I was thinking of using php just for some simple includes etc. Here's my inspiration post which explains what I'm after, and why generators in general are not the solution (for me): http://ankarstrom.se/~john/articles/html2/
If you are willing to use netlify and are willing to use PHP, what is the inhibition to use a static site generator like Hugo? It is supported by netlify and others. I believe GitHub pages also supports a bunch of static site generators.
I am using 11ty for the blog, and it works great, for now. But there's two reasons I don't like it:
1. I need to have two different versions of my site, dev and prod, which I find very restricting and fragile.
2. I don't like to rely on other people updating their npm packages just so I could publish static content. I've been burned too many times by abandoned npm packages or breaking changes.
Fascinating. TIL about `override-colors` in CSS. Now if we can automate the generation of color alternates and run it through a bunch of TextMate grammars to generate a font, this could be even more awesome.
> what other font editors moreover to Glyph (mac only) have good support for advanced contextual alternates ?
any font editor that supports writing opentype feature code manually. glyphs for mac won’t really help you here: while glyphs will do its best to autogenerate as much opentype for you as it can, it doesn’t do much (anything?) for calt features
Really a minor nitpick, but "It's as fast as plain text, because it is plain text" is definitely not true. Evaluating all these rules when shaping the text is not gonna be cheap, and if you overuse this type of font your end users' CPUs will probably be screaming. As would anyone who's implemented parts of the OpenType spec from scratch before. Worse still, this doesn't just impact rendering, since the output of shaping determines how big elements in the document are, so things like scrolling and window resizing and text selection can all lag.
Really fun to see a font that can do this though. Never would've expected it.
You are probably right, but I would like to see some actual benchmarks and tests. Unfortunately I didn't have time to do them myself yet, but I am curious what the difference between this, a basic font, and a highlighter script would be for a very large document.
It's definitely going to be significantly slower as a result of all the contextual lookups.
To get an idea, I repeatedly duplicated the CSS in the "tiny sandbox" until there were nearly 20,000 lines there, and then profiled the operation of changing the font-size by 1px in Firefox. Reflowing the textbox took about 1.6 seconds.
With `font-feature-settings: "calt" off` applied, so that the contextual lookups aren't being run, a font-size change for that same textbox content reflows in about 100 milliseconds.
(I don't have actual timings for the same example in Chrome, but subjectively, with "calt" disabled, the resize is near-instant, whereas with it enabled, it's quite laggy, similar to Firefox.)
In Firefox, 4000 lines of CSS: editing last lines is fast and instant, editing first lines is a bit jumpy refreshing every few edits in 0.2s-1.2s - however undo/redo is quite fast - but not so fast as with last lines, and.. from the first impression, guesses:
-= the highlight is always there, instantly =- (or.. it doesn't a look like a reason for jumpy lines because of redo) - definintly less cycles and memory overhead and I don't see that possible with DOM and JS - the delays could be there only because of inefficient editing operations with no impact of coloring.. and maybe with.. smaller textfragments (or ?) there could be no such delays at all - but simple* coloring coud be always so instant (and easy) ? (yes, I know, so far it may be also limiting and exclusive)
I find it amusing that the motivation for creating a complex font program that supports syntax highlighting internally is the desire to avoid a complex syntax highlighter JavaScript library. The complexity is still there; it's just been moved around.
Edit: Perhaps this is a reminder that custom fonts are a potential attack vector for security-sensitive websites since font rendering runs highly-complex programs, probably in a language that isn't memory safe.
This makes me curious; have there ever been security exploits that utilized the font rendering as an actual attack vector? To me it feels like font rendering should be pure (in the functional sense) and thus have no side-effects, but of course that doesn't mean anything in practice.
I think it is more that it doesn’t load any resources automatically from external sites, a common way to share visitor information with third parties without JavaScript. For me it’s a sensible way to earn trust .
There are many third party links that you can click on, which does align with the original ethos of the web.
It does, right? I imagine you would be map syntax highlighting rules and colour themes over to this convention in order to generate a new set of glyphs. I'm sure it isn't trivial but it could likely remove or at least alleviate one of the cons listed in the article.
This is one of the cooler things I've seen in a while!
Definitely! For example, Glyphs has a decent python scripting API which could be used to automate the whole process. I used it for automatically assigning each layer the correct color attribute to save me some clicks. And I also generated all the calt lookup rules from a list of keywords.
I wonder if you can load a font from base64 string like you can with images. Then, one may write a pure js library which generates such font from a set of rules on the fly. Catered to a specific use case.
This is actually really cool. I can see the usages in textarea and inputs but also even in plain HTML documents for large code formatting. No need for exploding the DOM with tons of spans and other HTML tags, if you can have the hardware accelerated font rendering software take care of everything. I’d imagine that would save lots of memory and CPU.
I imagine you'd want to have something that works like a parser generator up front to create the rules. What's not clear to me however is how expressive the rules are.
And about the lookup rules: they are extremely simple. You can search for a single character at a time, or search from a group ("class") of characters. AFAIK, that's it. But if anyone knows better, I would be happy to be proven wrong.
Yeah. Doing it all manually would take ages and be quite error prone. I did generate all the lookup rules with a script, just didn't think it important enough to mention in the blog post.
Great hack. Note that it's approx the same size as baseline highlight.js at ~45kB. However it has the great benefit of functioning in contexts where javascript is disabled, at the cost of runtime configurability and narrower language support.
There seems to be a bug in Chromium-based browsers where, if you type e.g. `color: blue` in the textarea, it only highlights the r before the colon, rather than the whole property name. If you copy+paste the text instead, it works properly. (Still, that doesn't detract from this being really cool, of course!)
at the end of the example, some of the letters of "div" would be green and some would be white.[1] Adding a class or something changed which letters were green and which white.
Just converted MonaspaceKrypton-SyntaxHighlighter-Regular.woff2 with FontCreator to .otf (or .ttf) and installed it in the system,
then in Firefox changed about:preferences -> fonts -> monospace to Monaspace Krypton, unchecked Allow pages to choose their own fonts .. - and now I have:
*every* monospace text in the browser *syntax colored * :)
(as expected.. wherever it does make sense or not.. - viewsource, text files, parts of pages.. :)
Would this work if the font was used in a PDF typeset using LaTeX? (EDIT: when running on top of a TeX engine which can access OpenType, or some other tool (such as InDesign)?)
Sometimes you end up with code in one language that has string literals which contain code in another language.
It would be so cool if you could override the contextual language (likely determined by file extension or explicitly in markdown or whatever) with a different one just by applying a font.
How is this syntax highlighting? If you put "<div>" in quotes as string, it's still green instead of white. Proper syntax highlighting must account for both languages and content in context, which can't be done with just a bunch of simple static rules.
You could compile an actual syntax highlighter to WebAssembly and embed that into the font. That would address all the issues. However, WebAssembly is an off-by-default experimental feature for harfbuzz, so at least for now the current solution is the only feasible one if you want it to work in most places.
Another issue you'd run into, I suspect, is that in practice text engines/editors/etc will often repaint only fragments of the content, and so the syntax highlighter would not necessarily see all the context it needs.
E.g. if you have a comment that starts with "/*" and ends several lines later with "*/", but the editor decides to just redraw a line in the middle, the highlighter won't "know" to use the comment color as the text-drawing operation won't include the delimiters.
(This issue already exists in a small way, as mentioned in https://news.ycombinator.com/item?id=41251114, but it'd only get worse as the syntax highlighter becomes more sophisticated.)
it seems like you should be able to use the same pattern you do for searching for keywords to handle string sections and comments, by assigning a .instring or .incomment state and having all characters check the prior character for .instring or .incomment and replacing itself with its .instring or .incomment variant as appropriate
(author) The problem is that the opentype "language" doesn't have loops or anything resembling regular expressions, where I could just tell it to sub everything between two characters. So, I could highlight text between quotes, but only up to a certain arbitary limit, so I chose not to implement that at all.
Or did I misunderstand your comment? Care to elaborate on your idea?
I'm not the person you're replying to, but it looks like it's possible to carry around a finite state machine, hence get the power of regular expressions (with caveats). I made an example that should highlight quoted text: https://news.ycombinator.com/item?id=41254638
It doesn't check that there's a closing quote, but it should work for highlighting syntactically valid code (once you fix my mistakes). I don't see a way for a character to affect others arbitrarily far back unless some fancier features like ignoring certain classes can be abused.
(Author) For specific keywords, yes, technically you could, but it would require making a separate glyph for each word, which would inflate the file size and require a lot of manual effort for no apparent upside. The substituting logic is as rigid as with calts. Unless you had some other idea how to use ligatures?
(Author) Sure, it depends on your requirements, what you want your syntax highlighter to do. I wouldn't use it as my coding font (in its current form at least). But just because a few words inside regular text could be highlighted, doesn't make the code that much less readable — in the context of code snippets on websites.
It certainly heavily limits the usability, but there’s plenty of use cases where there’s not much free text. Comment highlighting would still be an issue though
Maybe this could also be implemented using variable fonts[0]. This would probably make it easier to configure in non-browser environments (once this is better supported by font selectors etc).
Imagine inputting syntax highlighted code while running inference w LLMs? Currently its hard to read what's in your text area when asking ChatGPT a code question.
This is the most interesting post in a long time, bravo!
This is incredibly awesome. Looking forward to seeing this but with a nicer typeface (maybe Inconsolata or JetBrains Mono). I can actually see a lot of uses for this-- just displaying JSON in a nice way easily would be handy.
BTW, I'm not sure why it says adding syntax highlighting to a <textarea> is impossible -- as the linked post says, you can turn the text invisible and add styled text in a separate div at the same position.
(author) Because then you're not adding syntax highlighting to the textarea, but to the div... I've done this before and it's janky, with many edge cases that require extra css fiddling. It's just not great. I would rather use a library like codemirror even in seemingly simple cases like that, but fortunately I don't have to anymore : - )
I wonder if the author has experience with Excel, because some of those substitution "monstrosities" look similar to some monstrosities I've created with Excel formulas! Good work, this is an awesome hack.
Love how crazy this is. Small nit that shouldn't take away from the enjoyment - one downside not mentioned is that brackets aren't colour-coded based on depth, something I find very useful in my IDE.
I really like the idea of the hack, but I couldn't bring myself to actually load the webpage - thinking of crazy font-code exfiltrating all of my emails that have ligatures.
The design of this blog post is lovely! Despite seemingly having no right to be. It goes against all of my design instincts, but it somehow nails a vibe while remaining pleasant and easy on the eyes. I love it!
I was thinking the same thing. The style reminds me of 90’s computer/game magazines or something… So it looks sort of professional (sans the first letter in the headings) but also whimsy at the same time. Love it.
As an experiment, it is awesome. We should strive to push the limits of technology and try new things. Kudos. I tip my hat to the creativity.
As something practical, the author started with the wrong tool for the job (JavaScript), and then used an even more wrong tool for the same job (The font itself!)
Just use a code editor with Syntax Highlighting! Vim solved this decades ago!
Very interesting read, but don't try this at home.
Edit: Pandora's box is opened, I can see Jira and GitHub rushing to add support for this on their websites starting tomorrow =)
- but the opposite: this one have such potential that you could possibly just use the font with language-specific syntax embeded and forget any other syntax highlighting (to some limits) on web (JS), console (in.. VIM) or wherever - as obsolete (in any program that did not have any syntax highlighting before - just use that font for that language and solved ) ;)
> Just use a code editor with Syntax Highlighting! Vim solved this decades ago!
You are aware that this blog post is about displaying syntax highlighted code on the web? This is literally explained in the first paragraphs of the blog post..
As of now there were four ways of doing that:
1. using an image (which sucks)
2. manually setting the colors (good luck)
3. Generating the syntax highlighting server-side (preferred, but you need a backend that can do it)
4. Generating it client side using JS (waste of energy, takes a time to finish)
And now a font. Depending on the use case this could be a feasible option, although I am just waiting for this to be exploited in some fashion.
I still think computing once styling later is preferable.
This is for publishing web pages talking about code.
The author would like to hand code HTML for such web pages that would present syntax highlighted code to the web page reader, without requiring or resorting to typical js highlighting or screenshots.
None of editing in the terminal, nor an IDE, nor a browser are related to this use case.
However, the highlighting examples do not work with JS disabled (I think because JS is used to load the fonts) so what is the advantage of not using JS for highlighting over calling a JS highlighter on <code> tags?
One small note: the post seems slightly confused about the use of OpenType features, as it calls for:
but there's no 'colr' feature tag in the font's OpenType layout tables, so that's meaningless here.I suppose this was intended to "activate" the color glyph table (COLR) in the font; but that isn't an OpenType layout feature that would be controlled by font-feature-settings, and doesn't need to be "turned on" like this.
(In addition, the 'calt' feature is (according to the spec[1]) supposed to be active by default, so it shouldn't be necessary to explicitly set it either. And indeed, the font works for me in both Firefox and Chrome without this rule; sadly Safari doesn't seem to handle it.)
[1] https://learn.microsoft.com/en-gb/typography/opentype/spec/f...