6 ms·

I love this kind of stuff (ASCII art, if you will), but it just breaks down too easily as soon as Unicode characters (mainly CJK, as I'm Chinese) and fonts are

I love this kind of stuff (ASCII art, if you will), but it just breaks down too easily as soon as Unicode characters (mainly CJK, as I'm Chinese) and fonts are involved. For example, on your website, any chart or plot involving horizontal arrows breaks down because the assigned font-family (`ui-monospace, SFMono-Regular, Menlo, Consolas, monospace`, which ends up as Consolas on my machine) has no such glyph. Thus, it falls back to Segoe UI Symbol, which does not have the same fixed width (or is not fixed-width at all) as other characters: https://i.imgur.com/d2DPGHE.png https://i.imgur.com/d2DPGHE.png

uhoh-itsmaciek 2mo ago
I ran into this problem recently on one of our blog posts: we used some Claude output which included tables drawn with Unicode line drawing characters. However, our monospace font did not include these characters, and so rendering fell back to another font in our font stack with different width metrics. I fixed it by using a font that had similar metrics and did include those characters with `unicode-range` (to only select characters we needed) and `size-adjust` (to match font width more exactly), and adding it to the stack. It's a little hacky but works pretty well in practice.
- dolmen 2mo ago
  Claude Code with Opus 4.8 is also bad at aligning boxes with content in French (with accentuated letters such as "é" which are multibyte in UTF-8).
pmarreck 2mo ago
I was going to say, perhaps generate a failing test case, but testing for proper unicode rendering might be tricky??
biztos 2mo ago
Interesting. Thai characters can also blow it out, I imagine because of the difficulty mapping glyphs to width: https://biztos.com/hey/thai-mermaid-chart.png https://biztos.com/hey/thai-mermaid-chart.png To my surprise, Sublime Text gets it almost right: https://biztos.com/hey/sublime-thai-mermaid.png https://biztos.com/hey/sublime-thai-mermaid.png I tried finding a Thai monospace font and using that in the HTML but it was worse, probably didn't have the box drawing chars. Still a fun tool and useful for lots of ASCII cases!
- sirn 2mo ago
  The first issue is due to the assumption that character count equals character display width. Thai tone markers usually[1] should not contribute to the display width (เพื่อน is chars = 6, width = 4), so it caused a layout shift. The second issue is due to the program's layout engine not adjusting the glyph width of a fallback font to that of the main font. A lot of terminals do this, but it's not common for text editors or browsers (arguably this is the correct behavior for non-terminals, since you cannot assume everything must be snapped to a grid). Fun test for this: |กล้วยหอม| |Bananas| This has the same character width. Ghostty, etc., will render it correctly (| aligned). Most browsers and text editors will not. [1]: some layout engines render free-standing tone markers as 1 character; in that case, this rule only applies to when tone markers are following a character.
  - oneeyedpigeon 2mo ago
    That test looks pixel perfect in Chrome Android.
    - codethief 2mo ago
      Same in Firefox for Android!
  - biztos 2mo ago
    Thanks for the explanation. Safari on iPad lines these up almost perfectly - the second line is a tiny bit wider, I didn’t even notice it at first. That example had a tone mark but no vowels, so I will try one with both. E&OE. |ดื่มน้ำอยู่| |abcdef| [edit] These are even closer, but still imperfectly aligned on my iPad.
    - numpad0 2mo ago
      This isn't a renderer bug, it's font. Proportional fonts aren't designed with alignment in mind at all, and you can't just expect monospace fonts for all languages outside of ASCII range to be present on random systems, or a single font or font family to support multiple different languages consistently. You can't really control alignment of deeply Unicode characters like Thai or "→" against monospace characters without serving your own monospace fonts that are guaranteed to work for the characters you'll be sending out, assuming you can always have one in hand.
numpad0 2mo ago
Aren't those non-ASCII "rich" symbols from Japanese fonts around PC-98/Win95 domains anyway? For me with my background, it was always obvious that mixing full-width character in ASCII text never go well for various reasons. For ASCII arts, it is obvious that vertical lines never line up, and there are going to be tons of wasted spaces and different kinds of whitespaces needed to compensate for those. I wonder if specifying MS Gothic and retuning widths for it could help, at least for Windows/Linux.
- jack1243star 2mo ago
  Funny you'd mention MS Gothic (fixed-width), since old school Japanese ASCII-art on image boards assume MS PGothic (proportional!) instead. Some sites even try to detect ASCII art and force font-family on that specific post.
  - numpad0 2mo ago
    Yup, 2ch.net/5ch.net/5ch.io uses the MS P Gothic and there were app features and fonts like Mona Font to emulate that. They were not imageboards, though. I just thought that MS Gothic(non-P) should be kind of widely supported, have all the symbols you need, while also being a monospace, unlike most monospace fonts that only support ASCII symbols.
torginus 2mo ago
Considering the Chinese are one of the major contributors to AI, I would think this was a solved problem by now, at least in some other CLI based coding agent.
dspillett 2mo ago
> Thus, it falls back to Segoe UI Symbol, which does not have the same fixed width That seems like a glaring omission to me. If you are rendering fixed-width-per-character text and need to fall back, surely it makes sense to keep to the same character grid even if it does mess up the feel of your negative space somewhat (thin characters having a lot of space around them, wide characters butting into those beside them slightly). You've explicitly asked for text aligned to a grid, either by using a mono-spaced typeface, by using a <pre> tag, or with other relevant CSS choices, the browser should be trying to achieve that.

uhoh-itsmaciek 2mo ago

I ran into this problem recently on one of our blog posts: we used some Claude output which included tables drawn with Unicode line drawing characters. However, our monospace font did not include these characters, and so rendering fell back to another font in our font stack with different width metrics. I fixed it by using a font that had similar metrics and did include those characters with `unicode-range` (to only select characters we needed) and `size-adjust` (to match font width more exactly), and adding it to the stack. It's a little hacky but works pretty well in practice.

dolmen 2mo ago
Claude Code with Opus 4.8 is also bad at aligning boxes with content in French (with accentuated letters such as "é" which are multibyte in UTF-8).

pmarreck 2mo ago

I was going to say, perhaps generate a failing test case, but testing for proper unicode rendering might be tricky??

biztos 2mo ago

Interesting. Thai characters can also blow it out, I imagine because of the difficulty mapping glyphs to width: https://biztos.com/hey/thai-mermaid-chart.png https://biztos.com/hey/thai-mermaid-chart.png To my surprise, Sublime Text gets it almost right: https://biztos.com/hey/sublime-thai-mermaid.png https://biztos.com/hey/sublime-thai-mermaid.png I tried finding a Thai monospace font and using that in the HTML but it was worse, probably didn't have the box drawing chars. Still a fun tool and useful for lots of ASCII cases!

sirn 2mo ago
The first issue is due to the assumption that character count equals character display width. Thai tone markers usually[1] should not contribute to the display width (เพื่อน is chars = 6, width = 4), so it caused a layout shift. The second issue is due to the program's layout engine not adjusting the glyph width of a fallback font to that of the main font. A lot of terminals do this, but it's not common for text editors or browsers (arguably this is the correct behavior for non-terminals, since you cannot assume everything must be snapped to a grid). Fun test for this: |กล้วยหอม| |Bananas| This has the same character width. Ghostty, etc., will render it correctly (| aligned). Most browsers and text editors will not. [1]: some layout engines render free-standing tone markers as 1 character; in that case, this rule only applies to when tone markers are following a character.
- oneeyedpigeon 2mo ago
  That test looks pixel perfect in Chrome Android.
  - codethief 2mo ago
    Same in Firefox for Android!
- biztos 2mo ago
  Thanks for the explanation. Safari on iPad lines these up almost perfectly - the second line is a tiny bit wider, I didn’t even notice it at first. That example had a tone mark but no vowels, so I will try one with both. E&OE. |ดื่มน้ำอยู่| |abcdef| [edit] These are even closer, but still imperfectly aligned on my iPad.
  - numpad0 2mo ago
    This isn't a renderer bug, it's font. Proportional fonts aren't designed with alignment in mind at all, and you can't just expect monospace fonts for all languages outside of ASCII range to be present on random systems, or a single font or font family to support multiple different languages consistently. You can't really control alignment of deeply Unicode characters like Thai or "→" against monospace characters without serving your own monospace fonts that are guaranteed to work for the characters you'll be sending out, assuming you can always have one in hand.

numpad0 2mo ago

Aren't those non-ASCII "rich" symbols from Japanese fonts around PC-98/Win95 domains anyway? For me with my background, it was always obvious that mixing full-width character in ASCII text never go well for various reasons. For ASCII arts, it is obvious that vertical lines never line up, and there are going to be tons of wasted spaces and different kinds of whitespaces needed to compensate for those. I wonder if specifying MS Gothic and retuning widths for it could help, at least for Windows/Linux.

jack1243star 2mo ago
Funny you'd mention MS Gothic (fixed-width), since old school Japanese ASCII-art on image boards assume MS PGothic (proportional!) instead. Some sites even try to detect ASCII art and force font-family on that specific post.
- numpad0 2mo ago
  Yup, 2ch.net/5ch.net/5ch.io uses the MS P Gothic and there were app features and fonts like Mona Font to emulate that. They were not imageboards, though. I just thought that MS Gothic(non-P) should be kind of widely supported, have all the symbols you need, while also being a monospace, unlike most monospace fonts that only support ASCII symbols.

torginus 2mo ago

Considering the Chinese are one of the major contributors to AI, I would think this was a solved problem by now, at least in some other CLI based coding agent.

dspillett 2mo ago

> Thus, it falls back to Segoe UI Symbol, which does not have the same fixed width That seems like a glaring omission to me. If you are rendering fixed-width-per-character text and need to fall back, surely it makes sense to keep to the same character grid even if it does mess up the feel of your negative space somewhat (thin characters having a lot of space around them, wide characters butting into those beside them slightly). You've explicitly asked for text aligned to a grid, either by using a mono-spaced typeface, by using a <pre> tag, or with other relevant CSS choices, the browser should be trying to achieve that.