Skip to content

Styled underlines and underline colour, and the sub-parameters they need - #36

Merged
tomlm merged 8 commits into
tomlm:mainfrom
JohnCampionJr:styled-underlines
Aug 27, 2026
Merged

Styled underlines and underline colour, and the sub-parameters they need#36
tomlm merged 8 commits into
tomlm:mainfrom
JohnCampionJr:styled-underlines

Conversation

@JohnCampionJr

Copy link
Copy Markdown
Collaborator

SGR 4:3 asks for a curly underline — how an LSP marks an error. It was arriving as parameter 43.

The cause is bigger than underlines

The parser transitioned to CsiIgnore on a colon and abandoned the whole sequence. So 38:2::255:0:0 — the ITU-T spelling of a foreground colour, which plenty of programs emit — produced no colour and no error. Params.AddSubParam existed and nothing called it; GetSubParams was a stub returning an empty list, comment included:

// Sub-parameters are stored contiguously
// This is a simplified version
return result;

So most of this PR is sub-parameter support, and styled underlines are what it unlocks.

Sub-parameters are stored flat with a start offset per parameter — almost no sequence has any, and the ones that do have a handful, so a list per parameter would allocate on every CSI to describe nothing. An empty slot is kept as a real zero: dropping it would shift 58:2::255:0:0 by one and turn red into black.

One trap worth flagging for review. A colon case written beside the digit branch in the state machine can never be reached, because 0x3A sits inside the 0x30..0x3F parameter-byte range that branch already claims. I wrote it there first and it silently did nothing — which may be how the original gap went unnoticed. It belongs in Param(), next to the semicolon.

What lands on top

  • SGR 4:04:5 select the underline style; 21 is a double underline
  • SGR 58 sets the underline's own colour, in both the colon and semicolon spellings; 59 restores it to the foreground
  • IsUnderline() now reports the style rather than a separate flag, so the two can't disagree

The UnderlineStyle enum already existed and already matched Ghostty's — none/single/double/curly/dotted/dashed.

It costs nothing

AttributeData.Extended holds nine flags in a 32-bit field, so the style went in bits 9–11 and an interned colour id in 12–31. The struct is still twelve bytes and the cell still 24 bytes, and there's a test asserting that rather than trusting it.

Interning was chosen over two alternatives, both measured rather than argued:

  • Growing the struct — a bigger cell costs most on fills; going 24 → 32 bytes cost scroll-heavy output 22% in earlier work.
  • Interning the whole style, as Ghostty does — that needs reference counting, which forces every cell write to read the old id to release it. Measured at 240 ns per line against 165. An underline colour releases nothing, so it adds nothing to the write path; the lookup happens once per run per frame on the render side.

Full RGB is representable. What's bounded is how many distinct colours coexist — about a million, against the four an LSP uses.

Tests

18 new, covering every style, both 58 spellings, indexed and RGB, reset behaviour, interning identity, that the new bits don't disturb the other attributes, and that the struct didn't grow.

867 passing. Independent of #32, #34 and #35 — parser and attributes, no buffer. The renderer side (drawing curly, dotted, dashed) is Iciclecreek work and I'll open it separately.

SGR 4:3 asks for a curly underline, which is how an LSP marks an error. It arrived
as parameter 43.

The cause is worth stating plainly, because it is not specific to underlines: the
parser threw away every colon form there is. On a colon in a CSI it transitioned to
CsiIgnore and abandoned the whole sequence -- so 38:2::255:0:0, the ITU-T spelling
of a foreground colour that plenty of programs emit, produced no colour and no
error either. Params.AddSubParam existed and nothing called it; GetSubParams was a
stub returning an empty list, comment included.

Sub-parameters are now parsed and retrievable. They are stored flat with a start
offset per parameter, because almost no sequence has any and the ones that do have
a handful -- a list per parameter would allocate on every CSI to describe nothing.
An empty slot is kept as a real zero: dropping it would shift 58:2::255:0:0 by one
and turn red into black.

Worth recording where the colon case actually belongs. A branch for it beside the
digit branch in the state machine can never be reached, because 0x3A sits inside
the 0x30..0x3F parameter-byte range that branch already claims. It goes in Param,
next to the one for the semicolon. Written the obvious way it silently did nothing,
which is how it went unnoticed in the first place.

On top of that: SGR 4:0-4:5 select the underline style, 21 is a double underline,
58 sets the underline's own colour in both the colon and semicolon spellings, and
59 puts it back to the foreground. IsUnderline now reports the style rather than a
separate flag, so the two cannot disagree.

The feature costs nothing. AttributeData.Extended holds nine flags in a 32-bit
field, so the style went in bits 9-11 and an interned colour id in 12-31 -- the
struct is still twelve bytes and the cell still 24. Interning was chosen over
growing the struct after measuring that a bigger cell costs most on fills, and over
interning the whole style, which needs reference counting and therefore a read of
the old cell on every write. An underline colour releases nothing, so it adds
nothing to the write path. A test asserts the size rather than trusting it.

867 tests pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Comment thread src/XTerm.NET/Parser/EscapeSequenceParser.cs
JohnCampionJr and others added 5 commits August 27, 2026 17:52
From review. _inSubParam and _subParamValue are parser-lifetime state, and
FlushSubParam only runs on a separator or at dispatch -- neither of which happens
when a sequence is abandoned rather than finished. Left set, the digit branch
swallows every digit of the NEXT sequence up to its first separator, so its first
parameter reads as 0.

That is worse than a dropped sequence, because 0 means something for most of them:
ESC[31m becomes SGR 0 and resets every attribute instead of setting red, and
ESC[2;5H loses its row and homes the cursor. It self-corrects after one sequence,
which makes it a single silently misparsed sequence rather than an obvious fault.

Cleared where the parser already clears the rest of its transient state: on entry to
CsiEntry and DcsEntry, and in Reset. The second matters on its own -- without it an
application cannot recover in-band, so a partial write followed by RIS leaves the
terminal misreading the first sequence after the reset, and RIS is exactly what
someone reaches for.

All four ways in are covered plus the cursor case, and each fails without the fix.

Also closed the latent hole in UpdateLastParam. Its empty branch pushed to _params
and not to _subParamStart, and every sub-parameter lookup indexes the second by the
first's index -- so the next AddSubParam or GetSubParams would throw. The parser
cannot reach it today because entering CsiEntry always seeds a parameter, but Params
is public and the invariant is new with this work.

880 tests pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@JohnCampionJr

Copy link
Copy Markdown
Collaborator Author

@tomlm I also tested this against the perf work and there was no affect so thats good. anything we do going forward should run before/after tests to make sure we don't accidentally chip away at the hard won gains of that PR

@tomlm
tomlm merged commit 1023432 into tomlm:main Aug 27, 2026
1 check passed
@tomlm tomlm mentioned this pull request Aug 27, 2026
JohnCampionJr added a commit to JohnCampionJr/Iciclecreek.Avalonia.Terminal that referenced this pull request Aug 28, 2026
The squiggly underline an LSP puts under an error. XTerm.NET#36 parses SGR 4:1-4:5,
21 and 58; this draws them.

Underline stops going through Avalonia's TextDecorations, which cannot express
either half of the feature: there is no curly decoration, and SGR 58 gives the
underline a colour independent of the text. So it is drawn directly -- double
straddles where a single line sits, because a second line below falls out of the
cell; curly is a stroked sine; dotted and dashed differ only in mark length.
Strikethrough and overline still use TextDecorations, which draw them well enough.

Curly and the dash patterns are phase-locked to the cell's own x rather than
restarting per run. Restarting makes a squiggle read as a row of ticks and puts a
dash at every run boundary, which looks like a solid line.

Drawn on BOTH paths, which is the part worth reviewing. A line is painted by the
build path on the frame it changes and by the cached replay afterwards. Wiring only
the replay leaves every newly written underline missing until something else
invalidates the line -- I shipped exactly that bug locally, and a whole green test
suite did not notice, because the run list carried the right style the entire time.
The tests here assert what was DRAWN, using a colour nothing else on the frame
uses, and cover both paths.

Depends on the emulator exposing the style and colour, which is tomlm/XTerm.NET#36.
Until that ships in a package this does not compile -- the only two errors are
GetUnderlineStyle and TryGetUnderlineColor.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
JohnCampionJr added a commit to JohnCampionJr/Iciclecreek.Avalonia.Terminal that referenced this pull request Aug 28, 2026
The squiggly underline an LSP puts under an error. XTerm.NET#36 parses SGR 4:1-4:5,
21 and 58; this draws them.

Underline stops going through Avalonia's TextDecorations, which cannot express
either half of the feature: there is no curly decoration, and SGR 58 gives the
underline a colour independent of the text. So it is drawn directly -- double
straddles where a single line sits, because a second line below falls out of the
cell; curly is a stroked sine; dotted and dashed differ only in mark length.
Strikethrough and overline still use TextDecorations, which draw them well enough.

Curly and the dash patterns are phase-locked to the cell's own x rather than
restarting per run. Restarting makes a squiggle read as a row of ticks and puts a
dash at every run boundary, which looks like a solid line.

Drawn on BOTH paths, which is the part worth reviewing. A line is painted by the
build path on the frame it changes and by the cached replay afterwards. Wiring only
the replay leaves every newly written underline missing until something else
invalidates the line -- I shipped exactly that bug locally, and a whole green test
suite did not notice, because the run list carried the right style the entire time.
The tests here assert what was DRAWN, using a colour nothing else on the frame
uses, and cover both paths.

Depends on the emulator exposing the style and colour, which is tomlm/XTerm.NET#36.
Until that ships in a package this does not compile -- the only two errors are
GetUnderlineStyle and TryGetUnderlineColor.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
tomlm added a commit to tomlm/Iciclecreek.Avalonia.Terminal that referenced this pull request Aug 28, 2026
* Draw styled underlines, by hand

The squiggly underline an LSP puts under an error. XTerm.NET#36 parses SGR 4:1-4:5,
21 and 58; this draws them.

Underline stops going through Avalonia's TextDecorations, which cannot express
either half of the feature: there is no curly decoration, and SGR 58 gives the
underline a colour independent of the text. So it is drawn directly -- double
straddles where a single line sits, because a second line below falls out of the
cell; curly is a stroked sine; dotted and dashed differ only in mark length.
Strikethrough and overline still use TextDecorations, which draw them well enough.

Curly and the dash patterns are phase-locked to the cell's own x rather than
restarting per run. Restarting makes a squiggle read as a row of ticks and puts a
dash at every run boundary, which looks like a solid line.

Drawn on BOTH paths, which is the part worth reviewing. A line is painted by the
build path on the frame it changes and by the cached replay afterwards. Wiring only
the replay leaves every newly written underline missing until something else
invalidates the line -- I shipped exactly that bug locally, and a whole green test
suite did not notice, because the run list carried the right style the entire time.
The tests here assert what was DRAWN, using a colour nothing else on the frame
uses, and cover both paths.

Depends on the emulator exposing the style and colour, which is tomlm/XTerm.NET#36.
Until that ships in a package this does not compile -- the only two errors are
GetUnderlineStyle and TryGetUnderlineColor.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Fix the six findings from Tom's review, and the three notes besides

All six confirmed against the code; the high one was a real regression.

DOUBLE-WIDTH LINES LOST THEIR UNDERLINE. Taking underline out of
GetTextDecorations removed it from every caller, and only the
normal-line renderer gained the by-hand replacement -- so plain SGR 4 on
a DECDWL/DECDHL line, which underlined before this branch, silently drew
nothing. The double-width loop now derives style and colour the same way
the build path does and calls DrawUnderline with untransformed geometry,
letting the pushed matrix double it along with the glyphs. Three test
cases cover #6, #3 and #4, which had no coverage at all.

The curly wave stays inside its cell: centred ON baseY with the
amplitude chosen so a lobe plus half the pen ends exactly at the cell's
bottom edge. It used to sit ~1.5 thicknesses lower, where the next row's
background fill chopped it flat -- but only when that row HAD a fill, so
the same escape sequence rendered differently depending on the line
below it, and on the last row it painted outside the view.

The cached replay path no longer builds geometry per frame. The curly
geometry is built once per cached run, relative to the run's own origin
so it stays valid while the line scrolls, and translated into place;
dotted and dashed became what Pen.DashStyle is for -- one DrawLine with
the pattern in the pen and the phase-lock baked into its offset,
replacing a FillRectangle per dot; both pens are ImmutablePen, built
once and kept on the run. One quadratic bezier per half-period lobe
replaces eight line segments per period.

DrawUnderline now takes the caller's already-snapped width and row
height instead of re-deriving both from raw cell metrics, which put an
antialiased seam at every attribute change inside an underlined span and
let the baseline drift a device pixel between rows.

The replay test now proves the replay: it asserts the cache survived the
first render and is the SAME INSTANCE after the second, so a rebuild
cannot impersonate a replay. The drawn-assertions run for every style,
not just curly; Double is asserted as a PAIR of lines, since an Any()
would pass a Double that quietly drew one.

And the notes: DrawImageRun's stranded remarks moved back home, so
nothing carries two summary tags; the "both renderers" sentence is gone,
since in this repository there is only one; the incidental fix --
underline+strikethrough used to drop the strikethrough, and drawing
underline by hand fixed it -- is now stated in the PR description.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Tom Laird-McConnell <thermous@iciclecreek.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants