Search the scrollback without turning it into text - #47
Conversation
The perf gate on tomlm#47 flagged alt-redraw at +6.0% against a 6.0% gate with 2% noise, and it was right. NoteLink carried a remark saying "guarded at the CALL as well as inside" -- and the three call sites were bare. The guard lived only inside the helper, so every printed character and every batched chunk paid a call to be told there is no link and never has been. The exact shape that cost alt-redraw 12% in the placeholder work, caught this time by the CI gate that was calibrated on it. The three writers now test `_linkUrl is not null || line.HasLinks` themselves and call NoteLinkRun directly; the helper is gone, because three guarded call sites plus a helper whose comment lied is worse than no helper. 1191 tests pass unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
The
Fixed in |
|
some stacking here, sorry about that. should be the last time. this is the last of what I wanted to land. |
There was a problem hiding this comment.
🟡 Changes recommended
Confirmed correctness issues in the new search and hyperlink-span maintenance logic need fixes before the new APIs can be relied on.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR adds scrollback searching that operates directly on buffer cells (avoiding per-line string materialization/allocation) and expands line-level state to anchor OSC 8 hyperlinks and OSC 133 shell-integration marks to BufferLines for stable hit-testing and prompt navigation.
Changes:
- Introduces
BufferSearch(andSearchHit/SearchOptions) to find matches across logical (wrapped) lines without building strings. - Anchors OSC 133 marks (
LineMark) and OSC 8 hyperlink spans (LineHyperlink) toBufferLine, with print-path bookkeeping and prompt navigation helpers onTerminal. - Adds comprehensive tests covering wrapping, trimming, caps/truncation, hyperlink/mark anchoring, and allocation assertions.
File summaries
| File | Description |
|---|---|
| src/XTerm.NET/Terminal.cs | Adds prompt navigation helpers based on anchored OSC 133 marks. |
| src/XTerm.NET/Search/SearchTypes.cs | Adds search result and option types (SearchHit, SearchOptions). |
| src/XTerm.NET/Search/BufferSearch.cs | Implements allocation-free scrollback search over buffer cells. |
| src/XTerm.NET/InputHandler.cs | Anchors shell marks to lines and records hyperlink spans during printing. |
| src/XTerm.NET/Buffer/LineMark.cs | Defines line-anchored shell-integration mark structure. |
| src/XTerm.NET/Buffer/LineHyperlink.cs | Defines line-anchored hyperlink span structure. |
| src/XTerm.NET/Buffer/BufferLine.cs | Stores/updates marks and hyperlink spans on lines; clears on reuse. |
| src/XTerm.NET.Tests/ShellIntegrationMarkAnchorTests.cs | Tests mark anchoring and prompt navigation behavior. |
| src/XTerm.NET.Tests/HyperlinkAnchorTests.cs | Tests hyperlink span anchoring and mutation under overwrites. |
| src/XTerm.NET.Tests/BufferSearchTests.cs | Tests search behavior (wraps, stepping, trimming, cap, allocations). |
Review details
Suppressed comments (1)
src/XTerm.NET/Search/BufferSearch.cs:332
- Whole-word boundary check on the "after" side doesn't skip width-0 placeholder cells, so a match that ends before a wide glyph's placeholder may see the placeholder as the next character and incorrectly treat it as a boundary.
if (!Advance(lines, endRow, ref r, ref c, skipZeroWidth: false))
return false; // end of the logical line
- Files reviewed: 10/10 changed files
- Comments generated: 3
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Ghostty's most-requested feature until 1.3, and the design turns on one measurement: asking each line for a string and running IndexOf costs 9.7 ms and 16.3 MiB per search over a 10,000-line buffer, while reading codepoints out of the cells costs 3.7 ms and allocates nothing, for the same 25,325 hits. A find box searches per keystroke, so the first of those is roughly 80 MiB to type "error" -- in a library that went to some trouble to stop allocating per character. So BufferSearch walks cells. The walk moves a (row, column) cursor and compares codepoints, which also dissolves the offset-mapping problem: the position IS the cursor, and a match that crosses a wrap simply produces two runs because the cursor changed row part way through. The runs share a MatchId, the same shape as an OSC 8 link that wrapped, and stepping moves by match so a wrapped one is one stop. Logical lines, not physical: a search that stops at the right edge misses exactly the matches on long output, which are the ones worth finding. Width-0 cells are stepped over rather than compared -- the placeholder behind a wide glyph and an orphaned combining mark are not characters, and comparing them would fail a needle against text that reads exactly like it. Results follow the buffer. The ring drops lines as output arrives and a row index that is not adjusted goes stale silently -- the highlight lands somewhere else, and nothing about it looks wrong. Subscribes to Trimmed and shifts, as SelectionManager does, dropping what fell off the top. HitsOnRow answers by binary search over row-ordered hits and hands back a span in place, because the renderer asks once per row per frame and the answer is almost always nothing. The cap (10,000 runs) is announced through Truncated rather than silently applied: a count that quietly stops being true reads as a bug in the search rather than a limit on it. No regular expressions, deliberately. Regex takes a string, so supporting it means materialising the buffer as text -- the 16 MiB this design exists to avoid, paid by every search rather than the ones that asked. Substring, case-insensitive by default, whole-word as an option. One measurement trap found by this change's own test: it first asserted with GC.GetTotalAllocatedBytes, which is process-wide, and xUnit runs test classes in parallel -- so it counted 66 MB of other tests' allocations against the search and failed a design that was allocating nothing. GetAllocatedBytesForCurrentThread cannot be polluted. The bisection that found this took longer than the fix. Thirteen tests: wrap-straddling matches, whole-word across a wrap boundary, ring trimming, the cap admitting it bit, and the zero-allocation claim asserted per thread. 1191 tests pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The third finding -- SplitLinksOver appending the right fragment -- was fixed in b1fcc00 before this review posted; Copilot reviewed the pre-merge head. The needle is now codepoints, not chars. A cell stores a codepoint, and comparing it to UTF-16 chars one at a time can never match anything outside the BMP: an emoji in the needle is two chars and neither equals the cell. The needle folds once per Find; astral case pairs are left exact, which is honest rather than lossy. A cell holding a multi-codepoint cluster is still compared by its leading codepoint, and the remarks now say why: matching full cluster text means materialising it per cell, the same cost this class exists to avoid and the same reason it offers no regular expressions. The limitation is one-directional -- base-character searches still find cells carrying combining marks. And the whole-word left scan steps past width-0 cells. The placeholder behind a wide glyph is not a character, and treating it as the neighbour let a whole-word match sit flush against a CJK letter. Two regression tests. 1197 pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
b4dcd89 to
760fdec
Compare
Ghostty's most-requested feature until 1.3. The design note with the full reasoning is written up separately; the short version turns on one measurement.
The measurement
A 10,000-line scrollback at 240 columns, searched for
error, both ways, same 25,325 hits:IndexOfA find box searches per keystroke, so the obvious implementation is ~80 MiB to type "error" — in a library that just spent #38 getting per-character allocation to zero. So
BufferSearchwalks cells and never builds a string.What walking cells buys beyond the allocation
The offset-mapping problem dissolves. A string search returns character offsets that then have to be mapped back onto cells — wide characters, clusters, wraps. Here the position is the (row, column) cursor, and a match that crosses a wrap simply produces two runs because the cursor changed row part way through. The runs share a
MatchId— the same shape as an OSC 8 link that wrapped — andTryMoveNextsteps by match, so a wrapped one is one stop.Logical lines, not physical. A search that stops at the right edge misses exactly the matches on long output, which are the ones worth finding.
Width-0 cells are stepped over, not compared. The placeholder behind a wide glyph and an orphaned combining mark aren't characters; comparing them would fail a needle against text that reads exactly like it.
Results that survive the buffer moving
The ring drops lines as output arrives, and a stale row index fails silently — the highlight just lands somewhere else.
BufferSearchsubscribes toTrimmedand shifts, asSelectionManageralready does, dropping what fell off the top. There's a test that scrolls a match through a trim and checks the row it lands on still saysneedle.Shaped for the renderer
HitsOnRow(row)answers by binary search over row-ordered hits and returns a span in place — asked once per row per frame, and the answer is almost always nothing. The cap (10,000 runs) is announced throughTruncatedrather than silently applied: a count that quietly stops being true reads as a bug in the search rather than a limit on it.Deliberately no regex
Regextakes a string, so supporting it means materialising the buffer as text — the 16 MiB this design exists to avoid, and paid by every search rather than only the ones that asked. Substring, case-insensitive by default, whole-word as an option. If regex ever lands it should materialise one logical line at a time.A measurement trap worth recording
This change's own zero-allocation test initially failed at 66 MB — against code that was allocating nothing.
GC.GetTotalAllocatedBytesis process-wide and xUnit runs test classes in parallel, so it counted the rest of the suite's allocations on other threads against the search.GetAllocatedBytesForCurrentThreadcannot be polluted. Filed here because it's the same trap anyone adding an allocation assertion to this suite will hit.Tests
1191 pass, thirteen new: wrap-straddling matches, whole-word across a wrap, ring trimming, the cap admitting it bit, and the zero-allocation claim asserted per thread.
The renderer half — highlight in the render path, scroll-to-match, find-next forwarded to the control — is already written as Iciclecreek#88, and follows once this merges and a prerelease carries it. The terminal-side PRs deliberately come after the emulator ones: they cannot compile until the package exists.
🤖 Generated with Claude Code