Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
108 changes: 108 additions & 0 deletions FIXES.md
Original file line number Diff line number Diff line change
Expand Up @@ -317,3 +317,111 @@ dotnet test src/XTerm.NET.slnx --no-restore
```

Expected result: all tests pass.

---

# Sixel graphics

## Summary

Sixel images (`ESC P … q … ESC \`) are decoded and placed in the buffer. Each cell an image covers
carries a reference to one shared, immutable `TerminalImage` plus the coordinates of the tile it
shows, so a picture behaves like terminal content rather than an overlay.

Sixel was not merely unimplemented before this — it was unreachable. `EscapeSequenceParser`
collapsed `DcsEntry`/`DcsParam`/`DcsIgnore`/`DcsPassthrough` into a single "discard every byte until
ST" case, `_dcs` was allocated but never written to, and the `Dcs` event was marked `[Obsolete]`
because nothing raised it. The payload never reached anything that could decode it.

## Why storage on the cell

`BufferCell` is a struct, and `InputHandler.Print` builds a fresh one for every character. That
single fact gives the whole feature its semantics for free:

| Terminal action | Existing mechanism | Effect on images |
|---|---|---|
| Print over an image cell | `new BufferCell{…}` + `SetCell` | the new struct has no image, so that cell reverts to text |
| ED / EL / ECH / DECALN | `Fill`/`ReplaceCells` with `BufferCell.Space` | tiles cleared |
| Scroll / scrollback | `CircularList` moves whole `BufferLine` objects | tiles ride along |
| Line trimmed from scrollback | line dereferenced | the image is collected with its last tile |
| Selection / `TranslateToString` | reads `cell.Content` | image cells hold `" "`, so copying yields blanks |

A reference to a shared image rather than a per-cell bitmap slice: identical overwrite granularity,
one allocation per image instead of columns times rows, and a host can coalesce a run of adjacent
tiles into a single draw call. `BufferCell` grows from 32 to 40 bytes — the packed tile `int` fits
in padding the reference already forces — which is roughly 0.65 MB on an 80-column buffer with 1000
lines of scrollback.

## Two live bugs fixed along the way

- **`CSI ? 1;1;0 S` scrolled the screen.** XTSMGRAPHICS shares its final character with SCROLL UP,
and `ToCsiCommand` strips the private marker before the lookup, so a graphics capability query
was routed to the scroll handler. Every Sixel-capable program sends one during startup, which
made this routine rather than obscure. `ScrollUp` is now guarded on `isPrivate` and the query is
answered.
- **The primary DA reply did not advertise Sixel.** `libsixel`, `chafa`, `img2sixel` and everything
built on them read attribute `4` from `CSI c` and send text art instead of pictures without it.
The reply is now `CSI ? 1 ; 2 ; 4 c`, following `Options.SixelEnabled` so it never claims a
capability that is switched off.

## Decisions worth recording

- **Images are dropped on a column resize.** Reflow re-wraps a logical line by copying ranges of
cells between lines; tiles carried through it would reassemble as a shuffled mosaic — every piece
intact, in the wrong place. A change of row count alone moves whole lines and keeps them.
- **The DCS payload is streamed, not buffered.** A full-screen Sixel runs to hundreds of kilobytes.
The parser raises `DcsHook`/`DcsPut`/`DcsUnhook` and only accumulates a whole-payload string for
the legacy `Dcs` event when something is subscribed and the sequence stays under 4 KB.
- **An abandoned sequence is distinguishable from a finished one.** `DcsUnhook` reports whether a
string terminator ended it, so a truncated image is discarded rather than half-drawn. An `ESC`
mid-payload is resolved one character late, since `ESC \` terminates and anything else abandons.
- **Sixel colour registers are kept apart from `ColorPalette`.** They are a separate numbering that
an image may redefine as it draws, and doing that to the palette the renderer reads on its hot
path would repaint the text as a side effect of showing a picture.
- **Nothing in the decoder throws.** The payload is untrusted output from another process; a
nonsense register, an absurd repeat count or a truncated stream yields no image, not an exception
escaping into the parser.
- **A host must answer the window queries from the grid, not from its control.** Not a change here
-- an unhandled query still produces no reply, deliberately -- but the reason the README now spells
the handler out. An image viewer works out the cell size for itself by dividing the pixel size from
`CSI 14 t` by the row count it already has, so anything else in that figure (a scrollbar, window
chrome, or the strip below the last row, since the grid is a truncated division) is read back as
picture that does not fit. It runs off the bottom and scrolls the screen. The only safe answer is
`Cols * CellWidthPixels` by `Rows * CellHeightPixels`, which is also what xterm reports.
- **The image budget is swept by the byte, not by the picture.** A program animating with Sixel
draws one image per frame, and sweeping on each would walk every cell of both buffers ten times a
second. Bytes placed are counted instead, and a sweep runs only once a budget's worth has arrived
— one scan per budget rather than one per picture, at the cost of the buffer sitting up to one
budget over before it is trimmed.

## Files changed

- `src/XTerm.NET/Parser/EscapeSequenceParser.cs` — real DCS state machine; streaming hook/put/unhook
- `src/XTerm.NET/Common/Types.cs` — `ParserState.DcsIntermediate`
- `src/XTerm.NET/Events/ParserEvents.cs` — `DcsHookEventArgs`, `DcsPutEventArgs`, `DcsUnhookEventArgs`
- `src/XTerm.NET/Graphics/TerminalImage.cs` — immutable BGRA image plus tile geometry
- `src/XTerm.NET/Graphics/SixelDecoder.cs` — streaming DECSIXEL decoder
- `src/XTerm.NET/Graphics/SixelPalette.cs` — VT340 defaults, RGB and HLS colour
- `src/XTerm.NET/Buffer/BufferCell.cs` — `Image`, packed `ImageTile`, equality
- `src/XTerm.NET/Buffer/BufferLine.cs` — `ClearImages`, `HasImages`
- `src/XTerm.NET/Buffer/TerminalBuffer.cs` — `ClearImages`, dropped on column resize
- `src/XTerm.NET/InputHandler.cs` — DCS dispatch, `PlaceImage`, DA, modes 80/1070/8452, XTSMGRAPHICS
- `src/XTerm.NET/Terminal.cs` — parser wiring, Sixel mode flags, `EnforceImageBudget`
- `src/XTerm.NET/Options/TerminalOptions.cs` — `SixelEnabled`, cell pixel size, budgets
- `src/XTerm.NET.Tests/Parser/DcsSequenceTests.cs`
- `src/XTerm.NET.Tests/Graphics/SixelDecoderTests.cs`
- `src/XTerm.NET.Tests/Graphics/SixelPlacementTests.cs`
- `src/XTerm.NET.Tests/Graphics/ImageCellLifetimeTests.cs`
- `src/XTerm.NET.Tests/Graphics/GraphicsAttributesTests.cs`

## Validation

```powershell
dotnet test src/XTerm.NET.slnx
```

```text
Passed: 847
Failed: 0
Skipped: 0
```
96 changes: 94 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,12 @@
[![Build Status](https://github.com/tomlm/XTerm.NET/actions/workflows/BuildAndRunTests.yml/badge.svg)](https://github.com/tomlm/XTerm.NET/actions/workflows/BuildAndRunTests.yml) [![NuGet Version](https://img.shields.io/nuget/v/XTerm.NET.svg)](https://www.nuget.org/packages/XTerm.NET/) [![NuGet Downloads](https://img.shields.io/nuget/dt/XTerm.NET.svg)](https://www.nuget.org/packages/XTerm.NET/) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

A .NET terminal emulator library inspired by [xterm.js](https://github.com/xtermjs/xterm.js).
XTerm.NET provides a headless terminal emulator that parses and processes VT100/ANSI escape sequences,
making it easy to host conosole applications in your .NET applications.
XTerm.NET provides a headless terminal emulator that parses and processes VT100/ANSI escape sequences,
making it easy to host console applications in your .NET applications.

**Sixel graphics are supported.** Images arrive as ordinary cell content rather than as an overlay, so
they are overwritten by text, cleared by `ED`/`EL`, scrolled with their lines, and freed when they fall
out of the scrollback — see [Sixel Images](#sixel-images).

## Features

Expand All @@ -15,6 +19,8 @@ making it easy to host conosole applications in your .NET applications.
- **Rich Event System** — Subscribe to terminal events like title changes, bell, resize, and window manipulation
- **256 and True Color Support** — Full RGB and 256-color palette support
- **Unicode Support** — Proper handling of wide characters and Unicode text
- **Sixel Graphics** — Decodes Sixel images (`ESC P … q`) and stores them on the cells they cover, so
`img2sixel`, `chafa`, `lsix` and `timg` work against a host that renders them

## Installation

Expand Down Expand Up @@ -246,6 +252,92 @@ void RenderTerminal(Terminal terminal)

Wide characters (e.g., CJK ideographs, emoji) have `Width = 2`. The first cell contains the character, and the second cell has `Width = 0` as a placeholder — skip it during rendering but allocate space for the double-width glyph.

### Sixel Images

A Sixel image (`ESC P … q … ESC \`) is decoded and written into the cells it covers. Each covered
cell carries a reference to one shared `TerminalImage` plus the coordinates of the piece it shows,
so an image behaves like terminal content rather than an overlay: printing over a cell replaces
that part of the picture, `ED`/`EL` clear it, scrolling carries it, and the image is freed once the
last cell holding it is gone.

**Tell the terminal your cell size.** XTerm.NET is headless and cannot measure a font, so it cannot
work out how many columns an image covers unless you say. Set these from your renderer's metrics,
in *device* pixels:

```csharp
terminal.Options.CellWidthPixels = 8;
terminal.Options.CellHeightPixels = 17;
```

**Answer the window queries from these same numbers.** An image viewer works out the cell size for
itself, by dividing the pixel size it gets from `CSI 14 t` by the row and column counts it already
has. So your `WindowInfoRequested` handler must report the **grid**, not the control:

```csharp
case WindowInfoRequest.SizePixels: // CSI 14 t
e.WidthPixels = terminal.Cols * terminal.Options.CellWidthPixels;
e.HeightPixels = terminal.Rows * terminal.Options.CellHeightPixels;
e.Handled = true;
break;

case WindowInfoRequest.CellSizePixels: // CSI 16 t
e.CellWidth = terminal.Options.CellWidthPixels;
e.CellHeight = terminal.Options.CellHeightPixels;
e.Handled = true;
break;
```

Reporting your control's own size instead is the classic way to get this wrong. It includes the
scrollbar, any window chrome, and the strip below the last row — the grid is a truncated division, so
up to a whole row of the control's height belongs to no row at all. An application dividing that
figure by the row count is told the terminal is taller than it is, sizes a picture to fill it, and the
surplus runs off the bottom and scrolls the screen.

**Rendering the tiles.** Extend the per-cell loop above:

```csharp
BufferCell cell = line[col];

if (cell.Image is TerminalImage image &&
image.TryGetTileSource(cell.ImageCol, cell.ImageRow, out int sx, out int sy, out int sw, out int sh))
{
// Pixels are BGRA8888 with straight (unpremultiplied) alpha, top row first.
// Cache your framework's bitmap against the image object — a ConditionalWeakTable keyed on
// `image` lets the bitmap die when the image does, with no eviction list to maintain.
var bitmap = _bitmaps.GetOrCreate(image);

// Edge tiles are clipped, so scale the destination to match rather than stretching a partial
// tile over a whole cell.
double destW = cellWidth * sw / (double)image.CellWidth;
double destH = cellHeight * sh / (double)image.CellHeight;

DrawImage(bitmap,
source: (sx, sy, sw, sh),
dest: (col * cellWidth, row * cellHeight, destW, destH));
continue;
}
```

Adjacent cells sharing the same `Image` reference and `ImageRow` with consecutive `ImageCol` values
are contiguous, so a renderer can coalesce them into a single draw call per row instead of one per
cell. If you cache rendered rows, note that image cells must break a text run: compare `Image` by
reference as well as comparing `Attributes`.

Image cells hold `" "` as their content, so `TranslateToString` and selection copy yield blanks.

**Options:**

| Option | Default | Purpose |
|---|---|---|
| `SixelEnabled` | `true` | Decode images, and advertise Sixel in the primary Device Attributes reply |
| `CellWidthPixels` / `CellHeightPixels` | `10` / `20` | Cell size images are laid out against |
| `MaxSixelPixels` | `4_000_000` | Largest single image accepted |
| `MaxImageBytes` | `64 MB` | Budget for image data live in the buffer; oldest are dropped past it |

Images are dropped when the terminal is resized to a different **column** count, because reflow
re-wraps lines by copying ranges of cells and the pieces would reassemble in the wrong places. A
change of row count alone keeps them.

## License

This project is licensed under the MIT License — see the [LICENSE](LICENSE) file for details.
Expand Down
127 changes: 127 additions & 0 deletions src/XTerm.NET.Tests/Graphics/GraphicsAttributesTests.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
using XTerm.Options;

namespace XTerm.Tests.Graphics;

/// <summary>
/// XTSMGRAPHICS -- "CSI ? Pi ; Pa ; Pv S" -- and the bug it used to trigger.
///
/// <para>It shares its final character with SCROLL UP, and the CSI identifier has its private
/// marker stripped before the command is looked up, so the query was routed to the scroll handler.
/// Every Sixel-capable program sends one while working out what the terminal can do, which made
/// "the screen jumps when I run img2sixel" the visible symptom of a capability query going
/// unanswered.</para>
/// </summary>
public class GraphicsAttributesTests
{
private const string Esc = "\u001b";

private static Terminal Fresh() => new(new TerminalOptions
{
Cols = 40,
Rows = 6,
CellWidthPixels = 10,
CellHeightPixels = 20
});

private static (Terminal Terminal, List<string> Replies) Listening()
{
var terminal = Fresh();
var replies = new List<string>();
terminal.DataReceived += (_, e) => replies.Add(e.Data);
return (terminal, replies);
}

/// <summary>The regression: a capability query must not move the screen.</summary>
[Theory]
[InlineData("1;1;0", "colour register count")]
[InlineData("2;1;0", "Sixel geometry")]
[InlineData("1;4;0", "maximum colour registers")]
public void A_graphics_query_does_not_scroll_the_screen(string parameters, string what)
{
var (terminal, _) = Listening();
terminal.Write("top line\r\nsecond line");

terminal.Write($"{Esc}[?{parameters}S");

Assert.True(terminal.GetLine(terminal.Buffer.YBase) == "top line",
$"querying {what} scrolled the screen instead of answering");
}

[Fact]
public void The_colour_register_count_is_reported()
{
var (terminal, replies) = Listening();

terminal.Write($"{Esc}[?1;1;0S");

Assert.Equal($"{Esc}[?1;0;256S", Assert.Single(replies));
}

[Fact]
public void The_sixel_geometry_is_reported()
{
var (terminal, replies) = Listening();

terminal.Write($"{Esc}[?2;1;0S");

// 40 columns of 10 pixels, and whatever height the pixel budget allows across that width.
var reply = Assert.Single(replies);
Assert.StartsWith($"{Esc}[?2;0;400;", reply);
Assert.EndsWith("S", reply);
}

/// <summary>
/// The reported geometry has to be a size we would actually accept, or a program that sizes an
/// image to fit gets one we then throw away.
/// </summary>
[Fact]
public void The_reported_geometry_fits_within_the_pixel_budget()
{
var (terminal, replies) = Listening();

terminal.Write($"{Esc}[?2;1;0S");

var parts = replies[0].TrimEnd('S').Split(';');
var width = int.Parse(parts[^2]);
var height = int.Parse(parts[^1]);

Assert.True((long)width * height <= terminal.Options.MaxSixelPixels,
$"reported {width}x{height}, which is larger than the {terminal.Options.MaxSixelPixels} pixel budget");
}

[Fact]
public void An_unknown_item_is_refused()
{
var (terminal, replies) = Listening();

terminal.Write($"{Esc}[?9;1;0S");

Assert.Equal($"{Esc}[?9;1S", Assert.Single(replies));
}

/// <summary>
/// The limits are fixed, so accepting a request to change them and quietly not doing it would
/// be worse than refusing.
/// </summary>
[Fact]
public void A_request_to_change_a_limit_is_refused()
{
var (terminal, replies) = Listening();

terminal.Write($"{Esc}[?1;3;64S"); // action 3 is "set"

Assert.Equal($"{Esc}[?1;2S", Assert.Single(replies));
}

/// <summary>Without the private marker it is still SCROLL UP, and still has to scroll.</summary>
[Fact]
public void Scroll_up_still_works()
{
var terminal = Fresh();
terminal.Write("top line\r\nsecond line");

terminal.Write($"{Esc}[1S");

Assert.Equal("second line", terminal.GetLine(terminal.Buffer.YBase));
}
}
Loading
Loading