Skip to content

Autocomplete on KDE Plasma: show suggestions in an overlay pill instead of preedit - #563

Merged
stevyhacker merged 1 commit into
stevyhacker:masterfrom
SamLeatherdale:fix/kwin-autocomplete-overlay-pill
Nov 21, 2026
Merged

stevyhacker merged 1 commit into
stevyhacker:masterfrom
SamLeatherdale:fix/kwin-autocomplete-overlay-pill

Conversation

@SamLeatherdale

@SamLeatherdale SamLeatherdale commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Summary

On KDE Plasma (Wayland), lokalbot --kwin-input-method now draws autocomplete suggestions on a click-through zwp_input_panel_v1 overlay pill instead of zwp_input_method_context_v1.preedit_string. KWin bridges preedit_string to clients over zwp_text_input_v3 without a reset-commit string, and Chromium, Electron, Firefox, and Thunderbird commit active zwp_text_input_v3 preedit into the document on mousedown or blur before notifying KWin. Keeping unaccepted suggestions out of preedit prevents focus loss from inserting unaccepted text.

Screencast_20261121_145247-00.00.07.147-00.00.22.033.mp4

Review guide

Reading order:

  1. LinuxWindows/crates/lokalbot-platform/src/autocomplete/input_method/kwin_wire.rs — bumps VERSION from 1 to 2 and replaces Request::Preedit with Request::Ghost and Request::Nudge.
  2. LinuxWindows/crates/lokalbot-platform/src/autocomplete/input_method/overlay_pill.rs — binds zwp_input_panel_v1 (set_overlay_panel), rasterizes the pill with cosmic-text into an anonymous memfd wl_shm buffer, and scales via wp_fractional_scale_v1 + wp_viewporter.
  3. LinuxWindows/crates/lokalbot-platform/src/autocomplete/input_method/kwin_helper.rs and kwin.rs — drives OverlayPill from the helper and hides it on Activate, Deactivate, Reset, drop_app, and passed-through keys.
  4. LinuxWindows/crates/lokalbot-platform/src/autocomplete/input_method.rs, autocomplete/mod.rs, and lokalbot-app/src/autocomplete/mod.rs — adds Core::with_overlay_ghost() and GhostStyle (next-Tab emphasis length and dark/light palette) without disturbing routes that still use preedit.

Decisions and focus areas:

  • Why zwp_input_panel_v1 in kwin_helper instead of clearing preedit on Deactivate or opening a GPUI popup: Blink (InputMethodAuraLinux::ConfirmCompositionText) and Gecko (IMContextWrapper) commit active preedit inside the client before sending zwp_text_input_v3.disable() to KWin, so any Deactivate cleanup arrives too late. Ordinary Wayland clients cannot read caret screen coordinates or bind zwp_input_panel_v1; KWin grants that interface only on the WAYLAND_SOCKET passed to --kwin-input-method and anchors set_overlay_panel surfaces to the focused text input's cursorRectangle.
  • Privacy invariant in overlay_pill.rs: create_shm_file backs wl_shm_pool with libc::memfd_create (MFD_CLOEXEC), preserving PRIVACY.md's guarantee that --kwin-input-method writes no file on disk other than its runtime socket. cosmic-text 0.19.0 was already pinned in Cargo.lock via GPUI, so no new third-party crate or license entry is added.
  • Empty-preedit Request::Nudge: When Core::with_overlay_ghost() probes a silent zwp_text_input_v3 field (such as Chromium focused mid-text), kwin_helper sends preedit_string(serial, "", "") rather than suggestion text so KWin refreshes surrounding_text without starting a non-empty IME composition.

Validation

Automated checks (from LinuxWindows/ and repo root on Kubuntu 26.04.1, KDE Plasma 6.6.6 / KWin 6.6.7):

cargo fmt --all -- --check
cargo clippy --workspace --all-targets --locked -- -D warnings
TZ=UTC LC_ALL=C.UTF-8 cargo test --workspace --locked
bash crates/lokalbot-platform/scripts/live-kwin.sh
python3 -m unittest discover -s scripts/tests
bash scripts/third-party-licenses.sh --check
python3 ../Scripts/ci/check-sync-crate-notices.py

All commands exited 0, including live_input_method::kwin_input_method_shows_accepts_and_stays_out_of_passwords under headless kwin_wayland and the new overlay_pill unit tests across dark/light themes and 1.0x/1.5x/2.0x scales. Verified that commit_on_blur in live_input_method fails when preedit_string is restored and passes with the overlay pill.

Headless kwin_wayland client matrix (LOKALBOT_FAKE_AUTOCOMPLETE=" world", typing hello and blurring via second input click, blank-area click, and window switch):

  • Google Chrome (Wayland): Tab → 'hello world'; all three blur cases → 'hello' (was 'hello world').
  • Firefox (Wayland): Tab → 'hello world'; all three blur cases → 'hello' (intra-window clicks were 'hello world').
  • Qt 6 (QLineEdit, QTextEdit): Tab → 'hello world'; blur → 'hello'.

Manual desktop checks on a live KDE Plasma Wayland session (with on-device LFM2.5-1.2B-Instruct):

  • Suggestion display
  • partial (Tab) and full (`) acceptance,
  • Esc dismissal inside modals
  • click-through directly over the pill
  • bottom-of-screen clamping
  • light and dark themes
  • browser zoom
  • password/terminal refusal
  • closed-app passthrough
  • live monitor dragging between 150% and 200% displays
  • Tested across Chrome, Firefox, Electron apps, Kate (Qt 6), GTK 4 (zenity, gnome-text-editor), GTK 3 (mousepad), and LibreOffice Writer

Not run: macOS Xcode targets and Windows desktop builds (CI covers these).

Changed test expectations

  • live_input_method::kwin_input_method_shows_accepts_and_stays_out_of_passwords: expected preedit ' up on it' and preedit ' on it' on the GTK Entry while suggestions were visible → now asserts that no non-empty preedit ever reaches the field (assert_no_nonempty_preedit / expect_without_preedit) while Tab, replacement accepts, key passthrough, Esc, password refusal, and helper disconnect keep their expected text results.
    Why the old expectation was wrong: putting unaccepted suggestions into preedit assumed Wayland clients discard preedit on focus loss, whereas zwp_text_input_v3 clients (Blink and Gecko) commit active preedit into the document on mousedown/blur.

Linked issues

Fixes #562

Risk / rollout notes

  • Session upgrade handshake (kwin_wire::VERSION 1 → 2): KWin starts lokalbot --kwin-input-method once per desktop session. Updating the installed binary while a v1 helper is still running causes kwin.rs / kwin_helper.rs to log a version mismatch and disconnect cleanly rather than misparsing frames; autocomplete resumes once the user re-applies Virtual Keyboard in System Settings or logs in again.
  • Wayland compositor scope: This change moves only KDE Plasma (kwin / zwp_input_method_v1 + zwp_input_panel_v1) to Core::with_overlay_ghost(). Sway/Hyprland (wlroots.rs, which needs zwp_input_popup_surface_v2 rather than zwp_input_panel_v1) and GNOME (ibus.rs) still use Core::new (preedit).
  • Upstream LibreOffice Writer (qt6/kf6 VCL) caret-position bug: Consecutive Tab chunk accepts in LibreOffice Writer keep the pill at the starting caret position until the next physical keypress (Kate and other Qt 6 apps update normally).
    • We investigated nudging Qt after commit_string, and confirmed this cannot be worked around safely from the input method: QWaylandTextInputv3::zwp_text_input_v3_done already queries Qt::ImCursorRectangle right after commit_string, while LibreOffice's QtWidget::commitText defers visual cursor layout (SalEvent::ExtTextInputPos) to idle without calling QInputMethod::update(Qt::ImCursorRectangle) afterward. A follow-up empty preedit_string("") is ignored by QWaylandTextInputv3 (m_pendingPreeditString == m_currentPreeditString), and sending non-empty preedit would reintroduce active IME composition in other clients.

…ad of preedit

Putting an unaccepted suggestion into `zwp_input_method_context_v1.preedit_string`
made `zwp_text_input_v3` clients such as Chromium, Electron, Firefox, and
Thunderbird commit the suggestion into the document whenever the field lost
focus on click or blur, because KWin bridges `preedit_string` without its
reset-commit string and those toolkits finalize active IME compositions locally
before notifying the compositor.

- Draw KDE Plasma suggestions on a click-through `zwp_input_panel_v1`
  (`set_overlay_panel`) surface (`overlay_pill.rs`) anchored by KWin to the
  focused field's `cursor_rectangle`, rendered via `cosmic-text` into an
  anonymous `memfd` buffer with `wp_fractional_scale_v1` + `wp_viewporter`
  HiDPI scaling and light/dark theme tokens.
- Bump `kwin_wire::VERSION` to 2, replacing `Request::Preedit` with
  `Request::Ghost` and an empty-preedit `Request::Nudge` for quiet fields, and
  keep suggestions out of `Core`'s `preedit` expectation via
  `Core::with_overlay_ghost()`.
- Update `live_input_method` to assert zero non-empty preedit on the KWin route
  and add a focus-loss regression check (`commit_on_blur`), plus update
  `LinuxWindows/README.md`.

Co-authored-by: Cursor <cursoragent@cursor.com>
@SamLeatherdale
SamLeatherdale marked this pull request as ready for review November 21, 2026 03:40
@stevyhacker

Copy link
Copy Markdown
Owner

Thanks for the PR, will review now!

stevyhacker added a commit that referenced this pull request Nov 21, 2026
Land #563: KDE autocomplete overlay pill, onto the comment-free master
@stevyhacker
stevyhacker merged commit ef0da2a into stevyhacker:master Nov 21, 2026
21 of 26 checks passed
stevyhacker pushed a commit that referenced this pull request Nov 21, 2026
Sway and Hyprland forward input method preedit to apps over
zwp_text_input_v3 as KWin does, so Chromium, Electron, Firefox and
Thunderbird commit an unaccepted suggestion into the field on blur (#562).
The wlroots route now draws the overlay pill from #563 on a
zwp_input_popup_surface_v2. Sway before 1.10 accepts the popup but never
shows it, so the route keeps preedit there: the popup counts only once the
compositor sends text_input_rectangle, and without one within 300 ms of the
first suggestion the route falls back to preedit for the session.

Also from the #563 review:
- The KWin helper takes no keys when KWin offers no input panel, so a
  suggestion it cannot draw never eats Tab.
- The pill loads the system font database on its own thread when the
  route starts, not on the first suggestion.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QD3UkuaVf6PEtW1ptFCMB1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Autocomplete suggestions on KDE Plasma (Wayland) are committed into text fields on focus loss

2 participants