Skip to content

Make the framebuffer demos playable on touch devices - #1252

Open
mho22 wants to merge 1 commit into
mainfrom
touch/framebuffer-demo-touch-controls
Open

Make the framebuffer demos playable on touch devices#1252
mho22 wants to merge 1 commit into
mainfrom
touch/framebuffer-demo-touch-controls

Conversation

@mho22

@mho22 mho22 commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

Why

The framebuffer demos assume a desktop. fbDOOM is keyboard-driven, so on a phone or tablet there is no way to move, shoot, or even leave the demo attract loop. The modeset drawing demo listened only for mouse events, so a finger could not draw. Worse, tapping the fbDOOM canvas requested browser pointer lock; on a touch device there is no Esc key to release it, so a tap could strand the user in locked mode. Small screens had their own problems: the Launch New Machine pane trapped vertical scrolling inside the machines table, the theme popup clipped its lower entries so Dark mode was unreachable, and the dock could scroll its left edge out of reach.

What changed

  • New touch overlay for keyboard-driven framebuffer demos (apps/browser-demos/pages/kandelo/panes/TouchControls.tsx): a keyboard-shaped d-pad bottom-left, a FIRE button (LeftCtrl) bottom-right, and a MENU button (Esc) top-left. Buttons send the same Linux MEDIUMRAW bytes as the physical keyboard path, with a minimum 50 ms hold so clients that latch key state once per tic never miss a press.
  • A tap on the fbDOOM canvas sends Enter and Space together: the DOOM menu reads Enter (select) and ignores Space, the game reads Space (use — doors, switches) and ignores Enter, so one gesture covers both without extra buttons.
  • The overlay is gated by a new optional presentation.touchControls boolean carried through the demo-config pipeline (web-libs/kandelo-session), set on the doom profile in the three homebrew demo JSON sources and the builtin fallback, and shown only on coarse-pointer devices while a process owns the framebuffer.
  • The modeset demo now uses pointer events, so a touch acts as a left-button mouse: single-finger tracking, pointer capture for drags past the canvas edge, and coordinates clamped to the framebuffer bounds to stay in sync with the guest cursor.
  • Pointer-capture requests coming from touch input are ignored in host/src/framebuffer/browser-controls.ts, so a tap can never strand a touch user in pointer-lock mode, and the pointer-lock rejection thrown by touch-only platforms is swallowed.
  • Small-screen CSS: the launch pane scrolls as one page (title, filter, and machines list together) with horizontal overflow confined to the table itself, the theme popup scrolls vertically, and the dock keeps both edges reachable when it overflows.
  • The seven image packages that declare web-libs/kandelo-session/src/demo-config.ts as a build input get revision bumps (shell moves to canonical revision 24), the touch-enabled demo config digest is recorded in the lazy-shell artifact lock, and packages/registry/program-packages.json is regenerated against the current main tree so cached archives without the new field are rejected.
  • docs-site/guide/vfs-images.md documents presentation.touchControls.

Validation

  • cargo xtask build-deps program-index-check and program-index-context-check pass via scripts/dev-shell.sh: every row of the regenerated index matches the committed tree.
  • 142/142 Vitest tests pass via scripts/dev-shell.sh across the four affected suites: host/test/shell-vfs-build.test.ts (revision pins), host/test/shell-lazy-archive-inputs.test.ts, web-libs/kandelo-session/test/kandelo-session.test.ts (including the new touchControls fixture assertions), and tests/package-system/source-rootfs-shell-bridge.test.ts (the doom profile carries touchControls: true in both JSON sources).
  • The shell VFS image was rebuilt from source at revision 24 and a byte search of the decompressed image confirmed the doom profile carries "touchControls": true.
  • Manually verified in Chrome device emulation via ./run.sh browser: overlay layout portrait and landscape, d-pad movement, FIRE, MENU, tap-to-use on doors, tap-to-select in menus, theme popup reaching Dark mode, and launch-pane scrolling.
  • Not run: scripts/homebrew-main-shell-image-contract.ts (executes inside the package flow against a built image), conformance suites and benchmarks (no kernel, syscall, or hot-path changes), and Playwright.

Known unrelated failure: exact-linux-contract also fails on main because upstream Homebrew bumped ca-certificates to cacert-2026-08-13 and the checked-in homebrew/homebrew-native-compatibility-lock.json is behind; the lock refresh belongs in its own change on main.

@github-actions

github-actions Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Phase B-1 matrix build status — pr-1252-staging-run-31695608075-attempt-1

ABI v42. 0 built, 6 failed, 6 total.

Package Arch Status Sha
shell wasm32 failed
lamp wasm32 failed
nginx-php-vfs wasm32 failed
nginx-vfs wasm32 failed
node-vfs wasm32 failed
wordpress wasm32 failed

Auto-generated; replaced on each push. Raw data in the publish-status workflow artifact.

@mho22
mho22 marked this pull request as ready for review August 12, 2026 14:24
Keyboard-driven framebuffer demos were unusable on phones and tablets:
fbDOOM needs a keyboard, the modeset demo only listened for mouse
events, and a tap could strand the browser in pointer-lock mode with no
Esc key to leave it. The launch pane and theme popup also clipped or
trapped scrolling on small screens.

Add an on-screen touch overlay (d-pad, FIRE, MENU) gated by a new
presentation.touchControls demo-config field, make a tap on the fbDOOM
canvas send Enter and Space so menus select and doors open, drive the
modeset demo from pointer events so touch acts as a left-button mouse,
ignore pointer-capture requests from touch input, and fix the
small-screen layout of the launch pane, theme popup, and dock.

Bump the seven image-package revisions that declare demo-config.ts as a
build input, record the touch-enabled demo config digest in the
lazy-shell artifact lock, and regenerate the program index so every
row matches the committed tree.
@mho22
mho22 force-pushed the touch/framebuffer-demo-touch-controls branch from 7ee429f to 66db0d0 Compare August 13, 2026 11:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant