Repository navigation
Add Standard input actions and background popup tabs - #60
Conversation
Extract DOM input dispatch into input-actions with shared page helper serialization, add target-resolution and protocol-validator modules, scope selector resolution by frame, extend expect_download to history actions, rework the host protocol surface and RPC schemas, and make TypeScript and Python clients wait within the connection deadline for transient endpoint availability instead of replaying dispatched handshakes or mutations.
Chrome activates any window a page opens, so a click or press that triggered window.open or a new-window link stole focus during background task runs. The gesture now runs with window.open returning null and new-window anchor navigation cancelled; each requested http or https URL opens as an inactive tab owned by the acting task, next to the source tab, and the action result reports them in opened_tabs with opened, refused, or failed outcomes. The child has no window.opener, so script-written blank windows and unsupported schemes are refused and reported rather than half-executed. The prior focus-restore path remains for windows that escape this capture.
Chrome silently blocks every automatic download after the first one a page initiates, so a second expect_download action used to wait out its cursor with no Page.downloadWillBegin or downloadProgress events at all. Probing every available transport showed chrome.debugger sessions cannot opt into DevTools download behavior on any tested Chrome: Browser.setDownloadBehavior is absent from extension tab sessions, Page.setDownloadBehavior and a browser-target attach leave the throttle untouched, and overriding the site's automatic-downloads permission would be a persistent browser mutation outside task ownership. A download wait that expires while its cursor never began on a tab that already produced a download this page load now fails with multiple_download_throttle and names the recovery paths: allow automatic downloads for the site in Chrome settings, navigate the tab again, or expect one download per page load.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8b9108a657
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const cursor = !hasAmbiguityTombstone && candidates.length === 1 ? candidates[0] : undefined; | ||
| if (hasAmbiguityTombstone || candidates.length > 1) { | ||
| for (const candidate of candidates) candidate.state = "ambiguous"; | ||
| } | ||
| session.downloads.set(String(params.guid), { | ||
| startedAt: now, | ||
| status: "in_progress", | ||
| ...(typeof params.url === "string" ? { url: params.url } : {}), | ||
| ...(typeof params.suggestedFilename === "string" | ||
| ? { suggestedFilename: params.suggestedFilename } | ||
| : {}), | ||
| ...(cursor ? { cursor } : {}), |
There was a problem hiding this comment.
When Page.downloadWillBegin finds a unique cursor, the cursor is attached to the download but remains in the armed state until completion. Consequently, a slow download that exceeds the wait timeout is incorrectly reported as multiple_download_throttle because line 1373 still sees an armed cursor, and any intervening browser action can mark the already-correlated cursor ambiguous. Set the cursor to bound and record its GUID when this event is processed.
Useful? React with 👍 / 👎.
| z.object({ | ||
| kind: z.literal("press"), | ||
| ref, | ||
| key: z.enum(STANDARD_PRESS_KEYS), | ||
| }).strict(), |
There was a problem hiding this comment.
When pi.zod is present, makeExtension registers this strict schema rather than the updated piSchema, but these new branches permit only bare press actions. OMP therefore rejects valid new requests containing modifiers, frame_id, or expect_download, and it has no branches at all for hover, double_click, or context_click, so most headline actions cannot be invoked from OMP even though Core accepts them.
Useful? React with 👍 / 👎.
| if (mode === "screenshot") { | ||
| if (frameId !== undefined) { | ||
| throw Object.assign(new Error("Frame-scoped screenshots are not available"), { code: "frame_unavailable" }); | ||
| } |
There was a problem hiding this comment.
A screenshot request with frame_id is accepted by the updated JSON schema, Rust protocol, TypeScript SDK, Pi schema, and documentation, but this branch unconditionally rejects it with frame_unavailable. Clients can therefore construct a fully valid advertised request that can never succeed; either implement frame-scoped screenshot capture or remove frame_id from the screenshot contract.
Useful? React with 👍 / 👎.
![wolf-maintainer[bot]](https://avatars.githubusercontent.com/in/4804189?s=60&v=4){kind=small}
There was a problem hiding this comment.
P2 — broad cross-layer feature; explicit maintainer call, with blocking contract gaps before merge.
Blocking: OMP schema parity, advertised frame screenshots, selector-targeted select, and download cursor binding. Should-fix: native key activation and ARIA-disabled handling.
Focused input-action tests passed; OMP tests could not load the missing local typebox package.
Thanks for the coordinated protocol and reliability work.
| } | ||
| if (mode === "screenshot") { | ||
| if (frameId !== undefined) { | ||
| throw Object.assign(new Error("Frame-scoped screenshots are not available"), { code: "frame_unavailable" }); |
There was a problem hiding this comment.
blocking — frame_id is accepted for screenshot mode by the RPC schema, Rust/extension validators, TypeScript SDK, Pi schema, and the updated docs/mcp.md, but this branch unconditionally rejects every such valid request with frame_unavailable. The advertised frame-scoped screenshot path therefore cannot succeed. Either implement the capture or remove frame_id from screenshot mode across those public surfaces.
| const action = z.union([ | ||
| z.object({ kind: z.literal("click"), ref }).strict(), | ||
| z.object({ kind: z.literal("click"), selector }).strict(), | ||
| z.object({ |
There was a problem hiding this comment.
blocking — The strict OMP Zod surface only adds a bare press; it still rejects hover/double_click/context_click, press modifiers, frame_id, expect_download, dialog prompt_text, the new wait states/value/download cursor, and snapshot frame_id. pi-schema.ts exposes those additions, but OMP providers validate against DEFINITIONS, so most of the feature described by this PR is unavailable in OMP. Mirror the Core contract here and pin schema parity in packages/omp/test/extension.test.ts.
| if (hasAmbiguityTombstone || candidates.length > 1) { | ||
| for (const candidate of candidates) candidate.state = "ambiguous"; | ||
| } | ||
| session.downloads.set(String(params.guid), { |
There was a problem hiding this comment.
blocking — trackDownloadWillBegin() attaches the cursor to the tracked download but never transitions it from armed to the declared bound state (or records its guid). A following action therefore lets invalidatePendingDownloadAttribution() mark an already-started download ambiguous, and an in-progress download that times out is misreported as multiple_download_throttle because the cursor is still armed while session.downloads is non-empty. Bind the cursor when this event arrives and cover waiting/subsequent actions while the download is still in progress.
| "properties": { | ||
| "kind": { "const": "select" }, | ||
| "ref": { "$ref": "#/$defs/ref" }, | ||
| "frame_id": { "$ref": "#/$defs/frame_id" }, |
There was a problem hiding this comment.
blocking — The PR claims every element-target action accepts exactly one ref or unique selector, and StandardBrowserRuntime.act() now includes select in that shared target-resolution path, but this public schema still requires ref and has no selector. The Rust parser, extension validator, and SDK types repeat that restriction, so a selector-targeted select can never reach the implementation. Add the same ref/selector XOR contract here and across those callers.
| if (keyDownAccepted && keyPressAccepted) { | ||
| if (tabTarget) { | ||
| tabTarget.focus(); | ||
| } else if ((key === "Enter" || key === "Space") && isNativeActivationTarget(this)) { |
There was a problem hiding this comment.
should-fix — isNativeActivationTarget() includes checkbox and radio inputs, and this shared Enter/Space branch calls click() for both keys. Native keyboard behavior toggles those controls on Space, not Enter, so press with key: "Enter" now changes checked state unexpectedly. Split activation eligibility by key and add checkbox/radio regressions.
| } | ||
|
|
||
| function isEnabledFocusableWidget(target: Element): target is HTMLElement { | ||
| if (target instanceof HTMLElement === false || target.hasAttribute("disabled")) return false; |
There was a problem hiding this comment.
should-fix — isEnabledFocusableWidget() ignores aria-disabled="true". A focusable ARIA button/listbox can therefore receive keydown and run page handlers through press, although the click path and Tab candidate filtering both reject the same disabled state. Include the ARIA-disabled check before dispatch and cover a focusable custom widget.
1 participant
Summary
hover,double_click,context_click, andpressactions. Every element-target action, includingupload_file, takes exactly onerefor unique CSSselectorplus an optionalframe_id;pressaccepts the Standard navigation and editing keys with bounded modifiers.browser_waitwith frame-scoped selector states, selector value waits, and resumable download cursors;expect_downloadalso applies to history actions.clickandpressas inactive task-owned tabs instead of focused windows, reporting each inopened_tabs.multiple_download_throttleand its recovery paths.Verification
bun run workspace:typecheck,workspace:test,workspace:build, andextension:build.cargo fmt --checkandcargo test --workspaceforhost-rs; protocol schema, identity, permission, and forbidden-surface checks; Python SDK tests.press, wait, and finish against a live task tab.