Add Standard input actions and background popup tabs

CHANGELOG.md

@@ -1,6 +1,9 @@
 # Changelog
 
 ## Unreleased
+- Diagnosed Chrome's automatic multiple-download throttle instead of hanging on it. Chrome silently blocks every automatic download after the first one a page initiates, so a second `expect_download` action used to wait out its cursor with no lifecycle events at all; chrome.debugger sessions cannot opt into DevTools download behavior to lift that on any tested Chrome, and overriding the site's automatic-downloads permission would be a persistent browser mutation outside task ownership. A download wait that expires with an unstarted cursor on a tab that already downloaded now fails with `multiple_download_throttle` and names the recovery paths.
+- Stopped page popups from stealing focus. A `click` or `press` that calls `window.open` or follows a new-window link now opens each requested http or https URL as an inactive tab owned by the acting task instead of letting Chrome create and activate a window. Each action result reports those tabs in `opened_tabs` with an opened, refused, or failed outcome. The page receives `null` from `window.open`, so flows that need the returned window or `window.opener` are unsupported.
+- Added Standard `hover`, `double_click`, `context_click`, and `press` actions. Every element-target action, including `upload_file`, accepts exactly one `ref` or unique CSS `selector` plus an optional `frame_id` that scopes selector resolution. `press` accepts the Standard navigation and editing keys with optional unique modifiers; lowercase `a`, `z`, and `y` require Control or Meta. `browser_wait` adds frame-scoped selector states (attached, detached, visible, hidden, enabled), selector value waits, and download waits that resume after an opaque tab-owned cursor; `expect_download` also applies to history actions.
 - Added native OMP tool cards for terminals that speak the Tern Surface Protocol. Each AgentTab call shows its action and target in the card head with the outcome, tab, and page revision as facts. Approval, human-input, and uncertain states get a warning tone, a badge, and the lifecycle flow. Multi-action batches list their steps, screenshots appear as zoomable images, and the redacted structured result folds into a collapsed section. Typed values, staged tokens, and sensitive fields stay hidden as they are in the text renderer.
 - Added `bun run dev:deploy` for validated extension builds, backups, atomic receipt replacement, rollback, and bounded readiness checks. Reject symlinked writable paths and unsupported default reload platforms before deployment. Extension health checks now require a ready host and an actual extension round-trip.
 - Added a DOM editable fallback to accessibility snapshots. When Chrome's debugger-backed accessibility tree omits an editable element, for example a framework's pre-hydration composer textarea, full-tree snapshots lead with up to 40 `dom_fallback` textbox nodes minted from DOM query results, scanning past accessibility-covered and unsupported matches until the limit of accepted editables or a bounded inspection budget is reached. Each carries a normal revisioned ref that the existing fill, type, and click paths already accept; password inputs are skipped, partial (`root_ref`) snapshots are unchanged, the `max_nodes` budget and `truncated` flag now cover the combined result, and a fallback capture failure never fails the snapshot itself.

docs/mcp.md

@@ -63,22 +63,21 @@ For MCP, the capability store namespace is `mcp`; OMP uses `omp`; Pi uses `pi`.
 | Tool | Required input and behavior |
 |---|---|
 | `browser_open` | `mode: "create"` optionally accepts an `http`, `https`, or `about` URL, `background`, and `placement`. The default `placement: "task"` creates a tab in the task's existing window when possible. `placement: "new_window"` creates the first tab of an otherwise empty task in a separate unfocused normal window and rejects `background: false`. `mode: "adopt_active"` explicitly adopts only the currently active tab. The result includes task, tab, window, page-revision, and `automation_route` identifiers. |
-| `browser_snapshot` | Requires `tab_id`. Modes are `accessibility`, `text`, `html`, and `screenshot`. Only accessibility snapshots return revisioned node references; full-tree accessibility snapshots also append `dom_fallback` textbox nodes for editable elements that never entered Chrome's accessibility tree, so those refs still address the existing typed actions. Screenshot requests may select `png`, `jpeg`, or `webp`, set JPEG/WebP `quality`, bound dimensions with `max_width`/`max_height`, and cap compressed image size with `max_bytes`. Snapshots require the `full` automation route. |
-| `browser_act` | Requires `tab_id`, `expected_page_revision`, and one to 64 typed actions. Actions are click, type, fill, select, scroll, drag, navigate, history movement, reload, close, dialog decision, and staged file upload. No coordinate action exists in Standard mode. A `tab_only` route accepts explicit navigation, history movement, reload, and close only. Managed origin constraints disable history movement because Chrome does not expose its destination for authorization before navigation; use explicit navigation to an allowed URL instead. |
-| `browser_wait` | Requires `tab_id` and one load, URL, text, selector, network-idle, or download condition. `timeout_ms` is at most 120 seconds. A `tab_only` route accepts load and URL conditions only; network-idle and download attribution require the tab-scoped debugger connection available on the `full` route. |
+| `browser_snapshot` | Requires `tab_id`. Modes are `accessibility`, `text`, `html`, and `screenshot`; each may scope to an opaque CDP `frame_id`. Only accessibility snapshots return revisioned node references, each bound to its originating frame; full-tree accessibility snapshots also append `dom_fallback` textbox nodes for editable elements that never entered Chrome's accessibility tree, so those refs still address the existing typed actions. Screenshot requests may select `png`, `jpeg`, or `webp`, set JPEG/WebP `quality`, bound dimensions with `max_width`/`max_height`, and cap compressed image size with `max_bytes`. Snapshots require the `full` automation route. |
+| `browser_act` | Requires `tab_id`, `expected_page_revision`, and one to 64 typed actions. Actions are click, hover, double-click, context-click, press, type, fill, select, scroll, drag, navigate, history movement, reload, close, dialog decision, and staged file upload. Element-target actions accept exactly one `ref` or `selector` and optional `frame_id`; `press` accepts the Standard navigation/editing keys plus optional unique `Alt`, `Control`, `Meta`, and `Shift` modifiers. Lowercase `a`, `z`, and `y` require exactly one of Control or Meta, optionally Shift, never Alt. No coordinate action, generic key injection, clipboard shortcut, or cross-frame target is exposed in Standard mode. |
+| `browser_wait` | Requires `tab_id` and one load, URL, text, selector, value, network-idle, or download condition. Selector waits may set `attached`, `detached`, `visible`, `hidden`, or `enabled` state and scope to a `frame_id`; value waits require selector and expected value and may scope to a frame. A download wait may resume after only its opaque tab-owned `after` cursor. `timeout_ms` is at most 120 seconds. A `tab_only` route accepts load and URL conditions only; frame-scoped reads, value/selector state, network-idle, and download attribution require the tab-scoped debugger connection available on the `full` route. |
 | `browser_tabs` | Takes an empty object and lists only the current task's tabs, including each tab's `automation_route`. |
 | `browser_handoff` | Takes one of four operations. `request` requires a task tab, expected page revision, and prompt; optional `completion: {kind: "url"\|"selector", value}` and `timeout_ms` (default five minutes, reminder expiry only). It creates a `notice_id` scoped to the task and tab and returns immediately without pausing, focusing, or blocking any agent work. `status`, `resolve`, and `dismiss` take that `notice_id`. The agent tells the user in chat, verifies the page itself, then resolves or dismisses; a `resolve` against an unmet condition fails with `completion_not_met` and leaves the notice open. Works on a `tab_only` route, but selector completion requires the `full` route. |
 | `browser_commit` | Requires the staged token returned by a prior `commit_required` action and executes that one staged operation. On a `tab_only` route, only a staged close can execute; page-dependent staged actions require the `full` route. |
 | `browser_credentials` | `prepare` requires a task tab and expected page revision, then returns an opaque short-lived token when the default-enabled 1Password broker is available and one through three Login items match the host-derived current origin. Owner-only policy can disable it. `fill` consumes that token and selected username, password, or one-time-code field refs without returning any value. `next` advances to another bounded candidate. It never submits the form. |
 | `browser_finish` | Accepts `disposition: "auto" | "close" | "keep"` and optional task-owned `keep_tab_ids`. Automatic mode follows the popup cleanup policy: close task-created tabs while retaining adopted tabs, ask for confirmation, or retain all tabs. Successful finalization ungroups retained tabs, releases ownership, closes the Core connection, and returns closed and retained tab IDs. An open handoff notice never defers finalization; staged Commit review and other in-flight work defer it without destroying resumability. |
 
-For `click`, `type`, and `fill`, supply exactly one of `ref` or `selector`.
-A CSS selector is limited to 1–2048 characters and must match exactly one element;
-missing and ambiguous targets fail before mutation. The same task ownership,
-revision, sensitive-field, and Commit checks apply to either target form.
-Before Commit, the selector is resolved again and checked against the staged
-element identity. Execute against that checked element, never a later selector
-match. Existing `upload_file` selector first-match behavior is unchanged.
+For `click`, `hover`, `double_click`, `context_click`, `press`, `type`, `fill`, and `upload_file`, supply exactly one of `ref` or `selector`; `frame_id` scopes selector resolution when supplied. A CSS selector is limited to 1–2048 characters and must match exactly one element; missing and ambiguous targets fail before mutation. The same task ownership, revision, sensitive-field, and Commit checks apply to either target form. Before Commit, the selector is resolved again and checked against the staged element identity. Execute against that checked element, never a later selector match. Upload selectors now reject duplicate matches, including duplicates introduced between staging and Commit.
+A dialog action can provide `prompt_text` only when accepting a JavaScript prompt; the exact payload is included in review binding. Only click, press, double-click, context-click, navigation, history, and reload actions may set `expect_download: true`, which is also review-visible and fingerprint-bound. A successful opted-in action may return an opaque `download_cursor`; pass it only as `browser_wait` download `after`, never to another task or tab. AgentTab reports completion metadata, not a local download path. Gesture dispatch, site acceptance, and external side effects are not proven by accepting or staging a request.
+
+Chrome silently blocks every automatic download after the first one a page initiates, a throttle AgentTab cannot lift: chrome.debugger sessions are not allowed to opt into DevTools download behavior on any tested Chrome, and changing the site's automatic-downloads permission would be a persistent browser mutation outside task ownership. When a download wait expires while its cursor never began and the tab already produced a download this page load, the wait fails with `multiple_download_throttle` instead of a bare timeout: allow automatic downloads for the site in Chrome settings, navigate the tab again, or expect one download per page load.
+
+A `click` or `press` that asks the page to open a new window never lets Chrome create or activate that window. During the gesture, `window.open` returns `null` and a new-window anchor navigation is cancelled; AgentTab then opens each requested http or https URL as an inactive tab in the same task, next to the source tab. The action result lists them in `opened_tabs`, each with `outcome` `opened` (with `tab_id`), `refused` (an `about:blank` script-written window or an unsupported scheme), or `failed`. The child has no `window.opener`, so a flow that writes into the returned window or posts messages back to its opener is unsupported; `opener_severed: true` marks a request that expected an opener. Windows opened by a handler that stops click propagation or opens asynchronously after the gesture are outside this capture.
 
 
 Every existing-page mutation carries its expected page revision. If navigation or document replacement makes that revision stale, AgentTab rejects the operation rather than selecting a new target.

host-rs/crates/agenttab-host/src/guardrails.rs

@@ -203,8 +203,8 @@ impl Guardrails {
             MethodParams::Act(params) => {
                 for action in &params.actions {
                     match action {
-                        BrowserAction::Navigate { url } => self.authorize_url(url)?,
-                        BrowserAction::GoBack | BrowserAction::GoForward
+                        BrowserAction::Navigate { url, .. } => self.authorize_url(url)?,
+                        BrowserAction::GoBack { .. } | BrowserAction::GoForward { .. }
                             if self.has_origin_constraints() =>
                         {
                             return Err(RpcError::new(
@@ -624,6 +624,7 @@ mod tests {
             actions: vec![BrowserAction::UploadFile {
                 r#ref: Some("e1".into()),
                 selector: None,
+                frame_id: None,
                 files: vec![std::env::current_exe().unwrap().display().to_string()],
             }],
         });
@@ -693,7 +694,9 @@ mod tests {
         let history = MethodParams::Act(agenttab_protocol::BrowserActParams {
             tab_id: 1,
             expected_page_revision: 2,
-            actions: vec![BrowserAction::GoBack],
+            actions: vec![BrowserAction::GoBack {
+                expect_download: false,
+            }],
         });
         assert_eq!(
             guardrails
@@ -755,6 +758,7 @@ mod tests {
             actions: vec![BrowserAction::UploadFile {
                 r#ref: Some("e1".into()),
                 selector: None,
+                frame_id: None,
                 files: vec![link.display().to_string()],
             }],
         });
@@ -782,6 +786,7 @@ mod tests {
             actions: vec![BrowserAction::UploadFile {
                 r#ref: Some("e1".into()),
                 selector: None,
+                frame_id: None,
                 files: vec![hardlink.display().to_string()],
             }],
         });

host-rs/crates/agenttab-host/src/runtime.rs

@@ -1661,15 +1661,15 @@ fn is_tab_only_request(params: &MethodParams) -> bool {
             matches!(
                 action,
                 BrowserAction::Navigate { .. }
-                    | BrowserAction::GoBack
-                    | BrowserAction::GoForward
+                    | BrowserAction::GoBack { .. }
+                    | BrowserAction::GoForward { .. }
                     | BrowserAction::Reload { .. }
                     | BrowserAction::Close
             )
         }),
         MethodParams::Wait(params) => matches!(
             &params.condition,
-            WaitCondition::Load | WaitCondition::Url { .. } | WaitCondition::Download
+            WaitCondition::Load | WaitCondition::Url { .. } | WaitCondition::Download { .. }
         ),
         MethodParams::Handoff(BrowserHandoffParams::Request { .. }) => true,
         _ => false,
@@ -3452,7 +3452,7 @@ mod tests {
                     &json!({
                         "tab_id": 3,
                         "expected_page_revision": 7,
-                        "actions": [{"kind": "click", "ref": "e9"}]
+                        "actions": [{"kind": "click", "ref": "e9", "expect_download": false}]
                     }),
                 ),
             )