|
1 | 1 | # BROWSER.md — driving a real browser with `browser_execute` |
2 | 2 |
|
3 | | -Use the `browser_execute` tool to run JavaScript against a connected browser via the Chrome DevTools Protocol. The snippet runs in-process; `session` is bound to a long-lived CDP `Session` that survives across calls within the same bcode session. |
| 3 | +Use the `browser_execute` tool to run JavaScript against a connected browser via the Chrome DevTools Protocol. The snippet runs in-process; `session` is bound to a long-lived CDP `Session` that persists across calls within the same bcode session. You connect once, drive many. |
4 | 4 |
|
5 | 5 | **Locations:** |
6 | 6 |
|
7 | 7 | - Workspace (read/write your reusable scripts): `<projectRoot>/.bcode/agent-workspace/`. The bcode CLI runs from the project root, so `./.bcode/agent-workspace/foo.ts` works directly with the `read`/`write`/`edit` tools. |
8 | | -- Skills (read-only reference docs): `{{SKILLS_DIR}}/interaction-skills/` |
| 8 | +- Skills (read-only reference docs): `{{SKILLS_DIR}}/`. Run `read {{SKILLS_DIR}}/interaction-skills/` to list every available interaction skill before reading any one of them. |
9 | 9 |
|
10 | 10 | ## The model in one paragraph |
11 | 11 |
|
12 | | -`browser_execute` evaluates whatever JS you write against `session`. There is no auto-loaded library, no privileged file, no helper namespace — just `session` and standard JS globals. To reuse code from a previous snippet, save it as a `.ts` file under `./.bcode/agent-workspace/` (using the `write` tool) and `await import("/abs/path?t=" + Date.now())` it from a later snippet. The import takes an **absolute** path — construct it from `process.cwd()` inside the snippet, or shell out via the `bash` tool to get the project root. Same mechanism for a 5-line wrapper and a 500-line script. Skills under `{{SKILLS_DIR}}/interaction-skills/` are documentation you `read`, not modules you `import` — they teach you the CDP patterns; you write the code. |
| 12 | +`browser_execute` evaluates whatever JS you write against `session`. There is no auto-loaded library, no privileged file, no helper namespace — just `session` and standard JS globals. To reuse code from a previous snippet, save it as a `.ts` file under `./.bcode/agent-workspace/` (using the `write` tool) and `await import("/abs/path?t=" + Date.now())` it from a later snippet. The import takes an **absolute** path — construct it from `process.cwd()` inside the snippet. Same mechanism for a 5-line wrapper and a 500-line script. Skills under `{{SKILLS_DIR}}/` are documentation you `read`, not modules you `import` — they teach you the CDP patterns; you write the code. |
13 | 13 |
|
14 | 14 | ## Connecting |
15 | 15 |
|
16 | | -The first `browser_execute` call connects automatically by scanning OS-typical Chrome profile dirs for a `DevToolsActivePort` file (Chrome must be running with `--remote-debugging-port`). To attach explicitly: |
| 16 | +You always call `session.connect(...)` once at the start of your work. The `Session` is fresh on the first `browser_execute` call of an opencode session; subsequent calls reuse it. Three connection methods, in order of preference for typical tasks: |
| 17 | + |
| 18 | +**Way 1 — connect to the user's running Chrome (real profile, popup-gated).** Best when the task involves the user's actual logged-in sites. |
| 19 | + |
| 20 | +```js |
| 21 | +// Auto-detect the most-recently-launched Chrome with remote debugging enabled. |
| 22 | +await session.connect() |
| 23 | +``` |
| 24 | + |
| 25 | +The user must have ticked "Allow remote debugging for this browser instance" once at `chrome://inspect/#remote-debugging` (sticky per-profile), and on Chrome 144+ click "Allow" on the in-browser popup at first attach. If `connect()` fails with a 403/permission message, ask the user to do this. To wait for the click instead of erroring fast, pass `{ profileDir: "/abs/path", timeoutMs: 30000 }`. |
| 26 | + |
| 27 | +**Way 2 — connect to a Chrome you (or the user) launched with a debug port (isolated profile, no popups).** Best for unattended automation. |
| 28 | + |
| 29 | +```bash |
| 30 | +# User runs this once (or you run it via the `bash` tool): |
| 31 | +google-chrome --remote-debugging-port=9222 --user-data-dir=/tmp/bcode-chrome |
| 32 | +``` |
| 33 | + |
| 34 | +```js |
| 35 | +await session.connect({ wsUrl: "ws://127.0.0.1:9222/devtools/browser" }) |
| 36 | +// or, if you know the profile dir: |
| 37 | +await session.connect({ profileDir: "/tmp/bcode-chrome" }) |
| 38 | +``` |
| 39 | + |
| 40 | +The `--user-data-dir` must NOT be Chrome's platform default (`%LOCALAPPDATA%\Google\Chrome\User Data` on Windows, `~/Library/Application Support/Google/Chrome` on macOS, `~/.config/google-chrome` on Linux) — Chrome 136+ silently no-ops the port flag in that case. |
| 41 | + |
| 42 | +**Way 3 — provision and connect to a Browser Use cloud browser.** Best when the user can't see the browser, you need a clean profile, geo-located proxy, or fingerprint isolation. Read `{{SKILLS_DIR}}/cloud-browser.md` for the full pattern (provision, stop, swap profile/proxy). Briefly: |
17 | 43 |
|
18 | 44 | ```js |
19 | | -await session.connect({ profileDir: "/abs/path/to/Chrome/Default" }) |
20 | | -// or |
21 | | -await session.connect({ wsUrl: "ws://127.0.0.1:9222/devtools/browser/<id>" }) |
22 | | -// or for a Browser Use cloud browser, call the `browser_open_cloud` tool first. |
| 45 | +const r = await fetch("https://api.browser-use.com/api/v3/browsers", { |
| 46 | + method: "POST", |
| 47 | + headers: { "X-Browser-Use-API-Key": process.env.BROWSER_USE_API_KEY, "Content-Type": "application/json" }, |
| 48 | + body: "{}", |
| 49 | +}) |
| 50 | +const { id, cdp_url, live_url } = await r.json() |
| 51 | +await session.connect({ wsUrl: cdp_url }) |
| 52 | +console.log("liveUrl for the user to watch:", live_url) |
23 | 53 | ``` |
24 | 54 |
|
25 | | -After connect, attach to a page target: |
| 55 | +Requires `BROWSER_USE_API_KEY` in the environment (the user should have set this before launching bcode). If absent, tell the user to get a key at https://browser-use.com and `export BROWSER_USE_API_KEY=...`. |
| 56 | + |
| 57 | +## Attaching to a target |
| 58 | + |
| 59 | +After `connect()`, attach to a page target before driving the browser: |
26 | 60 |
|
27 | 61 | ```js |
28 | 62 | const targets = (await session.Target.getTargets({})).targetInfos |
@@ -65,7 +99,18 @@ const { data } = await session.Page.captureScreenshot({ format: "png" }) |
65 | 99 | // data is base64; write with the `write` tool or process in JS. |
66 | 100 | ``` |
67 | 101 |
|
68 | | -For the full menu of UI mechanics — dropdowns, dialogs, iframes, shadow DOM, uploads, scrolling, screenshots-with-highlights — read the relevant skill: `{{SKILLS_DIR}}/interaction-skills/<topic>.md`. |
| 102 | +For the full menu of UI mechanics — dropdowns, dialogs, iframes, shadow DOM, uploads, scrolling, screenshots-with-highlights — list `{{SKILLS_DIR}}/interaction-skills/` to see all available topics, then read the relevant one. |
| 103 | + |
| 104 | +## Switching browsers mid-session |
| 105 | + |
| 106 | +You own the connection. To swap: |
| 107 | + |
| 108 | +```js |
| 109 | +await session.close() |
| 110 | +await session.connect({ /* new opts */ }) |
| 111 | +``` |
| 112 | + |
| 113 | +Cloud cleanup is your responsibility — if you're done with a cloud browser, stop it explicitly (see `{{SKILLS_DIR}}/cloud-browser.md` for the PATCH call). Otherwise it persists until your API quota or BU's idle timer reclaims it. |
69 | 114 |
|
70 | 115 | ## Reusing code: write to the workspace, import from snippet |
71 | 116 |
|
@@ -110,4 +155,5 @@ Cache-bust (`?t=${Date.now()}`) is your responsibility: without it, edits to the |
110 | 155 | - **`session.Page.navigate` hangs forever** → the page is showing a native dialog. Use `session.Page.handleJavaScriptDialog({ accept: true })` to dismiss. |
111 | 156 | - **Selectors don't find elements that you can see** → likely an iframe or shadow DOM. Read `{{SKILLS_DIR}}/interaction-skills/iframes.md` or `shadow-dom.md`. |
112 | 157 | - **Actions silently no-op** → the page is mid-load. After `Page.navigate`, await `session.waitFor("Page.loadEventFired")` before driving inputs. |
113 | | -- **Connection refused or 403 on connect()** → Chrome wasn't started with `--remote-debugging-port`, or the user hasn't clicked "Allow" on the remote-debugging prompt. Pass `{ timeoutMs: 30000 }` to wait for the click. |
| 158 | +- **Connection refused or 403 on connect()** → Chrome wasn't started with `--remote-debugging-port`, or the user hasn't clicked "Allow" on the remote-debugging prompt. Pass `{ profileDir, timeoutMs: 30000 }` to wait for the click, or fall back to Way 2. |
| 159 | +- **Cloud `connect()` fails after a successful provision** → check that `cdp_url` came back in the POST response; some BU regions return `cdpUrl` (camelCase) — accept both. See `{{SKILLS_DIR}}/cloud-browser.md`. |
0 commit comments