fix(page-controller): index new (*-prefixed) elements in getElementTextMap - #633
Open
JOhnsonKC201 wants to merge 1 commit into
Open
fix(page-controller): index new (*-prefixed) elements in getElementTextMap#633JOhnsonKC201 wants to merge 1 commit into
JOhnsonKC201 wants to merge 1 commit into
Conversation
…xtMap getElementTextMap matched interactive lines with /^\[(\d+)\]<.../, but flatTreeToString renders new elements with a leading '*' (`*[i]<...`), which the system prompt documents as the "new clickable elements". The anchored regex never matched those lines, so new elements got no entry in elementTextMap — and on the first updateTree(), when every element is new, the map was empty entirely. As a result clickElement/inputText fell back to the bare index in their result messages (e.g. "Clicked element (5).") instead of the descriptive line, losing the traceability the tool feedback is meant to provide. Allow an optional leading '*' in the regex so new elements are indexed the same as existing ones. Existing lines are unaffected. Adds unit tests for the new-element and indented cases.
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
getElementTextMap(packages/page-controller/src/dom/index.ts) builds the index → line map thatPageControlleruses to echo a descriptive result forclickElement/inputText. It matched interactive lines with:But
flatTreeToStringrenders new elements with a leading*:which the system prompt documents ("Elements tagged with
*[are the new clickable elements"). The^\[anchor never matches a*[...]line, so:updateTree()every element is new, so the map is empty entirely.PageControllerthen falls back to the bare index —Clicked element (5).instead ofClicked element (*[5]<a role=button>Get started />).— losing traceability for exactly the elements the agent most often acts on, and for the whole first extraction.Fix
Allow an optional leading
*in the regex (/^\*?\[(\d+)\]<[^>]+>([^<]*)/). Existing (non-*) lines are unchanged, and plain text still can't match because the[i]<…>structure is still required.Tests
Adds
packages/page-controller/src/dom/index.test.tscovering the existing-element, new-element (*-prefixed), and indented cases.npm testpasses across all packages; typecheck and eslint are clean.