feat: click-xy 和 eval-val,提升 UI 自动化健壮性 (#24)#25
Merged
Conversation
Addresses issue #24 lessons from browser-harness. - neo click-xy <x> <y>: bypass a11y tree, click at absolute viewport coordinates via CDP Input.dispatchMouseEvent. Fixes combobox options, shadow DOM, portals, cross-origin iframes where resolveRef fails. - neo eval-val <selector> <value>: set input/textarea/select value using the framework-safe native setter pattern. Works with React/Vue/Angular controlled inputs that ignore naive '.value = x'. Dispatches input + change events with bubbles:true. - Extract buildClickXyEvents() and buildEvalValExpression() as pure helpers + unit tests (+6 tests, 173 passed). - Update help text (header comment + runtime usage).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
背景
解决 #24 — 从 browser-harness 实践里总结出来的 Neo UI 自动化痛点:a11y tree 抽象遇到 React/Material 自定义组件就崩。
变更
P0:
neo click-xy <x> <y>绕开 a11y tree,直接走 CDP
Input.dispatchMouseEvent在视口坐标点击。解决 #24 的 Bug 1/3:<div role="combobox">的 dropdown 选项,resolveRef 拿不到稳定 objectIdP1:
neo eval-val <selector> <value>用 native setter + 事件派发 模式写入 input/textarea/select:
解决 React/Vue/Angular 受控输入忽略
el.value = x的问题——框架监听的是原生 setter 被调用后的事件链,直接赋值不会进入它们的 state。为什么不加自动 fallback(#24 的 P3)
保持行为可预测。让用户/agent 显式选
clickvsclick-xy、fillvseval-val——静默 fallback 会让排错变难。未做:keyboard 命令
neo press已经存在并支持 Enter/Tab/Escape/Arrow/Ctrl+a 等(我来之前就有了),无需重做。未做:ref 稳定性 / selector fallback
#24 P4/P5 需要 snapshot 输出格式改造,留给后续独立 PR。当下 click-xy + eval-val 已能兜住绝大多数 snapshot 失败场景。
实现要点
buildClickXyEvents(x, y)和buildEvalValExpression(selector, value)commands.click/commands.fill既有行为,只新增验证
单元测试
新增 6 条:
CLI 冒烟
真实 Chrome 端到端
测试页:
<input id='e' value='old'><button onclick="e.value='clicked'">影响范围
closes #24 (P0 + P1 部分)