Repository navigation
Conversation
The GitHub Pages address of the web app now redirects to https://benchup.io, so the published app connects to the bridge from that origin. The bridge only allowed https://benchup3d.github.io and answered it with HTTP 403, which would have kept the published app from ever pairing. Both origins are now allowed, with a test. The agent evaluation can now run against the project open in BenchUp: setup.py --live --token writes prompts that use `benchup ... --live`, and gives new cases an empty project to open. Round 7 reran cases 01-12 this way on the desktop and in Chrome, for item 3 of BenchUp3D#55; Safari and the local network access prompt from an https page are still to be checked.
Round 7 (BenchUp3D#87) reran the agent cases against the open app. This fixes what it found and confirms the fixes with a short live round on the seven affected cases. - benchup tools lists agent_activity, marked live_only, with its schema. Inside a batch, BenchUp skips it with a note instead of failing the batch; in file mode the error says it is live-only. - check_model reports a board that runs right through another even when a housing joint (tongue and groove, rabbet, groove) joins them. - render marks text notes that run off the image (cut_off, texts_cut_off), instead of leaving agents to spot it in the PNG. - add_dimension documents where an axis dimension's line is anchored. - The agent guide covers agent_activity as a call of its own, dry-run ids, lengthening past a notch, the routed rabbet first, drawer runners, a plinth without a bottom panel and organising a model into modules.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This fixes the tool and guide issues found by the live agent round in #87, and confirms the fixes with a short live round on the seven cases they came from.
This PR depends on #87. It builds on that branch, which adds
setup.py --liveand round 7, so its first commit is #87's. Once #87 is merged, only the second commit will remain here.Fixes
agent_activity:benchup toolsdid not list it, and when it appeared inside a--batchthe whole batch failed with "unknown tool".live_only: true, andbenchup tools agent_activityshows its schema.$Nreferences of the calls that follow still line up.check_model:tongue_groove,rabbetorgroove) joins the two. In round 7, a 1200 mm shelf housed into the divider was flagged only for the right side.render: text notes that run off the image are markedcut_offand listed intexts_cut_offwith a hint. Before, agents only noticed the problem by looking at the PNG.add_dimension: the description now says that, withaxis, the line goes through the midpoint of its points.agent_activityis a call of its own, and that dry-run ids may differ from the real run.Round 8
The round ran on the live desktop build. Each agent learned the tools only from the CLI. Every result was then checked with
check_modeland an iso render taken from the app.No agent hit the
agent_activityerror. The two remaining errors were deliberate trials. The items still open are listed inresults/round-8-live.md, among them reporting which face of a router profile receives the width and which receives the depth.Testing
cargo test --locked -p benchup-tools -p benchup-clipasses, including the evals regression test.cargo test --locked -p benchup-app --lib agent_linkpasses, including a new test for activity reports inside a batch.trunk servewithout errors.