Repository navigation
Openers report reads stored opener ERDs instead of reducing every opener per request - #403
Merged
Merged
Conversation
A finished opener's ERD is stored by the worker that finished it, and an unfinished opener has none. The report now reads the stored rows and takes an unfinished opener's progress from the queue: a response group the queue never took needs no work, and one it took is solved when that branch is done. A word's response-group count is counted once from its stored decomposition and kept while the answer list is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PpacbZRqT46b4Duhxq5QNQ
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 75d34d1251
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…em solved queue add leaves out a word's already-solved groups, but also groups its filters skipped, and an unqueued group can be a loss. Each unfinished word is partitioned once from its stored decomposition; its unqueued groups are looked up in the cache, and only those still unsettled are looked up again on a later poll. A word is re-partitioned when its requests take more branches. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PpacbZRqT46b4Duhxq5QNQ
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The Openers report took 46–48 s per build on the live queue (2,760 openers), and longer under load. 93% of that was
_opener_erd_summaries: on every request it re-partitioned the answers for every opener in pure Python (ResponseCache(score_cache=None), which bypassed the stored decompositions) and looked up all ~311,000 response groups. Its in-memory cache was keyed on the cache file's size and modification time, which the running swarm changes every 0.5–2 s, so it was discarded before a build could finish. Once a build ran past the client's 60 s stuck-request timeout, the client retried while the abandoned server thread kept computing.opener_erd_by_policy(newScoreCache.stored_opener_erds, one query). A finished opener with no stored row shows no summary;reconcile-opener-erdsis what fills it.queue addleaves out groups that are already solved and also groups its filters (--max-branch-size,--pattern) skipped: only anexactgroup counts as solved, and alossmakes the opener infeasible.response_decomposition, using its direct branch keys (newERDQueue.opener_direct_branch_keys). The result is kept, and only groups still unsettled are looked up on later polls (none on the live queue today; its 26,425 unqueued multi-answer groups are allexact). A word is partitioned again when its requests take more branches._OPENER_ERD_SUMMARY_CACHE,_score_cache_file_signatureand the split between computing summaries for the whole list and computing them for one page.On the live queue a build now takes about 6 s (8 s for the first, which partitions every unfinished word), nearly all of it queue reads (
opener_rows, the branch totals, the membership rows).🤖 Generated with Claude Code
https://claude.ai/code/session_01PpacbZRqT46b4Duhxq5QNQ