RELOPS-2575: Test the chocolatey-free Chrome path on hw-alpha - #963
Merged
Merged
Conversation
Pins win11-64-24h2-hw-alpha (nuc13-018/071/134) to ronin_puppet#1404 relops-2575-chrome-without-chocolatey @ bff9baf6. Image unchanged. That branch installs and updates Chrome from Google directly (version from versionhistory.googleapis.com, MSI from dl.google.com) instead of the public chocolatey community feed, whose per-IP 429s escalate to Set-PXE from both the deploy-time puppet resource and maintainsystem's boot-time upgrade check. On 2026-09-17 that put 96 production nodes into a re-image loop. No other pool is touched; production stays on master f328b45d. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Jira: RELOPS-2575 · ronin: #1404
win11-64-24h2-hw-alphawin11-24h2-hw-20260908-172915)relops-2402-fleetbench→relops-2575-chrome-without-chocolateyc353d2b6→bff9baf6Nothing else moves. Production (
win11-64-24h2-hw,win11-64-24h2-hw-ref) stays onmaster@f328b45d; ref-alpha, perf-debug, perf-sheriff and relops1213 are untouched. pools.yml re-verified: 7 pools / 172 nodes / 0 duplicates.Why
Chrome is installed at deploy and upgraded at every boot through the public chocolatey community feed, and a transient feed failure escalates to
Set-PXEin both paths — the puppetpackageresource (429 → exit 1 → puppet 4/6 → bootstrap.ps1 → Set-PXE) andGet-LatestGoogleChromein maintainsystem (choco upgradenon-zero → Set-PXE, on healthy nodes outside any wave). On 2026-09-17, Chrome 154 publishing plus the #961 prod re-image wave tripped the feed's per-IP limit and put 96 production nodes into a re-image loop.ronin #1404 replaces both call sites with one script that reads the installed version from the registry, asks
versionhistory.googleapis.comwhat current stable is, and installs the samedl.google.comenterprise MSI the chocolatey package was downloading anyway — so Google-side traffic is unchanged and the community feed leaves the Chrome path entirely. A failed refresh logs a warning and lets the worker start instead of re-imaging it.What to watch on the three nodes
Update-GoogleChrome :: installed … / current stable …in Papertrail, then Chrome landing at the API's current versionbootstrap_stage=complete, worker-runner Running, nodes claiming tasksSet-PXEanywhere in the Chrome path, and nochocoin the Chrome path at allNote that ronin #1404 also has
hardware_observabilitycommented out of the alpha role for the duration of this test, so fleetbench's 900s pre-worker-runner gate doesn't slow each iteration. Both that and this pin get reverted before #1404 merges.