Skip to content

Run the script contract tests on Linux - #1011

Merged
pauljoda merged 1 commit into
mainfrom
ci/script-contracts-on-linux
Nov 6, 2026
Merged

pauljoda merged 1 commit into
mainfrom
ci/script-contracts-on-linux

Conversation

@pauljoda

@pauljoda pauljoda commented Nov 5, 2026 •

Copy link
Copy Markdown
Owner

What changed

The Release and repository contracts job ran on macos-latest. On 2026-11-05 GitHub twice never gave it a runner. In run 42077955348 (#975) and run 42103909261 (#978) it ended with "The job was not acquired by Runner of type hosted even after multiple attempts" and a notice about macOS arm64 capacity. Both times it was cancelled with no steps run and had to be rerun by hand. The xcode-27 build jobs in the same runs got runners within seconds. The first cancellation came after 24 minutes, past the 15-minute timeout, so a longer timeout or one that ignores queue time would not have helped.

  • The job now runs on ubuntu-latest and installs zsh with apt, as the core job does. Nothing the suite runs uses ripgrep (only validate-identity.sh does, through bootstrap.sh), so the job no longer installs it.
  • 41 of the 47 tests are portable. The other six need macOS plutil. Three in test_version_contract.py run check-version.sh --static, which reads the Info.plists with it. Three in test_xcode_cloud_configuration.py run the Xcode Cloud guards, which read the archive plists with it (the post-clone guard does so through check-version.sh --static). They now skip when plutil is missing.
  • I didn't add a small macOS job for those six, because it would wait for a macOS runner just the same. Instead, Build Chromium product host runs the whole suite after it prepares the build. That job already holds an xcode-27 runner and already lints Swift, so this costs about 15 seconds and no extra runner. When a change builds only iOS, Build iOS runs the suite instead, as it does for the Swift lint. A documentation-only change runs neither, and none of the six reads a file in the documentation area.
  • Documentation/RepositoryGuardrails.md now says where the suite runs and what it needs.

No app code changes, so there is no version bump or release note.

Validation

  • macOS, Python 3.14: python3 -m unittest discover -s Scripts/Tests ran 47 tests, OK, none skipped. The checkout is unchanged afterwards, so the build step that follows sees the same tree.
  • Linux, ubuntu:24.04 container (aarch64, Python 3.12.3, run as a non-root user) with only zsh and clang added and no ripgrep:
    • Before this change, 6 tests failed: command not found: plutil in check-version.sh, and an unset application path from ci_post_xcodebuild.sh.
    • After it, 47 tests ran, OK (skipped=6). The six skips are the plutil tests above.
  • With Python 3.9, two test_chromium_fork tests fail on hashlib.file_digest, so the guardrails page now says Python 3.11 or newer.
  • git diff --check, Scripts/check-public-source.py and Scripts/check-version.sh --static passed.
  • This pull request builds both apps, so its CI covers the Linux job and the new Chromium host step. The Build iOS fallback step, which runs only when the Mac builds are skipped, is not exercised here.

AI usage

  • AI tools helped produce this change.
  • I ran the crest-contribution skill (.agents/skills/crest-contribution/SKILL.md), followed it, and reviewed every generated line.

Boundaries

  • Space and browsing-profile isolation are preserved.
  • The macOS and mobile platform behavior remains intentionally native.
  • User-visible or architecture-significant changes add an explicit entry to Documentation/ReleaseNotes.json.
  • Every verified fix commit includes its patch bump and passes Scripts/check-version.sh --fix-commit.
  • I have signed off every commit as described in CONTRIBUTING.md.

Summary by CodeRabbit

  • Tests
    • Script contract checks now run in the Linux CI job and in the applicable macOS build jobs.
    • Tests requiring macOS tools are skipped on Linux and run on macOS runners instead.
    • Test requirements and platform coverage are documented, including the need for Python 3.11 or newer, patch, a C++ compiler, and plutil.

The "Release and repository contracts" job waited for a hosted
macos-latest runner on every pull request. Twice on 2026-11-05 GitHub
never assigned one ("The job was not acquired by Runner of type hosted
even after multiple attempts"), so the job was cancelled with no steps
run and had to be rerun by hand. The xcode-27 build jobs in the same
runs got runners within seconds.

- The job now runs on ubuntu-latest and installs zsh with apt. Nothing
  in the suite uses ripgrep, so the job no longer installs it.
- Six tests run check-version.sh or the Xcode Cloud guards, which read
  plists with plutil, a macOS-only tool. They skip when plutil is
  missing.
- The Chromium host build, which already holds a macOS runner and lints
  Swift, runs the whole suite after preparing the build, so those six
  still run on every change that builds the Mac app. When a change
  builds only iOS, the iOS build runs it instead, as it does for the
  Swift lint.
- RepositoryGuardrails.md says where the suite runs and what it needs.

Signed-off-by: Paul Davis <pauldavis101@gmail.com>
@coderabbitai

coderabbitai Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The script test job now runs on Ubuntu. Apple build jobs run the script contracts, and tests that require plutil skip when it is unavailable. The repository guardrails documentation reflects the runner coverage and prerequisites.

Changes: Script test routing

Layer / File(s) Summary
Guard tests for plutil availability
Scripts/Tests/test_version_contract.py, Scripts/Tests/test_xcode_cloud_configuration.py
Tests that require plutil now skip when the tool is unavailable.
Route script tests across CI runners
.github/workflows/ci.yml, Documentation/RepositoryGuardrails.md
The script test job runs on Ubuntu. Apple build jobs run the script contracts, with the iOS job covering cases where the macOS build is skipped. The documentation lists the runner coverage and prerequisites.

Estimated code review effort: 2 (Simple) | ~12 minutes

Change: Other

Merge Risk: Minimal · up to 27f69

The Linux job runs portable contract tests, while the Apple build runs the full suite for script changes. The reviewed routing preserves macOS-only coverage, so no actionable merge risk remains.

Pre-merge checks: Passed 5

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly describes the move of the script contract tests to Linux.
Docstring Coverage Passed No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 0 functions across 2 files. (2 skipped: 2 unsupported.)

Finishing Touches

Generate docstrings

  • Commit to this branch
  • Create a new PR
Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 8d4fd26c-c482-48d1-bddb-9cde62b6a211
Commits

Reviewing files that changed from the base of the PR and between db7c3dd and 27f692c.

Files selected for processing: 4

  • .github/workflows/ci.yml
  • Documentation/RepositoryGuardrails.md
  • Scripts/Tests/test_version_contract.py
  • Scripts/Tests/test_xcode_cloud_configuration.py

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 0 remain after this review.

@pauljoda
pauljoda merged commit bdf2259 into main Nov 6, 2026
13 checks passed
@pauljoda
pauljoda deleted the ci/script-contracts-on-linux branch November 6, 2026 15:16

This branch was previously deployed

1 inactive deployment
production — 27f692c6 Deployed Nov 6, 2026 by pauljoda via Sign, notarize, and publish #519
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant