Skip to content

OCPBUGS-101781: Isolate OCP-42982 kubeconfig writes - #2337

Merged
openshift-merge-bot[bot] merged 1 commit into
openshift:mainfrom
not-stbenjam:agent/isolate-oc-42982-kubeconfig
Aug 4, 2026
Merged

OCPBUGS-101781: Isolate OCP-42982 kubeconfig writes#2337
openshift-merge-bot[bot] merged 1 commit into
openshift:mainfrom
not-stbenjam:agent/isolate-oc-42982-kubeconfig

Conversation

@not-stbenjam

@not-stbenjam not-stbenjam commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

What changed

Pass --skip-config-write when OCP-42982 creates its explicit p42982-1 project. The project is still created, and the test already supplies -n p42982-1 to the subsequent command.

Root cause

openshift-tests runs extension tests concurrently with the same injected KUBECONFIG. OCP-42982 used oc new-project p42982-1 without --skip-config-write, so oc rewrote that shared file to select a context named p42982-1. Other test processes could load the kubeconfig during that update and fail before their test body with either:

invalid configuration: context was not found for specified context: p42982-1/...

or the same error plus cluster has no server defined.

The 5.0 CI stability study found 79 such observations across 72 builds from April 7 through July 24, 2026, spanning AWS, Azure, GCP, and bare metal. Every observed dangling context was named p42982-1, while the victim test varied; the long-running OCP-42982 test overlapped the sampled failures.

Impact

Retained JUnit was streamed for all 72 affected builds. Fifty-seven builds had a passing retry, 15 had at least one unrecovered context failure, and 6 failed Prow jobs had no independent blocking failure. The conservative historical counterfactual is therefore 6 additional green Prow jobs; no aggregate parent is additionally counted because each affected parent retained an unrelated aggregation failure.

Validation

GOTOOLCHAIN=auto GOMAXPROCS=2 go test -p=2 -run '^$' ./test/e2e

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: LGTM mode

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Aug 1, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@not-stbenjam: This pull request explicitly references no jira issue.

Details

In response to this:

What changed

Pass --skip-config-write when OCP-42982 creates its explicit p42982-1 project. The project is still created, and the test already supplies -n p42982-1 to the subsequent command.

Root cause

openshift-tests runs extension tests concurrently with the same injected KUBECONFIG. OCP-42982 used oc new-project p42982-1 without --skip-config-write, so oc rewrote that shared file to select a context named p42982-1. Other test processes could load the kubeconfig during that update and fail before their test body with either:

invalid configuration: context was not found for specified context: p42982-1/...

or the same error plus cluster has no server defined.

The 5.0 CI stability study found 69 such non-RT observations across 63 builds from April 7 through July 24, 2026, spanning AWS, Azure, GCP, and bare metal. Every observed dangling context was named p42982-1, while the victim test varied; the long-running OCP-42982 test overlapped the sampled failures.

Impact

Retained JUnit was streamed for all 63 affected builds. Forty-eight builds had a passing retry, 14 had at least one unrecovered context failure, and 6 failed Prow jobs had no independent blocking failure. The conservative historical counterfactual is therefore 6 additional green Prow jobs; no aggregate parent is additionally counted because each affected parent retained an unrelated aggregation failure.

Validation

GOTOOLCHAIN=auto GOMAXPROCS=2 go test -p=2 -run '^$' ./test/e2e

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 1, 2026
@openshift-ci

openshift-ci Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Hi @not-stbenjam. Thanks for your PR.

I'm waiting for a openshift member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Tip

We noticed you've done this a few times! Consider joining the org to skip this step and gain /lgtm and other bot rights. We recommend asking approvers on your previous PRs to sponsor you.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-ci openshift-ci Bot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Aug 1, 2026
@coderabbitai

coderabbitai Bot commented Aug 1, 2026

Copy link
Copy Markdown

Walkthrough

The quota test now passes --skip-config-write when it creates project p42982-1.

Changes

Quota test update

Layer / File(s) Summary
Project creation option
test/e2e/cli.go
The quota test adds --skip-config-write to the new-project invocation for p42982-1.

Estimated code review effort: 1 (Trivial) | ~2 minutes

🚥 Pre-merge checks | ✅ 14 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Test Structure And Quality ⚠️ Warning The OCP-42982 It block has multiple Expect(err).NotTo(HaveOccurred()) assertions without failure messages, which violates the assertion-message requirement. Add meaningful operation-specific messages to each assertion, and fail when quota output lacks the expected units instead of only logging a warning.
✅ Passed checks (14 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed The PR changes only a CLI argument. The affected Ginkgo title remains a static string and contains no runtime-generated value.
Microshift Test Compatibility ✅ Passed The diff changes one argument in an existing OCP-42982 Ginkgo test; it adds no new It, Describe, Context, or When test, so this check is not applicable.
Single Node Openshift (Sno) Test Compatibility ✅ Passed The PR changes one command in existing test/e2e/cli.go; it adds no new Ginkgo test, so the SNO multi-node compatibility check is not applicable.
Topology-Aware Scheduling Compatibility ✅ Passed The commit changes only test/e2e/cli.go, adding --skip-config-write to oc new-project; it adds no manifests, operator code, controllers, or scheduling constraints.
Ote Binary Stdout Contract ✅ Passed The only change adds --skip-config-write to an oc command inside the OCP-42982 g.It callback; it adds no process-level stdout or logging write.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed The commit changes one existing OCP-42982 command by adding --skip-config-write; it adds no Ginkgo test or IPv4/external-network dependency, and the test already skips disconnected clusters.
No-Weak-Crypto ✅ Passed The only PR change adds --skip-config-write to an oc new-project test command; it introduces no weak crypto, custom crypto, or secret comparison.
Container-Privileges ✅ Passed The PR changes only a Go CLI argument. It adds no privileged, host namespace, SYS_ADMIN, root, or allowPrivilegeEscalation setting.
No-Sensitive-Data-In-Logs ✅ Passed The PR only adds --skip-config-write to an existing command; it adds no logging or sensitive value, and the existing username log is unchanged.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the issue and the main change: isolating kubeconfig writes during the test.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@stbenjam

stbenjam commented Aug 3, 2026

Copy link
Copy Markdown
Member

/ok-to-test

@stbenjam
stbenjam marked this pull request as ready for review August 3, 2026 17:11
@openshift-ci openshift-ci Bot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Aug 3, 2026
@openshift-ci
openshift-ci Bot requested review from ardaguclu and tchap August 3, 2026 17:11
@stbenjam stbenjam changed the title NO-JIRA: Isolate OCP-42982 kubeconfig writes OCPBUGS-101781.: Isolate OCP-42982 kubeconfig writes Aug 3, 2026
@stbenjam stbenjam changed the title OCPBUGS-101781.: Isolate OCP-42982 kubeconfig writes OCPBUGS-101781: Isolate OCP-42982 kubeconfig writes Aug 3, 2026
@openshift-ci-robot openshift-ci-robot added the jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. label Aug 3, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@not-stbenjam: This pull request references Jira Issue OCPBUGS-101781, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (5.0.0) matches configured target version for branch (5.0.0)
  • bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

What changed

Pass --skip-config-write when OCP-42982 creates its explicit p42982-1 project. The project is still created, and the test already supplies -n p42982-1 to the subsequent command.

Root cause

openshift-tests runs extension tests concurrently with the same injected KUBECONFIG. OCP-42982 used oc new-project p42982-1 without --skip-config-write, so oc rewrote that shared file to select a context named p42982-1. Other test processes could load the kubeconfig during that update and fail before their test body with either:

invalid configuration: context was not found for specified context: p42982-1/...

or the same error plus cluster has no server defined.

The 5.0 CI stability study found 79 such observations across 72 builds from April 7 through July 24, 2026, spanning AWS, Azure, GCP, and bare metal. Every observed dangling context was named p42982-1, while the victim test varied; the long-running OCP-42982 test overlapped the sampled failures.

Impact

Retained JUnit was streamed for all 72 affected builds. Fifty-seven builds had a passing retry, 15 had at least one unrecovered context failure, and 6 failed Prow jobs had no independent blocking failure. The conservative historical counterfactual is therefore 6 additional green Prow jobs; no aggregate parent is additionally counted because each affected parent retained an unrelated aggregation failure.

Validation

GOTOOLCHAIN=auto GOMAXPROCS=2 go test -p=2 -run '^$' ./test/e2e

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@ardaguclu

Copy link
Copy Markdown
Member

/lgtm
/approve

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Aug 3, 2026
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling required tests:
/test e2e-agnostic-ovn-cmd
/test e2e-aws-ovn
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-aws-ovn-upgrade

@openshift-ci

openshift-ci Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ardaguclu, not-stbenjam

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 3, 2026
@stbenjam

stbenjam commented Aug 4, 2026

Copy link
Copy Markdown
Member

/verified by CI

@openshift-ci-robot openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Aug 4, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@stbenjam: This PR has been marked as verified by CI.

Details

In response to this:

/verified by CI

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@stbenjam

stbenjam commented Aug 4, 2026

Copy link
Copy Markdown
Member

/override ci/prow/e2e-aws-ovn-serial-2of2

@openshift-ci

openshift-ci Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

@stbenjam: Overrode contexts on behalf of stbenjam: ci/prow/e2e-aws-ovn-serial-2of2

Details

In response to this:

/override ci/prow/e2e-aws-ovn-serial-2of2

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-ci

openshift-ci Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

@not-stbenjam: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-merge-bot
openshift-merge-bot Bot merged commit 9c7ccc4 into openshift:main Aug 4, 2026
15 checks passed
@openshift-ci-robot

Copy link
Copy Markdown

@not-stbenjam: Jira Issue Verification Checks: Jira Issue OCPBUGS-101781
✔️ This pull request was pre-merge verified.
✔️ All associated pull requests have merged.
✔️ All associated, merged pull requests were pre-merge verified.

Jira Issue OCPBUGS-101781 has been moved to the MODIFIED state and will move to the VERIFIED state when the change is available in an accepted nightly payload. 🕓

Details

In response to this:

What changed

Pass --skip-config-write when OCP-42982 creates its explicit p42982-1 project. The project is still created, and the test already supplies -n p42982-1 to the subsequent command.

Root cause

openshift-tests runs extension tests concurrently with the same injected KUBECONFIG. OCP-42982 used oc new-project p42982-1 without --skip-config-write, so oc rewrote that shared file to select a context named p42982-1. Other test processes could load the kubeconfig during that update and fail before their test body with either:

invalid configuration: context was not found for specified context: p42982-1/...

or the same error plus cluster has no server defined.

The 5.0 CI stability study found 79 such observations across 72 builds from April 7 through July 24, 2026, spanning AWS, Azure, GCP, and bare metal. Every observed dangling context was named p42982-1, while the victim test varied; the long-running OCP-42982 test overlapped the sampled failures.

Impact

Retained JUnit was streamed for all 72 affected builds. Fifty-seven builds had a passing retry, 15 had at least one unrecovered context failure, and 6 failed Prow jobs had no independent blocking failure. The conservative historical counterfactual is therefore 6 additional green Prow jobs; no aggregate parent is additionally counted because each affected parent retained an unrelated aggregation failure.

Validation

GOTOOLCHAIN=auto GOMAXPROCS=2 go test -p=2 -run '^$' ./test/e2e

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. verified Signifies that the PR passed pre-merge verification criteria

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants