OTA-2024: Add unit test CI job for openshift/agentic-skills

[jhadvig]

Summary

Add a unit presubmit job for openshift/agentic-skills that runs Python unit tests for the plc_lookup.py CLI tool (added in openshift/agentic-skills#35).

Changes

• ci-operator config: new unit test entry running python3 -m unittest discover against cluster-update/product-lifecycle/scripts/tests/
• presubmits: auto-generated by make ci-operator-prowgen

The job reuses the existing skill-scanner image (Python 3.11 + repo source) and skips on docs-only and evals-only changes.

Context

Part of OTA-2024 — evaluation work for AI-guided cluster updates. The unit tests cover the Product Life Cycle API CLI tool: status passthrough, phase extraction, package-to-product lookup with duplicate handling, fallback search, and API error handling (33 tests, 0.02s).

Test plan

• Unit tests pass locally: python3 -m unittest discover -s cluster-update/product-lifecycle/scripts/tests -v (33 pass)
• make ci-operator-config ran successfully
• make ci-operator-prowgen ran successfully

🤖 Generated with Claude Code

Summary by CodeRabbit

Updates OpenShift CI presubmit configuration for openshift/agentic-skills. Adds a unit job that runs python3 -m pytest --ignore=evals -v in the skill-scanner image. The job skips changes limited to non-impacting files, including evals/. The existing eval job is unchanged.

CI feedback reports that OTA-2024 is valid, but its openshift-5.1 target version does not match the expected 5.0.* or openshift-5.0.* target.

[openshift-ci-robot]

@jhadvig: This pull request references OTA-2024 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target either version "5.0." or "openshift-5.0.", but it targets "openshift-5.1" instead.

Details

In response to this:

Summary

Add a unit presubmit job for openshift/agentic-skills that runs Python unit tests for the plc_lookup.py CLI tool (added in openshift/agentic-skills#35).

Changes

• ci-operator config: new unit test entry running python3 -m unittest discover against cluster-update/product-lifecycle/scripts/tests/
• presubmits: auto-generated by make ci-operator-prowgen

The job reuses the existing skill-scanner image (Python 3.11 + repo source) and skips on docs-only and evals-only changes.

Context

Part of OTA-2024 — evaluation work for AI-guided cluster updates. The unit tests cover the Product Life Cycle API CLI tool: status passthrough, phase extraction, package-to-product lookup with duplicate handling, fallback search, and API error handling (33 tests, 0.02s).

Test plan

• Unit tests pass locally: python3 -m unittest discover -s cluster-update/product-lifecycle/scripts/tests -v (33 pass)
• make ci-operator-config ran successfully
• make ci-operator-prowgen ran successfully

🤖 Generated with Claude Code

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 4bea3300-5130-4da0-9511-040cfaa03d0e

📥 Commits

Reviewing files that changed from the base of the PR and between 6fd9109 and 3c5e96f.

⛔ Files ignored due to path filters (1)

• ci-operator/jobs/openshift/agentic-skills/openshift-agentic-skills-main-presubmits.yaml is excluded by !ci-operator/jobs/**

📒 Files selected for processing (1)

• ci-operator/config/openshift/agentic-skills/openshift-agentic-skills-main.yaml

🚧 Files skipped from review as they are similar to previous changes (1)

• ci-operator/config/openshift/agentic-skills/openshift-agentic-skills-main.yaml

---

Walkthrough

The OpenShift agentic-skills CI configuration installs pytest in the skill-scanner image and adds a unit job that runs pytest while ignoring evals/.

Changes

Agentic skills CI configuration

Layer / File(s)	Summary
Configure the unit test job ci-operator/config/openshift/agentic-skills/openshift-agentic-skills-main.yaml	Installs pytest and adds a unit job that runs verbose pytest, ignores evals/, and skips configured non-impacting changes.

Estimated code review effort: 1 (Trivial) | ~3 minutes

Suggested labels: rehearsals-ack

Suggested reviewers: harche

🚥 Pre-merge checks | ✅ 15

✅ Passed checks (15 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: adding a unit test CI job for openshift/agentic-skills.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names	✅ Passed	The PR changes only CI YAML and generated presubmit YAML; it adds no Ginkgo tests or It/Describe/Context/When titles, dynamic or otherwise.
Test Structure And Quality	✅ Passed	The PR changes only CI YAML and generated Prow configuration; it adds a pytest command and contains no Ginkgo It blocks or cluster test operations to review.
Microshift Test Compatibility	✅ Passed	The diff adds only CI YAML and generated presubmit configuration; it adds no Ginkgo tests or references to MicroShift-incompatible APIs, namespaces, or assumptions.
Single Node Openshift (Sno) Test Compatibility	✅ Passed	The pull request changes only YAML CI configuration and adds no Ginkgo e2e tests or multi-node cluster assumptions.
Topology-Aware Scheduling Compatibility	✅ Passed	The PR changes only CI configuration and generated presubmit job files; they add test execution and contain no deployment or topology-dependent scheduling constraints.
Ote Binary Stdout Contract	✅ Passed	The PR changes only CI YAML for a Python pytest job; no OTE binary, Go process code, Ginkgo setup, or stdout logging is introduced.
Ipv6 And Disconnected Network Test Compatibility	✅ Passed	The pull request changes only YAML CI configuration and adds a Python pytest job; it adds no Ginkgo e2e tests or IPv4/external-network test logic.
No-Weak-Crypto	✅ Passed	The PR changes only CI YAML and adds pytest; no MD5, SHA1, DES, RC4, Blowfish, ECB, custom crypto, or secret comparisons appear in the changed lines.
Container-Privileges	✅ Passed	The PR adds pytest and a unit job only; changed CI/Kubernetes files contain no privileged, host namespace, SYS_ADMIN, allowPrivilegeEscalation, securityContext, or explicit root declarations.
No-Sensitive-Data-In-Logs	✅ Passed	The changed files add CI configuration only; no logging statements or sensitive values appear, and secret entries are standard file-path references rather than credentials.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[jhadvig]

/pj-rehearse auto-ack

[openshift-merge-bot]

@jhadvig: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

[matzew]

/retest

[jhadvig]

/pj-rehearse auto-ack

[openshift-merge-bot]

@jhadvig: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

[Cali0707]

Maybe instead we can run python3 -m pytest --ignore=evals -v ?

I think this should autodiscover any unittest.TestCases classes, so we don't need to point to a specific set of unit tests for a single skill

[Cali0707]

/pj-rehearse

[openshift-merge-bot]

@Cali0707: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

[Cali0707]

@jhadvig you may need to install pytest:

/opt/app-root/bin/python3: No module named pytest

[jhadvig]

/pj-rehearse auto-ack

[openshift-merge-bot]

@jhadvig: your /pj-rehearse request was not processed because the request waited in queue for longer than 5 minutes. Please retry in a few minutes.

[openshift-merge-bot]

[REHEARSALNOTIFIER]
@jhadvig: the pj-rehearse plugin accommodates running rehearsal tests for the changes in this PR. Expand 'Interacting with pj-rehearse' for usage details. The following rehearsable tests have been affected by this change:

Test name	Repo	Type	Reason
pull-ci-openshift-agentic-skills-main-unit	openshift/agentic-skills	presubmit	Presubmit changed
pull-ci-openshift-agentic-skills-main-eval	openshift/agentic-skills	presubmit	Ci-operator config changed
pull-ci-openshift-agentic-skills-main-images	openshift/agentic-skills	presubmit	Ci-operator config changed

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

[jhadvig]

/pj-rehearse auto-ack

[openshift-merge-bot]

@jhadvig: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

[jhadvig]

/pj-rehearse pull-ci-openshift-agentic-skills-main-eval

[openshift-merge-bot]

@jhadvig: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

[Cali0707]

@jhadvig the eval job is currently failing for unrelated reasons, I don't think that should block here

[jhadvig]

@Cali0707 thank you
could you please add the necessary labels in that case ?

/pj-rehearse ack

[openshift-merge-bot]

@jhadvig: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

[Cali0707]

/lgtm
/approve

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Cali0707, jhadvig

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• ci-operator/config/openshift/agentic-skills/OWNERS [Cali0707]
• ci-operator/jobs/openshift/agentic-skills/OWNERS [Cali0707]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci]

@jhadvig: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/rehearse/openshift/agentic-skills/main/eval	3c5e96f	link	unknown	/pj-rehearse pull-ci-openshift-agentic-skills-main-eval

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.