OPCT-353: fix: add UserAPI groups test to known failures filter

[rvanderp3]

The [sig-auth][Feature:UserAPI] users can manipulate groups test is a
permanent false-positive in OPCT environments. The test checks the
authenticated user's group membership, expecting either system:masters
or system:cluster-admins. OPCT runs tests using a ServiceAccount
(sonobuoy-serviceaccount) whose identity always includes
system:serviceaccounts groups instead.

Group membership is determined by the authentication layer (token
claims), not by RBAC bindings. A ServiceAccount can never be a member
of system:masters even though it has equivalent API permissions via
the opct-scc-privileged ClusterRole with wildcard rules.

Adding this test to the Filter5 (Known Failures) list prevents it
from appearing as a failure in partner reviews and CI results, reducing
false-positive noise for the Cert-Ops team and partners.

Reference: https://issues.redhat.com/browse/OPCT-353

Checklist

• Subject and description added to both, commit and PR.
• Relevant issues have been referenced.
• This change includes docs.
• This change includes unit tests.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 449a449b-2270-4646-8ed5-366617fe043e

📥 Commits

Reviewing files that changed from the base of the PR and between ed2e844 and 6b66c7f.

📒 Files selected for processing (2)

• internal/opct/summary/consolidated.go
• internal/opct/summary/consolidated_test.go

---

📝 Walkthrough

Summary by CodeRabbit

• Bug Fixes
  • Updated conformance result filtering to exclude an additional known false-positive UserAPI test failure.
  • Prevented this environment-specific failure from being incorrectly reported as a genuine test failure.
• Tests
  • Added coverage to verify known failures are excluded while unknown failures remain visible.
  • Added validation that the expected known-failure entries are reported correctly.

Walkthrough

The known-failure filter now recognizes the UserAPI groups conformance test signature. Unit tests verify filtering, retained failures, exclusion counts, and the expected known-failure list.

Changes

Known failure filtering

Layer / File(s)	Summary
Add UserAPI known failure and validate filtering internal/opct/summary/consolidated.go, internal/opct/summary/consolidated_test.go	The known-failure filter now includes the UserAPI groups test signature, with tests covering filtered failures, retained unknown failures, exclusion counts, and the complete known-failure list.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested reviewers: bshaw7, jcpowermac

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: adding the UserAPI groups test to the known-failures filter.
Description check	✅ Passed	The description is directly related to the change and explains the false-positive being added to known failures.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[rvanderp3]

/kind bug

[rvanderp3]

@mtulio I'm not sure this is what we were going for on this issue. i think we should be scoping the service account rather than skipping, right?

cc @bshaw7

[bshaw7]

Hello @rvanderp3 ,

i think we should be scoping the service account rather than skipping
--> For this specific test (UserAPI) Scoping the SA won't resolve the failure. Because it's an identity problem. The test at test/extended/user/basic.go:47-61 calls GET users/~ and does reflect.DeepEqual, expecting exactly ["system:authenticated", "system:masters"] or ["system:authenticated", "system:cluster-admins"].

I deployed OPCT on a live cluster and checked what users/~ returns for the real sonobuoy-serviceaccount:

bshaw-mac:cluster bshaw$  export API_URL=https://api.bshaw.splat.devcluster.openshift.com:6443

bshaw-mac:cluster bshaw$ SA_TOKEN=$(oc create token sonobuoy-serviceaccount -n opct --duration=10m)
bshaw-mac:cluster bshaw$ curl -sk -H "Authorization: Bearer $SA_TOKEN" "${API_URL}/apis/user.openshift.io/v1/users/~" | jq '{name: .metadata.name, groups: .groups}'
{
  "name": "system:serviceaccount:opct:sonobuoy-serviceaccount",
  "groups": [
    "system:authenticated",
    "system:serviceaccounts",
    "system:serviceaccounts:opct"
  ]
}
bshaw-mac:cluster bshaw$ 

I also tested replacing the wildcard opct-scc-privileged ClusterRole with a narrow, scoped role — users/~ output was identical. These groups are SA token claims injected by the API server token authenticator, not configurable or removable. RBAC scoping changes what the SA can do, not what it is.

This puts it in the same category as the other Filter5 entries ([sig-arch] External binary usage, [sig-mco] MCP complete upgrade) — permanent false-positive caused by the OPCT environment. A fix would require changing the upstream test in openshift/origin to accept SA-based identities. For now, the known-failure filter is the correct interim mitigation.

[rvanderp3]

/assign @bshaw7

[bshaw7]

/test all

[openshift-ci]

@bshaw7: No presubmit jobs available for redhat-openshift-ecosystem/opct@main

Details

In response to this:

/test all

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[bshaw7]

Bug: Filter5 string doesn't match actual test name in pipeline

I tested this PR against a real OPCT archive from the latest periodic job (periodic-ci-openshift-release-main-nightly-5.0-opct-platform-external-aws-ccm, build 2078077374410264576).

Finding

The filter string at consolidated.go:548 does not match the actual test name flowing through the filter pipeline. The filter uses exact hashmap lookup (hashExclusion[v]), so any mismatch = no filtering.

Filter entry in PR:

[sig-auth][Feature:UserAPI] users can manipulate groups [apigroup:user.openshift.io][apigroup:authorization.openshift.io][apigroup:project.openshift.io]

Actual test name in sonobuoy results (includes Suite suffix):

$ cat /tmp/opct-verify/plugins/20-openshift-conformance-validated/sonobuoy_results.yaml | grep -iA1 "users can manipulate groups"
    - name: '[sig-auth][Feature:UserAPI] users can manipulate groups [apigroup:user.openshift.io][apigroup:authorization.openshift.io][apigroup:project.openshift.io]
        [Suite:openshift/conformance/parallel]'

The YAML parser joins these two lines into one string with the [Suite:openshift/conformance/parallel] suffix. The filter entry is missing this suffix, so the exact match fails.

Reproduction

Built opct-fixed from this PR branch, ran report against the CI archive:

$ ./opct-fixed report *.tar --log-level debug 2>&1 | grep "Filter (KF)"
Filter (KF) results: plugin=20-openshift-conformance-validated in=filter(17) out=filter(17) filterExcluded(0)

17 in, 17 out, 0 excluded. The UserAPI test was not filtered. It still appears in failures:

$ ./opct-fixed results *.tar 2>&1 | grep -A 30 "Failed tests:" | head -60 | grep -i "UserAPI"
[sig-auth][Feature:UserAPI] users can manipulate groups [apigroup:user.openshift.io][apigroup:authorization.openshift.io][apigroup:project.openshift.io] [Suite:openshift/conformance/parallel]

Unit test gap

The unit test also has this issue -- it uses the same short name (without Suite suffix) for both the input failures and the filter, so the test passes but doesn't reflect the real data shape.

Fix options

1. Add the Suite suffix to the filter string to match the real test name
2. Change filter matching from exact (hashExclusion[v]) to prefix/substring matching -- more robust but changes behavior for all Filter5 entries

cc @rvanderp3 @mtulio

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from bshaw7. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment