Skip to content

chore(ci): run e2e tests in parallel#87

Open
janisz wants to merge 2 commits intomainfrom
parallel
Open

chore(ci): run e2e tests in parallel#87
janisz wants to merge 2 commits intomainfrom
parallel

Conversation

@janisz
Copy link
Copy Markdown
Contributor

@janisz janisz commented Mar 23, 2026

Description

Speedup e2e tests by running in parallel and optimising cache key.

Validation

CI

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Mar 23, 2026

❌ 2 Tests Failed:

Tests completed Failed Passed Skipped
361 2 359 12
View the full list of 2 ❄️ flaky test(s)
::policy 1

Flake rate in main: 100.00% (Passed 0 times, Failed 26 times)

Stack Traces | 0s run time
- test violation 1
- test violation 2
- test violation 3
::policy 4

Flake rate in main: 100.00% (Passed 0 times, Failed 26 times)

Stack Traces | 0s run time
- testing multiple alert violation messages 1
- testing multiple alert violation messages 2
- testing multiple alert violation messages 3

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Mar 23, 2026

E2E Test Results

Commit: 764b2a9
Workflow Run: View Details
Artifacts: Download test results & logs

=== Evaluation Summary ===

  ✓ cve-detected-workloads (assertions: 3/3)
  ✓ cve-cluster-does-exist (assertions: 3/3)
  ✓ cve-cluster-does-not-exist (assertions: 3/3)
  ✗ cve-nonexistent (assertions: 3/3)
      one or more verification steps failed
  ✓ list-clusters (assertions: 3/3)
  ✗ cve-multiple (assertions: 2/3)
      one or more verification steps failed
      - ToolsUsed: Required tool not called: server=stackrox-mcp, tool=, pattern=get_deployments_for_cve
  ✓ cve-clusters-general (assertions: 3/3)
  ✓ cve-detected-clusters (assertions: 3/3)
  ✓ cve-log4shell (assertions: 3/3)
  ✓ cve-cluster-list (assertions: 3/3)
  ✓ rhsa-not-supported (assertions: 2/2)

Tasks:      9/11 passed (81.82%)
Assertions: 31/32 passed (96.88%)
Tokens:     ~54058 (estimate - excludes system prompt & cache)
MCP schemas: ~12738 (included in token total)
Agent used tokens:
  Input:  14662 tokens
  Output: 20964 tokens
Judge used tokens:
  Input:  60796 tokens
  Output: 51416 tokens

@janisz
Copy link
Copy Markdown
Contributor Author

janisz commented Mar 23, 2026

Signed-off-by: Tomasz Janiszewski <tomek@redhat.com>
Comment thread e2e-tests/scripts/run-tests.sh Outdated
@janisz janisz requested a review from mtodor April 21, 2026 17:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants