Skip to content

Enable API retries by default in E2E tests to handle transient provider failures #3039

Closed

Description

@chmouel

Problem

E2E tests occasionally fail due to transient GitHub API errors (e.g., 502 from GraphQL when fetching .tekton files). When this happens, no PipelineRun is created and the test times out. A recent example is PR #3010 where TestGithubPullRequestCustomGitOpsCommandPrefix failed because GitHub returned a 502 "Unicorn" page during GetTektonDir.

Context

PR #2854 added configurable API retry support (enable-api-retry, api-retry-max-attempts, api-retry-max-wait-seconds) but the feature is disabled by default. This is the right default for production, but E2E environments would benefit from having retries enabled to avoid false-negative test failures caused by transient provider outages.

Proposal

Enable enable-api-retry: "true" in the PAC ConfigMap used by E2E tests. This would let the controller retry transient 5xx responses and rate-limit errors during test runs, reducing flaky failures that are unrelated to the code under test.

If the E2E results remain stable with retries enabled, we could consider making retries the default for all deployments in a future release.

Related

  • feat: add gitlab/github provider API retries #2854 — Added configurable Git provider API retries
  • SRVKP-12884 — Add configurable Git provider API retries to Pipelines as Code
  • SRVKP-12892 — Operator: expose PAC API retry settings in TektonConfig CR

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions