Skip to content

Add RetryUntilSuccessOrExhausted and NonRetryableError to internal/common/util - #5085

Open
mauriceyap wants to merge 3 commits into
masterfrom
dlq-ingester-2
Open

Add RetryUntilSuccessOrExhausted and NonRetryableError to internal/common/util#5085
mauriceyap wants to merge 3 commits into
masterfrom
dlq-ingester-2

Conversation

@mauriceyap

Copy link
Copy Markdown
Collaborator

Add a bounded-retry helper which retries performAction up to maxAttempts times. It calls onExhausted with the last error if the budget runs out. It short-circuits immediately if the error is wrapped in the new NonRetryableError.

This will be used in the ingestion pipeline to decide when to dead-letter a message after retrying it.

@mergify

mergify Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

  • Queue this pull request

@greptile-apps

greptile-apps Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Adds a bounded retry utility and sentinel-based non-retryable error handling.

  • Retries failed actions up to a configured maximum.
  • Stops immediately when an error wraps ErrNonRetryable.
  • Reports exhaustion with either the final action error or a descriptive error when no attempts are allowed.
  • Adds coverage for success, exhaustion, cancellation, non-retryable errors, and non-positive attempt budgets.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains; the non-positive budget path now supplies a descriptive error, and the sentinel-based non-retryable API removes the nil-cause dereference path.

Important Files Changed

Filename Overview
internal/common/util/retry.go Adds the bounded retry helper, sentinel-based short-circuiting, cancellation handling, and a descriptive zero-attempt exhaustion error; both previously reported defects are resolved.
internal/common/util/retry_test.go Adds focused tests covering successful retries, exhaustion, cancellation, sentinel wrapping, and non-positive attempt budgets.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Start retry helper] --> B{Context cancelled?}
    B -- Yes --> C[Return false]
    B -- No --> D{Attempts available?}
    D -- No --> E[Call onExhausted]
    D -- Yes --> F[Perform action]
    F --> G{Succeeded?}
    G -- Yes --> H[Return true]
    G -- No --> I{Error is non-retryable?}
    I -- Yes --> E
    I -- No --> J[Call onError]
    J --> B
    E --> K[Return false]
Loading

Reviews (7): Last reviewed commit: "ErrNonRetryable" | Re-trigger Greptile

Comment thread internal/common/util/retry.go
Comment thread internal/common/util/retry.go Outdated
@mauriceyap
mauriceyap changed the base branch from dlq-ingester to master August 3, 2026 14:15
@mauriceyap
mauriceyap force-pushed the dlq-ingester-2 branch 2 times, most recently from b995f3b to 74c6ce3 Compare August 4, 2026 10:16

@nikola-jokic nikola-jokic left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added few suggestions that I think might be useful ☺️

Comment thread internal/common/util/retry.go Outdated
Comment thread internal/common/util/retry.go
@datadog-armadaproject

datadog-armadaproject Bot commented Aug 5, 2026

Copy link
Copy Markdown

Pipelines

⚠️ Warnings

🚦 1 Pipeline job failed

CI | All jobs succeeded   View in Datadog   GitHub Actions

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: b5dfb49 | Docs | Datadog PR Page | Give us feedback!

…mmon/util

Add a bounded-retry helpter which retries `performAction` up to `maxAttempts` times. It calls `onExhausted` with the last error if the budget runs out. It short-circuits immediately if the error is wrapped in the new `NonRetryableError`.

This will be used in the ingestion pipeline to decide when to dead-leatter a message after retrying it.

Signed-off-by: Maurice Yap <mauriceyap@hotmail.co.uk>
Signed-off-by: Maurice Yap <mauriceyap@hotmail.co.uk>
Signed-off-by: Maurice Yap <mauriceyap@hotmail.co.uk>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants