Skip to content

fix(responses): use protocol-neutral keepalive before response starts - #8948

Closed
tufw95 wants to merge 3 commits into
diegosouzapw:release/v3.8.50from
tufw95:fix/responses-keepalive-lifecycle
Closed

fix(responses): use protocol-neutral keepalive before response starts#8948
tufw95 wants to merge 3 commits into
diegosouzapw:release/v3.8.50from
tufw95:fix/responses-keepalive-lifecycle

Conversation

@tufw95

@tufw95 tufw95 commented Jul 30, 2026

Copy link
Copy Markdown

Problem

The Responses endpoint emitted a synthetic rs_omniroute_keepalive reasoning item while waiting for a slow provider. That item had response.output_item.added and summary events but no response.output_item.done. Codex can therefore keep the turn in a running state (spinner/pause) after the real response emits response.completed.

Fix

Use the existing SSE comment heartbeat for the Responses startup frame. Comments keep the HTTP connection alive without creating a fake response/output item or polluting the Responses lifecycle. The route continues to forward real response.created and response.completed events unchanged.

Also add coverage for the observed nested response.failed + server_is_overloaded SSE shape so the HTTP Codex capacity fallback converts it to 503.

Verification

  • DISABLE_SQLITE_AUTO_BACKUP=true node --import tsx/esm --import ./open-sse/utils/setupPolyfill.ts --test tests/unit/early-stream-keepalive.test.ts (17 passing)
  • node --import tsx/esm --test tests/unit/early-stream-keepalive.test.ts tests/unit/codex-sse-capacity-fallback.test.ts (21 passing)
  • git diff --check

@tufw95
tufw95 requested a review from diegosouzapw as a code owner July 30, 2026 04:24
@tufw95
tufw95 force-pushed the fix/responses-keepalive-lifecycle branch from 018ccc7 to c2ff95a Compare July 30, 2026 04:24
@tufw95

tufw95 commented Jul 30, 2026

Copy link
Copy Markdown
Author

Follow-up commit bfb0901b adds bounded Codex account rotation for transient upstream 5xx overloads.

Evidence: NAS logs showed repeated Codex Our servers are currently overloaded responses normalized by Omni to HTTP 502; v3.8.49's direct Codex failover branch only rotated on 429, so no same-request CODEX_FAILOVER occurred. This patch rotates only codex requests on 429/502/503/504 (max existing 3 attempts), excludes failed connection IDs, records a per-model temporary scope cooldown, and preserves non-Codex and arbitrary 5xx behavior. The existing SSE peek still maps nested response.failed/server_is_overloaded 200 streams to 503.

Focused tests:

  • node --test --import tsx tests/unit/codex-failover.test.ts tests/unit/codex-sse-capacity-fallback.test.ts — 15 passed
  • git diff --check — passed

Not merged or deployed.

@tufw95
tufw95 force-pushed the fix/responses-keepalive-lifecycle branch from bfb0901 to 2269b20 Compare July 30, 2026 04:58
@tufw95

tufw95 commented Jul 30, 2026

Copy link
Copy Markdown
Author

Amended follow-up commit to 2269b20 (same patch; removed unused helper parameter). Tests unchanged: 15 focused Codex tests pass.

@tufw95 tufw95 closed this by deleting the head repository Aug 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant