fix(integration): harden inbound ingress idempotency and delivery durability#591
Merged
Merged
Conversation
…ability
- Include pluginId in the ingress dedup unique key (pluginId, instanceId, providerDeliveryId).
instanceId is only unique within a plugin, so two plugins sharing an instanceId string dropped
each other's deliveries as false duplicates. Portable DROP/CREATE index migration + entity index.
- Derive a deterministic delivery id (sha256 of pluginId/instanceId/route/rawBody) when the provider
sends no dedup header, instead of a random UUID that silently disabled dedup and BullMQ job
idempotency and caused duplicate sends on provider retries.
- IngressEnqueueService.enqueue now returns a {queued|dispatched|failed} outcome (still never throws);
redrive only retires a DLQ row when the replay was accepted, so a swallowed inline-dispatch failure
keeps the row redrivable instead of permanently losing the event.
- Make conversation-mapping upsert a race-safe insert-or-converge: a forward-key race converges by
updating; a genuine reverse-unique conflict throws a typed ConversationMappingConflict instead of an
uncaught QueryFailedError that DLQ'd the job. Extract a shared isUniqueViolation helper.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Several inbound-webhook (ingress) durability and idempotency gaps in the Integration Fabric.
Changes
(instanceId, providerDeliveryId), butinstanceIdis only unique within a plugin — two plugins sharing an instanceId string would drop each other's deliveries as false duplicates. Widened to(pluginId, instanceId, providerDeliveryId)via a portable DROP/CREATE unique-index migration (loosening only — no data loss) plus the matching entity index.sha256(pluginId · instanceId · route · rawBody)(no timestamp) so a retry dedups.IngressEnqueueService.enqueuenow returns a{queued|dispatched|failed}outcome (still never throws); redrive only marks a DLQ row handled when the replay was actually accepted, so a swallowed inline-dispatch failure keeps the row redrivable.findOne-then-savewas a TOCTOU that raised an uncaughtQueryFailedError(DLQ'ing the job) under a concurrent write or a reverse-unique collision. Now insert-or-converge: a forward-key race converges by updating; a genuine reverse-unique conflict throws a typedConversationMappingConflict. Extracted a sharedisUniqueViolationhelper.Verification
npm run build✓ ·npm test✓ (1849/1849, +4 new) · lint ✓ ·migration:showlists the new migration. New/updated tests cover the cross-plugin dedup key, deterministic id (retry dedups, different body differs), redrive-keeps-row-on-failure, and the reverse-unique conflict.