bug(consolidation): structured dedup path can receive retain-style payload {"facts": []}

[Lang-code-ops]

bug(consolidation): structured dedup path can receive retain-style payload {"facts": []}

Summary

During live consolidation recovery, the structured dedup path was observed receiving retain-style parsed content shaped like:

{"facts": []}

That payload is meaningful in retain/fact-extraction contexts, but it is incompatible with the dedup decision schema, which expects something shaped like _DedupDecision(action, text, reason).

Why this is a bug

This suggests a structured-output contract mismatch between the consolidation dedup caller and the parser/model output handling.

A retain-style wrapper should not be accepted as the final parsed payload for a consolidation dedup decision. If it does appear, it should be normalized or rejected as a recoverable mismatch instead of flowing into _DedupDecision validation and crashing the op.

Expected behavior

Consolidation dedup should only accept payloads matching the dedup decision contract, or should gracefully coerce/reject mismatched retain-style payloads before schema validation.

Actual behavior

A retain-style payload like {"facts": []} can reach the dedup decision validation path, after which _DedupDecision fails because action is missing.

Live evidence

Recovery notes recorded this exact mismatch during re-drive of consolidation op 8610de21-c7da-49ba-bbdb-6a122dbd1f4e:

live input sometimes arrived as {"facts": []}
terminal error: 1 validation error for _DedupDecision / action / Field required

Source: 2026-06-24-hindsight-retain-failed-batch-recovery/README.md

Suggested fix direction

One of these should happen before _DedupDecision validation:

1. reject retain-style payloads as a recoverable structured-output mismatch and retry cleanly
2. normalize only truly compatible forms into a dedup decision
3. harden the parser/caller boundary so retain extraction wrappers never reach consolidation dedup parsing

Version

Observed on live ghcr.io/vectorize-io/hindsight:latest-slim / 0.8.3-slim runtime.

[nicoloboschi]

it looks like you're mixing up requests, what provider are you using?

Hindsight already does the right thing and fail/retry