Skip to content

Make batching stricter: errors become batch boundaries - #97

Open
destel wants to merge 1 commit into
mainfrom
batch-error-aware
Open

Make batching stricter: errors become batch boundaries#97
destel wants to merge 1 commit into
mainfrom
batch-error-aware

Conversation

@destel

@destel destel commented Jul 26, 2026

Copy link
Copy Markdown
Owner

Batch and Unbatch used to split the stream into separate value and error channels internally. That preserved the order of values relative to each other, and likewise the order of errors, but not the order between the two. Both functions now process the stream as a single sequence and preserve the original value-error interleaving:

  • An error acts as a batch boundary: the current partial batch is emitted first, then the error. Values that entered before an error come out before it.
  • Unbatch(Batch(s)) reproduces the input stream exactly, errors included.
  • Timeout semantics are unchanged: the deadline still counts from a batch's first item.

This change is bakwards compatible, and for most workloads there are no observable changes. Pipelines that fail fast on the first error, or only aggregate errors at the end, behave as before. Workloads where the relative value-error order matters, now have it deterministic and documented. The one visible difference: streams with mid-stream errors may produce smaller batches, since each error closes the current one.

@codecov

codecov Bot commented Jul 26, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.69%. Comparing base (766b6a3) to head (3cd3a8e).

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #97      +/-   ##
==========================================
+ Coverage   99.22%   99.69%   +0.46%     
==========================================
  Files          13       12       -1     
  Lines         648      647       -1     
==========================================
+ Hits          643      645       +2     
+ Misses          5        2       -3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant