Skip to content

Handle empty distributed partitions and DataFrame error aggregation - #977

Merged
wangyinz merged 3 commits into
masterfrom
codex/issue-835-distributed-boundaries
Aug 24, 2026
Merged

wangyinz merged 3 commits into
masterfrom
codex/issue-835-distributed-boundaries

Conversation

@wangyinz

@wangyinz wangyinz commented Aug 13, 2026 •

Copy link
Copy Markdown
Member

Summary

  • Return an empty result for empty distributed partitions.
  • Convert DataFrame rows in partition order and report all conversion errors once, in that same order.
  • Close internally created MongoDB cursors on success and failure paths.
  • Add Dask, Spark, and cursor-lifecycle regression coverage.

Closes #835

@codecov

codecov Bot commented Aug 13, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 73.10%. Comparing base (5ebd29d) to head (a80cdd0).

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #977      +/-   ##
==========================================
+ Coverage   73.07%   73.10%   +0.02%     
==========================================
  Files         182      182              
  Lines       36169    36173       +4     
  Branches     2532     2532              
==========================================
+ Hits        26432    26445      +13     
+ Misses       8832     8823       -9     
  Partials      905      905              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@wangyinz
wangyinz marked this pull request as draft August 14, 2026 06:03
@wangyinz
wangyinz marked this pull request as ready for review August 14, 2026 15:46
@wangyinz
wangyinz force-pushed the codex/issue-835-distributed-boundaries branch from af410d6 to a80cdd0 Compare August 24, 2026 05:52
@wangyinz

Copy link
Copy Markdown
Member Author

Review decision: ready after final CI. Empty partitions return an empty result; DataFrame conversion reports every row error once and in input order; internally created cursors close on all paths. The combined #976/#977 suite passes (24 passed, 4 Mongo-dependent tests skipped locally; full CI supplies MongoDB).

@wangyinz
wangyinz merged commit 387023f into master Aug 24, 2026
18 checks passed
@wangyinz
wangyinz deleted the codex/issue-835-distributed-boundaries branch August 24, 2026 06:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Handle empty distributed partitions and DataFrame error aggregation

1 participant