Skip to content

Make GridFS waveform replacement failure-safe - #955

Merged
wangyinz merged 5 commits into
masterfrom
codex/issue-814-gridfs-overwrite
Aug 23, 2026
Merged

wangyinz merged 5 commits into
masterfrom
codex/issue-814-gridfs-overwrite

Conversation

@wangyinz

@wangyinz wangyinz commented Aug 12, 2026 •

Copy link
Copy Markdown
Member

Fixes #814.

Summary

  • Stage replacement samples under a new GridFS ID.
  • Switch public update_data and save_data(overwrite=True) references with compare-and-set.
  • Delete the old blob only after the new waveform reference is durable.
  • Keep the low-level GridFS helper stage-only.
  • Reject write_distributed_data(overwrite=True) before scheduler or storage execution because distributed bulk insert cannot safely replace an existing waveform reference.
  • Restrict rollback handling to ordinary exceptions.

Failure contract

A failed stage or waveform-reference update preserves the previous readable GridFS object. Cleanup targets only the newly staged object. Failure to delete an old object after commit is reported without reverting the committed new reference.

Verification

  • Black and Python compilation checks passed.
  • Focused distributed-overwrite rejection tests passed locally.
  • MongoDB-backed overwrite contracts remain covered by the PR test suite and CI.

@codecov

codecov Bot commented Aug 12, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 90.80460% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 72.09%. Comparing base (f1cc7e7) to head (019e225).

Files with missing lines Patch % Lines
python/mspasspy/db/database.py 90.58% 8 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #955      +/-   ##
==========================================
+ Coverage   72.06%   72.09%   +0.02%     
==========================================
  Files         182      182              
  Lines       35966    36014      +48     
  Branches     2532     2532              
==========================================
+ Hits        25918    25963      +45     
- Misses       9142     9145       +3     
  Partials      906      906              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@wangyinz
wangyinz marked this pull request as draft August 14, 2026 06:01

Copy link
Copy Markdown
Member Author

Draft decision required (renewed review)

Atomic and ensemble GridFS overwrite paths are repaired, but distributed bulk overwrite still lacks compare-and-set semantics for an existing waveform reference. Keep this PR draft until the distributed overwrite contract is designed and tested.

@wangyinz
wangyinz force-pushed the codex/issue-814-gridfs-overwrite branch from f007386 to 1e4e680 Compare August 22, 2026 20:59
@wangyinz
wangyinz marked this pull request as ready for review August 22, 2026 20:59
@wangyinz
wangyinz merged commit 26c8901 into master Aug 23, 2026
19 of 20 checks passed
@wangyinz
wangyinz deleted the codex/issue-814-gridfs-overwrite branch August 23, 2026 04:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make GridFS overwrite preserve the previous readable sample

1 participant