Skip to content

Persist GridFS storage mode during waveform migration - #953

Merged
wangyinz merged 4 commits into
masterfrom
codex/issue-813-storage-mode
Aug 26, 2026
Merged

wangyinz merged 4 commits into
masterfrom
codex/issue-813-storage-mode

Conversation

@wangyinz

@wangyinz wangyinz commented Aug 12, 2026 •

Copy link
Copy Markdown
Member

Summary

  • persist storage_mode="gridfs" in the waveform document when update_data migrates samples from file storage
  • treat storage_mode="file" as authoritative even when an earlier failed migration left a stale gridfs_id
  • stage new GridFS data without deleting any pre-existing object; if the waveform reference update fails or matches no document, remove only the new blob and restore the caller's exact storage keys
  • preserve the original file-backed document and verify successful results through the public read_data path

Root cause

Database.update_data changed the in-memory object and wrote new samples to GridFS, but did not persist storage_mode. The first repair recognized only pristine file documents without a gridfs_id; it therefore could not recover the mixed storage_mode=file + stale gridfs_id state produced by the original bug. Using the normal overwrite path for that state would also delete the pre-existing GridFS object before the waveform reference became durable.

Validation

  • MongoDB 8.0.29 contract suite: 12 passed across TimeSeries/Seismogram, pristine/stale file states, success/exception/unmatched-update paths
  • contract suite plus neighboring TestDatabase::test_update_data: 13 passed
  • exact baseline 61b06b9 against the final suite: 10 failed, 2 passed
  • failure snapshots prove the waveform document, readable file samples, caller storage keys, and every pre-existing GridFS file/chunk remain unchanged while the newly staged blob is removed
  • Black 26.5.1, Python 3.12/3.13 py_compile, and git diff --check: passed

This remains a draft pending the repository-wide scientific/API re-audit.

Fixes #813

@codecov

codecov Bot commented Aug 12, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.55%. Comparing base (85eba33) to head (6a00a1f).

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #953      +/-   ##
==========================================
+ Coverage   74.53%   74.55%   +0.01%     
==========================================
  Files         183      183              
  Lines       36535    36544       +9     
  Branches     2540     2540              
==========================================
+ Hits        27232    27246      +14     
+ Misses       8395     8390       -5     
  Partials      908      908              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@wangyinz
wangyinz marked this pull request as draft August 14, 2026 06:01
@wangyinz
wangyinz marked this pull request as ready for review August 14, 2026 15:42
@wangyinz
wangyinz force-pushed the codex/issue-813-storage-mode branch from 5b3a3f1 to 0a952b8 Compare August 26, 2026 05:44
@wangyinz

Copy link
Copy Markdown
Member Author

Independent compatibility audit complete on the current #955/#1002 commit/rollback path. File storage remains authoritative beside a stale gridfs_id; success persists the new GridFS mode/reference without deleting that unrelated stale blob, and failure removes only the staged blob and restores the caller storage keys. Python 3.10/3.13, MongoDB, formatting, and Docker CI are fully green.

@wangyinz
wangyinz force-pushed the codex/issue-813-storage-mode branch from deb36b1 to 6a00a1f Compare August 26, 2026 06:52
@wangyinz
wangyinz merged commit f50a69c into master Aug 26, 2026
22 checks passed
@wangyinz
wangyinz deleted the codex/issue-813-storage-mode branch August 26, 2026 07:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Persist storage_mode when update_data migrates file samples to GridFS

1 participant