fix(stats): persist tool events immediately to close ROI/Home divergence - #1066
fix(stats): persist tool events immediately to close ROI/Home divergence#1066cedric013 wants to merge 4 commits into
Conversation
stats::record buffered events in a process-local STATS_BUFFER flushed only every 30s or on graceful shutdown, so abrupt MCP stdio restarts dropped unflushed stats while the append-only savings ledger kept every event. This let ROI (verified ledger) exceed Home (estimated stats) by a large factor. Persist each stats::record and stats::adjust_savings immediately, matching the ledger's per-event durability, and reword the dashboard so Home and ROI read as independent accounting scopes where either total may legitimately be larger. Closes yvgude#1064
Only advance the in-memory baseline after acquiring the cross-process lock and successfully replacing stats.json. This prevents an immediate flush from silently marking an event persisted when lock contention or I/O failure actually dropped the write. Add a regression test covering lock contention and retry.
3e231d7 to
aba5be9
Compare
|
Fixed the PR-specific durability bug and rebased onto current main. |
|
Follow-up: inspected the actual failed CI logs. Root causes were an unformatted pair of upstream OCLA files and an unused |
yvgude
left a comment
There was a problem hiding this comment.
Code Review: fix(stats): persist tool events immediately
Verdict: Changes Requested
The stats durability fix itself is correct, but this PR mixes 3 unrelated concerns:
1. Stats durability (the actual fix) — io.rs, mod.rs — Good:
write_to_disk()now returnsboolfor error propagationmerge_file_stats()chains write success viaOptionmaybe_flush()runs on every tool event, not just on timer- Tests cover baseline preservation
2. CI import fixes — compression_provider.rs, tool_lifecycle.rs — Already on main:
These are identical to the now-closed PR #1098. R5 integration already applied these fixes (commit 01b630409). This will create merge conflicts.
3. Dashboard text changes — cockpit-overview.js, cockpit-roi.js — Unrelated:
Copy changes ("of which" → removed, "since" → "recording since", ROI divergence note) are not about stats durability.
4. Content port fix — content_port.rs — Unrelated:
Removing unused _jailed binding is a separate clippy fix.
What to do
Please rebase on current main (the import fixes will auto-resolve as they are identical) and remove the unrelated dashboard text changes from this PR. The stats durability changes in io.rs and mod.rs are clean and ready to merge once isolated.
Alternatively: squash the unrelated changes into a separate commit so we can merge only the stats fix.
|
Merged to main via CLI rebase+merge. Import/CI commits were dropped (already on main). Only the stats durability fix + dashboard copy improvements remain. |
…ROI/Home divergence Closes #1064
What
Persist stats per event and reword the dashboard so Home (estimated) and ROI (verified) are presented as independent accounting scopes.
Closes #1064
Why
core/savings_ledgerappends every measured event immediately under a cross-process lock, whilecore/stats::recordbuffered events in a process-localSTATS_BUFFERand only flushed afterFLUSH_INTERVAL_SECS(30s) or on gracefulshutdown(). MCP stdio servers are frequently terminated without a graceful shutdown, so ledger events survived while buffered stats were lost — Home then under-reported real savings and could sit far below the ROI ledger total. The existing UI copy ("of which … are signed", "auditable floor", "Why smaller here") wrongly impliedverified <= estimated, which is false for two independent accumulators.Changes
core/stats::recordandcore/stats::adjust_savingsnow persist each event immediately (merge_and_save), matching the ledger's per-event durability. The 30s cadence no longer gates whether a completed event reaches disk.cockpit-overview.jsbridge line,cockpit-roi.jsmethodology + hero banner) now frames Home and ROI as independent surfaces where either total may legitimately be larger, instead of a subset/floor relationship.Testing
cargo test --lib core::stats::tests— 10 passedcargo check --lib,cargo clippy --lib -- -D warnings— cleannode --checkon both edited components — cleanNotes
Behaviour-preserving for the split-data-dir aggregation (#830), corruption quarantine (#706) and daily-history bounds — only the flush timing changed from deferred to immediate.