From 95827e709e3d462ee4d79b24e85ad141a14441c0 Mon Sep 17 00:00:00 2001 From: "Guocheng(Eric) Song" <132118805+feichai0017@users.noreply.github.com> Date: Thu, 15 Oct 2026 14:25:13 +0800 Subject: [PATCH 1/2] perf: reduce DB view capture and transient COW debt Batch frozen root capture and persist named-tree epoch summaries at checkpoint instead of manufacturing dirty roots on reads. Retire detached versions against their actual visibility interval and prune unpersisted images only outside older checkpoint dependency frontiers. Preserve dirty WAL debt, escaped reader leases and GC pins. Cover epoch exhaustion, foreign-tree COW, checkpoint graph protection and synced-WAL recovery without a final checkpoint. --- ARCHITECTURE.md | 31 ++ docs/db-view-cow.md | 48 +++ src/api/db.rs | 116 +++++- src/api/tree.rs | 42 +- src/checkpoint/round.rs | 13 +- src/engine/walker/erase.rs | 8 +- src/engine/walker/insert.rs | 15 +- src/store/buffer_manager/mod.rs | 693 ++++++++++++++++++++++++++------ tests/snapshot.rs | 143 +++++++ 9 files changed, 952 insertions(+), 157 deletions(-) create mode 100644 docs/db-view-cow.md diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index b63e3a1..34ce515 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -396,6 +396,37 @@ objects through a bounded planner → I/O queue: maintenance fence still excludes topology changes. Pinned GUIDs return to the FIFO; crash leftovers are handled by an explicit full reachability GC. +Snapshot capture does not change that persistence boundary. `DB::view` copies +the frozen roots and registers their independent leases in one registry pass; +the DB epoch high-water is summarized on the catalog root during checkpoint +capture, rather than dirtying each named root on every read. Standalone trees +retain their live-root high-water publication. Both use the same header format. + +A detached version is protected only by live epochs in its closed visibility +interval (creation through detachment). It waits on the newest such epoch; +retirement reassigns only that epoch's waiters. Snapshots created after the +edge changed cannot extend the old version's lifetime. GC-owned root pins also +retain epoch protection, including after the last public read handle drops. + +At an idle checkpoint pipeline, dirty capture takes one retirement-set image +and omits all retired dirty versions together: an unreachable old parent must +not be written with a child that has already been elided. With older flushing +claims present, capture retains those versions for the older dependency graph. +Dirty capture and transient pruning share a short exclusion; pruning waits for +all flushing claims, not only claims on the candidate itself. Child mutations +publish WAL debt before releasing their write latch, and structural promotion +also transfers the detached child's debt to its rewritten parent. A nonempty +retired queue does not repeatedly wake the planner on every view drop; its +empty-to-nonempty transition wakes once, then normal cadence/pressure applies. + +There is one strictly in-memory exception to waiting for a clean frontier: +bounded batches of retired versions absent from the inner store may be +discarded without writing or deleting any store data. The successor inherits +the predecessor's unflushed WAL low-watermark. A GC-generation/delete fence +excludes checkpoint claims during cancellation, and any existing pin or +flushing claim defers the candidate. Persisted GUIDs always retain the +clean-frontier/full-reachability requirement above. + The background checkpointer has three threads: a planner/orchestrator, an I/O worker that executes complete dependency-ordered epochs from a bounded `crossbeam-channel`, and an independent cold-blob eviction sweep. `Drop` joins diff --git a/docs/db-view-cow.md b/docs/db-view-cow.md new file mode 100644 index 0000000..3c3699b --- /dev/null +++ b/docs/db-view-cow.md @@ -0,0 +1,48 @@ +# DB view capture and COW debt + +Scope: optimize DB::view capture and transient COW/reclamation cost; +keep the public read, atomic, acknowledgement, and persistence contracts unchanged. + +## Implementation and invariants + +1. Preserve the pre-change release executable and the existing workspace workload. +2. Batch snapshot registration under the coordinated tree freeze, avoid redundant + root initialization and empty delta flushes, and avoid dirtying named-tree + roots merely to persist the clock. DB checkpoints stage one catalog-root + epoch summary, preserving monotonicity and exhausted-clock reopen behavior; + DB open still checks all reachable frame epochs. Standalone trees retain + their root epoch high-water update. +3. Track an orphan's closed visibility interval, from its creation to detachment. + Queue it against the newest live epoch inside that interval. Later snapshots + cannot prolong its lifetime. Retiring an epoch examines only its own waiters. +4. Discard bounded batches of unreachable, never-persisted COW images only when + no pin, checkpoint claim, or deletion fence owns them. Preserve the replaced + image's dirty WAL low-watermark in its successor. Persisted GUIDs still wait + for a clean durable frontier or reachability GC; no inline disk deletion. + Earlier checkpoint parents can depend on a different retired child: exclude + pruning while any flushing claim remains, and share an exclusion with the + complete dirty capture. At an idle capture, omit the retirement-set image + atomically so dead parents cannot reference elided children. Structural + promotion also carries the child's WAL debt into the rewritten parent. + Wake the planner only on the reclaim queue's empty-to-nonempty transition. +5. GC-owned snapshot root pins retain epoch protection until the walk releases + them. Escaped views, ranges and cursors retain their existing leases and scan + fences. No on-disk format or durability policy changes. +6. Add targeted lifecycle, concurrent checkpoint, epoch exhaustion, duplicate + scope, and reopen tests, run the existing regression suite, then interleave + baseline/patched runs of the unchanged agent-workspace workflow. Report + throughput, tail latency, peak RSS, COW debt, disk footprint and final drain. + +## Deferred alternatives + +A new persistent root descriptor, per-tree frame ownership tags, and a new +on-disk format are not required for this iteration. Removing scan fences or +substituting point reads would weaken semantics and is not an optimization here. + +## Acceptance + +Full ordered-domain fingerprints must match across builds and reopen. Atomic +multi-tree captures must remain consistent. Pinned/escaped readers and in-flight +checkpoint images must never be discarded. Existing crash/replay and W2D tests +remain mandatory. Performance claims use process-level balanced A/B repetitions, +not the earlier baseline alone. diff --git a/src/api/db.rs b/src/api/db.rs index a48cde8..d065448 100644 --- a/src/api/db.rs +++ b/src/api/db.rs @@ -256,6 +256,8 @@ impl DB { // delta flush can serve. let epoch_recovery_start = std::time::Instant::now(); db.restore_epoch_high_water()?; + db.store + .set_snapshot_epoch_root(root_guid_for_tree_id(DB_CATALOG_TREE_ID)); db.open_stats.epoch_recovery_micros = epoch_recovery_start .elapsed() .as_micros() @@ -652,9 +654,18 @@ impl DB { { let view = { let _maintenance = self.maintenance_gate.enter_shared(); - let mut scoped = Vec::with_capacity(scopes.len()); + let mut scoped: Vec<(u64, String, &[u8], Tree)> = Vec::with_capacity(scopes.len()); for (name, prefix) in scopes { let name_bytes = validate_tree_name(name)?; + // Preserve the existing last-scope-wins behavior without + // copying/registering roots that HashMap::insert discarded. + if let Some((_, _, previous, _)) = scoped + .iter_mut() + .find(|(_, captured, _, _)| captured == name) + { + *previous = *prefix; + continue; + } let tree_id = self.catalog_lookup_live(name_bytes)? .ok_or_else(|| Error::TreeNotFound { @@ -675,12 +686,30 @@ impl DB { .map(|(_, gate)| gate.enter_exclusive()) .collect::>(); let _commit = self.commit_gate.enter_writer(); - for (tree_id, _, _, _) in &scoped { - self.store.flush_write_deltas_for_tree(*tree_id)?; - } + self.store + .flush_write_deltas_for_trees(gates.iter().map(|(id, _)| *id))?; + let roots = scoped + .iter() + .map(|(_, _, _, tree)| tree.prepare_snapshot_root()) + .collect::>(); + let first_epoch = match self.store.register_snapshots(&roots) { + Ok(first_epoch) => first_epoch, + Err(error) => { + for (guid, pin) in roots { + drop(pin); + self.store.discard_snapshot_root(guid); + } + return Err(error); + } + }; let mut trees = HashMap::with_capacity(scoped.len()); - for (_, name, prefix, tree) in scoped { - trees.insert(name, tree.snapshot_unlocked(prefix)?); + for (((_, name, prefix, tree), (guid, pin)), epoch) in + scoped.into_iter().zip(roots).zip(first_epoch..) + { + trees.insert( + name, + tree.snapshot_from_registered(prefix, guid, pin, epoch, true), + ); } DBView { trees } }; @@ -3352,6 +3381,81 @@ mod tests { ); } + #[test] + fn db_read_only_capture_clock_survives_checkpoint_reopen() { + let dir = tempdir().unwrap(); + let cfg = || { + let mut cfg = TreeConfig::new(dir.path()); + cfg.checkpoint.enabled = false; + cfg + }; + let high_water; + { + let db = DB::open(cfg()).unwrap(); + let tree = db.create_tree("objects").unwrap(); + tree.put(b"key", b"value").unwrap(); + db.checkpoint().unwrap(); + for _ in 0..32 { + db.view(&[("objects", b"")], |_| Ok(())).unwrap(); + } + high_water = db.store.current_epoch(); + assert_eq!(db.store.dirty_count(), 0); + assert!(db.store.has_unstaged_snapshot_epoch()); + db.checkpoint().unwrap(); + assert!(!db.store.has_unstaged_snapshot_epoch()); + } + let db = DB::open(cfg()).unwrap(); + assert_eq!(db.store.current_epoch(), high_water); + assert_eq!( + db.open_tree("objects") + .unwrap() + .snapshot(b"") + .unwrap() + .epoch(), + high_water + ); + } + + #[test] + fn db_view_does_not_dirty_named_roots_and_duplicate_scope_keeps_last_prefix() { + let db = DB::open(TreeConfig::memory()).unwrap(); + let tree = db.create_tree("objects").unwrap(); + tree.put(b"first/key", b"one").unwrap(); + tree.put(b"last/key", b"two").unwrap(); + db.checkpoint().unwrap(); + assert_eq!(db.store.dirty_count(), 0); + let before = db.store.current_epoch(); + let escaped = db + .view(&[("objects", b"first/"), ("objects", b"last/")], |view| { + let captured = view.tree("objects").unwrap(); + assert_eq!( + captured.get(b"last/key")?.as_deref(), + Some(b"two".as_slice()) + ); + assert!(matches!( + captured.get(b"first/key"), + Err(Error::OutsideViewScope { .. }) + )); + Ok(captured.clone()) + }) + .unwrap(); + assert_eq!( + db.store.current_epoch(), + before + 1, + "one unique root capture" + ); + assert_eq!( + db.store.dirty_count(), + 0, + "read-only capture must not manufacture checkpoint debt" + ); + tree.put(b"last/key", b"new").unwrap(); + assert_eq!( + escaped.get(b"last/key").unwrap().as_deref(), + Some(b"two".as_slice()) + ); + } + #[test] fn db_view_waits_for_checkpoint_delta_flush_before_registering_barrier() { let db = DB::open(TreeConfig::memory()).unwrap(); diff --git a/src/api/tree.rs b/src/api/tree.rs index 3e9465c..59511a6 100644 --- a/src/api/tree.rs +++ b/src/api/tree.rs @@ -2174,10 +2174,7 @@ impl Tree { prefix: &[u8], fence_live_writers: bool, ) -> Result { - use crate::store::STRUCTURAL_SEQ; - - let snap_root = engine::fresh_blob_guid(); - let root_pin = self.store.install_snapshot_root(snap_root, &self.root_pin); + let (snap_root, root_pin) = self.prepare_snapshot_root(); let epoch = match self.store.register_snapshot(snap_root, &root_pin) { Ok(epoch) => epoch, Err(error) => { @@ -2187,17 +2184,38 @@ impl Tree { } }; - // Persist the bumped epoch on the live root so a reopened tree - // restores `current_epoch` above every frame's `created_epoch`. - { + Ok(self.snapshot_from_registered(prefix, snap_root, root_pin, epoch, fence_live_writers)) + } + + pub(crate) fn prepare_snapshot_root(&self) -> (BlobGuid, Arc) { + let guid = engine::fresh_blob_guid(); + (guid, self.store.install_snapshot_root(guid, &self.root_pin)) + } + + pub(crate) fn snapshot_from_registered( + &self, + prefix: &[u8], + snap_root: BlobGuid, + root_pin: Arc, + epoch: u64, + fence_live_writers: bool, + ) -> Snapshot { + // Standalone open restores its clock from the live root. Named DB + // trees summarize their clock on the catalog root at checkpoint, + // rather than dirtying every captured root on every read. + if self.tree_id == 0 { let mut root = self.root_pin.write(); crate::layout::set_frame_epoch_high_water( root.as_mut_slice(), self.store.current_epoch(), ); + drop(root); + self.store.mark_dirty_cached( + self.root_guid, + crate::store::STRUCTURAL_SEQ, + self.root_pin.as_ref(), + ); } - self.store - .mark_dirty_cached(self.root_guid, STRUCTURAL_SEQ, self.root_pin.as_ref()); let snapshot_lease = crate::store::SnapshotLease::new( Arc::clone(&self.store), @@ -2218,7 +2236,7 @@ impl Tree { ) }), ); - Ok(Snapshot::new(view, epoch)) + Snapshot::new(view, epoch) } /// Return `true` if no live key starts with `prefix`. @@ -2563,6 +2581,7 @@ impl Tree { )> { if let Some(journal) = journal { let _commit = commit_gate.enter_checkpoint(); + store.stage_snapshot_epoch()?; store.flush_write_deltas()?; let wal_up_to = journal.queued_work(); let snap_dirty = store.snapshot_dirty(); @@ -2579,6 +2598,7 @@ impl Tree { } } else { let _commit = commit_gate.enter_checkpoint(); + store.stage_snapshot_epoch()?; store.flush_write_deltas()?; let snap_dirty = store.snapshot_dirty(); let snap_pending = store.snapshot_pending_deletes(); @@ -2754,6 +2774,7 @@ impl Tree { && store.pending_delete_count() == 0 && store.orphan_staging_count() == 0 && store.write_delta_count() == 0 + && !store.has_unstaged_snapshot_epoch() && !store.needs_flush() { journal.truncate()?; @@ -2774,6 +2795,7 @@ impl Tree { && store.pending_delete_count() == 0 && store.orphan_staging_count() == 0 && store.write_delta_count() == 0 + && !store.has_unstaged_snapshot_epoch() && !store.needs_flush() && journal.is_none_or(|journal| !journal.needs_checkpoint()) } diff --git a/src/checkpoint/round.rs b/src/checkpoint/round.rs index 5a26e36..14c819f 100644 --- a/src/checkpoint/round.rs +++ b/src/checkpoint/round.rs @@ -174,6 +174,7 @@ impl Pipeline { && shared.bm.pending_delete_count() == 0 && shared.bm.orphan_staging_count() == 0 && shared.bm.write_delta_count() == 0 + && !shared.bm.has_unstaged_snapshot_epoch() && !shared.bm.needs_flush(); if !clean { return Ok(()); @@ -272,7 +273,11 @@ pub(super) fn run_round(shared: &Arc, pipeline: &mut Pipeline) -> Result { let _maintenance = shared.maintenance_gate.enter_shared(); let _commit = shared.commit_gate.enter_checkpoint(); - if let Err(e) = shared.bm.flush_write_deltas() { + if let Err(e) = shared + .bm + .stage_snapshot_epoch() + .and_then(|()| shared.bm.flush_write_deltas()) + { shared.rounds_failed.fetch_add(1, Ordering::Relaxed); shared .last_round_micros @@ -298,7 +303,11 @@ pub(super) fn run_round(shared: &Arc, pipeline: &mut Pipeline) -> Result } else { let _maintenance = shared.maintenance_gate.enter_shared(); let _commit = shared.commit_gate.enter_checkpoint(); - if let Err(e) = shared.bm.flush_write_deltas() { + if let Err(e) = shared + .bm + .stage_snapshot_epoch() + .and_then(|()| shared.bm.flush_write_deltas()) + { shared.rounds_failed.fetch_add(1, Ordering::Relaxed); shared .last_round_micros diff --git a/src/engine/walker/erase.rs b/src/engine/walker/erase.rs index 962840d..6078c22 100644 --- a/src/engine/walker/erase.rs +++ b/src/engine/walker/erase.rs @@ -454,12 +454,14 @@ fn lock_coupled_erase_in_blob( child_touched }; + // Match insert's COW inheritance boundary: no later fork may copy these + // bytes before their WAL low-watermark is visible in dirty bookkeeping. + if child_touched && !is_top_blob { + bm.mark_dirty_cached(current_guid, seq, current_entry); + } drop(guard); if child_touched { bm.note_compaction_candidate(current_guid); - if !is_top_blob { - bm.mark_dirty_cached(current_guid, seq, current_entry); - } } Ok(EraseOutcome { diff --git a/src/engine/walker/insert.rs b/src/engine/walker/insert.rs index 85af065..a5167b4 100644 --- a/src/engine/walker/insert.rs +++ b/src/engine/walker/insert.rs @@ -741,14 +741,14 @@ fn insert_batch_in_pinned_blob( } } + if dirty && !is_top_blob { + bm.mark_dirty_cached(current_guid, dirty_seq, current_entry); + } drop(guard); if needs_compaction { bm.note_compaction_candidate(current_guid); } - if dirty && !is_top_blob { - bm.mark_dirty_cached(current_guid, dirty_seq, current_entry); - } Ok(InsertBatchOutcome { root_dirty: is_top_blob && dirty, @@ -886,13 +886,16 @@ fn lock_coupled_insert_in_blob( (current_dirty, false) } }; + // A snapshot on another DB tree can advance the global COW + // barrier while this tree is writing. Publish WAL debt before + // releasing the child latch so a subsequent fork inherits it. + if blob_dirty && !is_top_blob { + bm.mark_dirty_cached(current_guid, seq, current_entry); + } drop(guard); if needs_compaction { bm.note_compaction_candidate(current_guid); } - if blob_dirty && !is_top_blob { - bm.mark_dirty_cached(current_guid, seq, current_entry); - } return Ok(InsertOutcome { root_dirty: is_top_blob && blob_dirty, diff --git a/src/store/buffer_manager/mod.rs b/src/store/buffer_manager/mod.rs index 01a5e28..ac4470b 100644 --- a/src/store/buffer_manager/mod.rs +++ b/src/store/buffer_manager/mod.rs @@ -49,8 +49,10 @@ //! //! Invariants: //! -//! - **I1**: a `(guid, _)` entry exists in `dirty` iff the cached -//! image of `guid` is newer than the store image. +//! - **I1**: a `(guid, _)` entry exists in `dirty` iff the authoritative cached +//! image of `guid` is newer than the store image. Retired unreachable images +//! can be cancelled together at an idle capture after their WAL debt has +//! transferred to the live replacement; they are no longer authoritative. //! - **I2**: WAL `trim_id <= min(dirty.values()) - 1` (or //! `next_seq - 1` if `dirty` is empty). //! - **I3**: [`BufferManager::snapshot_dirty`] drains the map @@ -184,28 +186,13 @@ struct SnapshotState { /// be upgraded under the registry lock so GC holds a stable root while /// a last derived view/cursor concurrently releases its epoch lease. live: BTreeMap, - /// COW detachments grouped by the parent whose edge was repointed. - /// A child stays non-reclaimable here until that exact parent publishes - /// dirty/flushing debt; this closes the last-lease-drop window between - /// edge mutation and `mark_dirty(parent)`. - cow_pending: HashMap>, - /// Frames forked away from the live tree, tagged with the - /// `created_epoch` of the forked-away version. Once the fork barrier - /// drops below the tag no live snapshot can reference it. Retirement only - /// moves the exact GUID into the reclaim FIFO; physical deletion waits for - /// a clean durable checkpoint frontier or a full reachability sweep. - orphans: Vec<(BlobGuid, u64)>, - /// Structural detachments grouped by their rewritten parent. These do - /// not install a visibility fence because a stable snapshot may still - /// lazily pin the old child. Parent dirty publication promotes them to - /// `structural_orphans` or the reclaim FIFO. - structural_pending: HashMap>, - /// Children detached by structural merge while at least one snapshot - /// lease is live. Unlike a COW fork, a merge has no per-snapshot epoch - /// tag: any copied root may still point at the detached child. Hold the - /// exact GUID until the last lease retires, then move it to the same - /// post-checkpoint FIFO as ordinary retired COW frames. - structural_orphans: HashSet, + /// Detachments stay staged until the exact rewritten parent publishes + /// dirty/flushing debt, closing the final-lease-drop publication window. + pending: HashMap>, + /// Each orphan waits on the newest live epoch in its visibility interval. + /// Newer snapshots cannot see a detached frame and must not retain it. + /// Retirement touches only that epoch's waiters, not the entire backlog. + waiting: BTreeMap>, /// Copy-on-write and structural frames eligible for exact reclaim after /// their final snapshot epoch has retired. A clean checkpoint drains this /// FIFO in bounded batches without an all-store reachability walk; pinned @@ -217,6 +204,39 @@ struct SnapshotState { struct SnapshotRoot { guid: BlobGuid, pin: Weak, + /// The public lease plus any GC walks that have captured this root. + holders: usize, +} + +#[derive(Clone, Copy)] +struct SnapshotOrphan { + guid: BlobGuid, + first_epoch: u64, + last_epoch: u64, + dirty_seq: u64, +} + +impl SnapshotState { + /// Return whether this enqueue added immediately eligible reclaim work. + fn queue_orphan(&mut self, orphan: SnapshotOrphan) -> bool { + let protector = (orphan.first_epoch <= orphan.last_epoch) + .then(|| { + self.live + .range(orphan.first_epoch..=orphan.last_epoch) + .next_back() + }) + .flatten() + .map(|(&epoch, _)| epoch); + if let Some(epoch) = protector { + self.waiting.entry(epoch).or_default().push(orphan); + false + } else if self.retired_orphan_set.insert(orphan.guid) { + self.retired_orphans.push_back(orphan.guid); + true + } else { + false + } + } } /// Shared lifetime token for one copy-on-write snapshot epoch. @@ -261,6 +281,7 @@ impl Drop for SnapshotLease { /// Strong snapshot-root pin owned by one GC reachability pass. pub(crate) struct PinnedSnapshotRoot { store: Arc, + epoch: u64, guid: BlobGuid, pin: Option>, } @@ -274,6 +295,7 @@ impl PinnedSnapshotRoot { impl Drop for PinnedSnapshotRoot { fn drop(&mut self) { + self.store.retire_snapshot(self.epoch); drop(self.pin.take()); self.store.discard_snapshot_root(self.guid); } @@ -517,6 +539,11 @@ pub struct BufferManager { /// snapshot knows whether it must fork the frame instead of /// overwriting it in place. current_epoch: AtomicU64, + /// DB clock summary is staged once per checkpoint instead of dirtying + /// every captured named root. Standalone trees keep their original root + /// high-water publication. The on-disk header layout is unchanged. + snapshot_epoch_root: Mutex>, + staged_snapshot_epoch: AtomicU64, /// Highest epoch held by any LIVE snapshot — the copy-on-write /// fork barrier. A frame whose `created_epoch <= fork_barrier` /// may be visible to a snapshot and so must be forked before an @@ -528,6 +555,10 @@ pub struct BufferManager { /// inline. Physical deletion requires a clean durable checkpoint frontier /// or a full reachability sweep. snapshots: Mutex, + /// Serialize transient-image cancellation with a complete dirty capture. + /// An older checkpoint may still own a parent image pointing at a retired + /// child; pruning is disabled until every such flushing claim is gone. + cow_prune_capture: Mutex<()>, /// Even while idle and odd while physical GC deletion is active. /// Optimistic readers use this sequence to distinguish a concurrent /// reachability sweep from a stable missing-child corruption. @@ -607,6 +638,39 @@ impl BufferManager { self.current_epoch.store(epoch.max(1), Ordering::Release); } + pub(crate) fn set_snapshot_epoch_root(&self, root: BlobGuid) { + *self.snapshot_epoch_root.lock().unwrap() = Some(root); + } + + pub(crate) fn has_unstaged_snapshot_epoch(&self) -> bool { + self.current_epoch() > self.staged_snapshot_epoch.load(Ordering::Acquire) + && self.snapshot_epoch_root.lock().unwrap().is_some() + } + + /// Called under checkpoint-exclusive admission, before dirty capture. + /// The clock summary becomes ordinary structural dirty debt, retaining + /// retries and the existing WAL/data/manifest ordering on every failure. + pub(crate) fn stage_snapshot_epoch(&self) -> Result<()> { + if !self.has_unstaged_snapshot_epoch() { + return Ok(()); + } + let guid = self + .snapshot_epoch_root + .lock() + .unwrap() + .expect("DB epoch root"); + let epoch = self.current_epoch(); + let root = self.pin_silent(guid)?; + { + let mut bytes = root.write(); + let high_water = crate::layout::frame_epoch_high_water(bytes.as_slice()); + crate::layout::set_frame_epoch_high_water(bytes.as_mut_slice(), high_water.max(epoch)); + } + self.mark_dirty_cached(guid, STRUCTURAL_SEQ, &root); + self.staged_snapshot_epoch.store(epoch, Ordering::Release); + Ok(()) + } + /// The copy-on-write fork barrier: the highest epoch any live /// snapshot holds. A frame with `created_epoch <= fork_barrier` /// may be referenced by a snapshot and must be forked before an @@ -683,9 +747,24 @@ impl BufferManager { new_guid: BlobGuid, seq: u64, ) -> Result> { - let mut buf = self.alloc_blob_buf_zeroed(); + // The full frame is overwritten before any byte is read. + let mut buf = self.alloc_blob_buf_uninit(); buf.as_mut_slice().copy_from_slice(src_bytes); crate::layout::set_frame_blob_guid(buf.as_mut_slice(), new_guid); + // A transient predecessor may be discarded before checkpoint. Its + // successor must retain all unflushed WAL debt, not just the WAL seq + // of the next user write. Otherwise pruning could raise the trim + // watermark above mutations present only in this copied image. + let source_guid = BlobFrameRef::wrap(src_bytes).header().blob_guid; + let seq = self + .mutation_shard(source_guid) + .lock() + .unwrap() + .dirty + .get(&source_guid) + .copied() + .unwrap_or(seq) + .min(seq); self.install_new_blob(new_guid, buf, seq); self.pin(new_guid) } @@ -708,7 +787,8 @@ impl BufferManager { src: &CachedBlob, ) -> Arc { let guard = src.read(); - let mut buf = self.alloc_blob_buf_zeroed(); + // Unlike a newly initialized ART frame, this is a complete overwrite. + let mut buf = self.alloc_blob_buf_uninit(); buf.as_mut_slice().copy_from_slice(guard.as_slice()); crate::layout::set_frame_blob_guid(buf.as_mut_slice(), new_guid); self.insert_owned_into_cache(new_guid, buf, PinAccess::Silent) @@ -723,28 +803,39 @@ impl BufferManager { root_guid: BlobGuid, root_pin: &Arc, ) -> Result { + self.register_snapshots(&[(root_guid, Arc::clone(root_pin))]) + } + + /// Atomically register a coordinated capture, retaining an independent + /// lease per root. The owning tree gates remain exclusive throughout root + /// copying and this publication; other trees may keep writing normally. + pub(crate) fn register_snapshots(&self, roots: &[(BlobGuid, Arc)]) -> Result { + if roots.is_empty() { + return Ok(self.current_epoch()); + } + let count = u64::try_from(roots.len()).map_err(|_| Error::SnapshotEpochExhausted)?; let mut snaps = self.snapshots.lock().expect("snapshot registry poisoned"); - // The returned pre-bump value is this snapshot's barrier; the - // post-bump value is persisted as the root high-water and stamps new - // frames. MAX-1 is a valid but exhausted durable high-water, while - // MAX can never be emitted by a successful registration. A checked - // atomic update prevents MAX from wrapping to zero and clearing the - // fork barrier while an exhausted snapshot remains live. - let epoch = self + // MAX-1 is an exhausted durable high-water; never emit MAX or wrap. + // Reserving the whole interval first makes exhaustion all-or-nothing. + let first = self .current_epoch .fetch_update(Ordering::AcqRel, Ordering::Acquire, |current| { - (current < u64::MAX - 1).then_some(current + 1) + current.checked_add(count).filter(|next| *next < u64::MAX) }) .map_err(|_| Error::SnapshotEpochExhausted)?; - snaps.live.insert( - epoch, - SnapshotRoot { - guid: root_guid, - pin: Arc::downgrade(root_pin), - }, - ); - self.fork_barrier.store(epoch, Ordering::Release); - Ok(epoch) + for ((guid, pin), epoch) in roots.iter().zip(first..) { + snaps.live.insert( + epoch, + SnapshotRoot { + guid: *guid, + pin: Arc::downgrade(pin), + holders: 1, + }, + ); + } + self.fork_barrier + .store(first + count - 1, Ordering::Release); + Ok(first) } /// Stage a frame forked away from the live tree until its rewritten parent @@ -757,11 +848,7 @@ impl BufferManager { guid: BlobGuid, created_epoch: u64, ) { - let mut snapshots = self.snapshots.lock().expect("snapshot registry poisoned"); - let pending = snapshots.cow_pending.entry(parent_guid).or_default(); - if !pending.iter().any(|(candidate, _)| *candidate == guid) { - pending.push((guid, created_epoch)); - } + self.stage_snapshot_orphan(parent_guid, guid, created_epoch); } /// Stage a child detached by structural merge under the parent whose @@ -769,12 +856,32 @@ impl BufferManager { /// `mark_dirty(parent)` promotes it only after the new parent image is /// protected by dirty/flushing checkpoint debt. pub(crate) fn stage_structural_reclaim(&self, parent_guid: BlobGuid, child_guid: BlobGuid) { + // Conservative lower bound: structural rewrites may move older + // contents into a newer frame, so do not infer their first visibility + // from the detached frame's creation stamp. + self.stage_snapshot_orphan(parent_guid, child_guid, 0); + } + + fn stage_snapshot_orphan(&self, parent: BlobGuid, guid: BlobGuid, first_epoch: u64) { + let dirty_seq = self + .mutation_shard(guid) + .lock() + .unwrap() + .dirty + .get(&guid) + .copied() + .unwrap_or(STRUCTURAL_SEQ); let mut snapshots = self.snapshots.lock().expect("snapshot registry poisoned"); - snapshots - .structural_pending - .entry(parent_guid) - .or_default() - .insert(child_guid); + let last_epoch = snapshots.live.keys().next_back().copied().unwrap_or(0); + let pending = snapshots.pending.entry(parent).or_default(); + if !pending.iter().any(|candidate| candidate.guid == guid) { + pending.push(SnapshotOrphan { + guid, + first_epoch, + last_epoch, + dirty_seq, + }); + } } /// Promote every COW/structural detachment owned by `parent_guid` after @@ -782,37 +889,36 @@ impl BufferManager { /// snapshot retirement can only make a staged child eligible here, /// after dirty/flushing state already closes the clean frontier. fn publish_parent_orphans(&self, parent_guid: BlobGuid) { + let pending = self + .snapshots + .lock() + .expect("snapshot registry poisoned") + .pending + .remove(&parent_guid) + .unwrap_or_default(); + if pending.is_empty() { + return; + } + // Structural merge moves child contents into this parent; COW also + // conservatively carries the debt through its rewritten edge. Once a + // retired dirty image is omitted from checkpoint, its replacement + // must still cover the oldest WAL mutation copied out of that image. + let dirty_seq = pending + .iter() + .map(|orphan| orphan.dirty_seq) + .min() + .unwrap_or(STRUCTURAL_SEQ); + if dirty_seq != STRUCTURAL_SEQ { + let cached = self.get_cached_with_access(parent_guid, PinAccess::Silent); + self.mark_dirty_with_hint(parent_guid, dirty_seq, cached.as_deref()); + } let wake = { let mut snapshots = self.snapshots.lock().expect("snapshot registry poisoned"); - let cow = snapshots - .cow_pending - .remove(&parent_guid) - .unwrap_or_default(); - let structural = snapshots - .structural_pending - .remove(&parent_guid) - .unwrap_or_default(); - let barrier = snapshots.live.keys().next_back().copied().unwrap_or(0); - let mut wake = false; - for (guid, created_epoch) in cow { - if barrier != 0 && created_epoch <= barrier { - snapshots.orphans.push((guid, created_epoch)); - } else if snapshots.retired_orphan_set.insert(guid) { - snapshots.retired_orphans.push_back(guid); - wake = true; - } + let was_empty = snapshots.retired_orphans.is_empty(); + for orphan in pending { + snapshots.queue_orphan(orphan); } - for guid in structural { - if snapshots.live.is_empty() { - if snapshots.retired_orphan_set.insert(guid) { - snapshots.retired_orphans.push_back(guid); - wake = true; - } - } else { - snapshots.structural_orphans.insert(guid); - } - } - wake + was_empty && !snapshots.retired_orphans.is_empty() }; if wake { self.wake_checkpointer(); @@ -821,8 +927,8 @@ impl BufferManager { /// Retire the snapshot at `epoch`: lower the fork barrier to the /// highest remaining live snapshot epoch (or `0`), evict the - /// snapshot's root frame, and stop tracking every orphan whose - /// forked-away version is now newer than the barrier. + /// snapshot's root frame, and reconsider only the orphans waiting on this + /// exact epoch. Snapshots created after detachment cannot prolong a frame. /// /// Persisted blobs are deliberately retained. The live parent that /// stopped referencing a copy-on-write child may still have its older @@ -831,39 +937,109 @@ impl BufferManager { /// requires a clean durable checkpoint frontier or a full reachability /// sweep. Idempotent for an unknown epoch. pub(crate) fn retire_snapshot(&self, epoch: u64) { - let (root, wake) = { + let (root, wake, prune) = { let mut snaps = self.snapshots.lock().expect("snapshot registry poisoned"); + let Some(root) = snaps.live.get_mut(&epoch) else { + return; + }; + if root.holders > 1 { + root.holders -= 1; + return; + } let root = snaps.live.remove(&epoch); let barrier = snaps.live.keys().next_back().copied().unwrap_or(0); self.fork_barrier.store(barrier, Ordering::Release); - let mut active = Vec::with_capacity(snaps.orphans.len()); - for (guid, created_epoch) in std::mem::take(&mut snaps.orphans) { - if barrier != 0 && created_epoch <= barrier { - active.push((guid, created_epoch)); - } else if snaps.retired_orphan_set.insert(guid) { - snaps.retired_orphans.push_back(guid); - } + let was_empty = snaps.retired_orphans.is_empty(); + for orphan in snaps.waiting.remove(&epoch).unwrap_or_default() { + snaps.queue_orphan(orphan); } - snaps.orphans = active; - if barrier == 0 { - let structural = std::mem::take(&mut snaps.structural_orphans); - for guid in structural { - if snaps.retired_orphan_set.insert(guid) { - snaps.retired_orphans.push_back(guid); - } - } - } - (root, !snaps.retired_orphans.is_empty()) + ( + root, + was_empty && !snaps.retired_orphans.is_empty(), + snaps.retired_orphans.len() >= 32, + ) }; if let Some(root) = root { self.discard_snapshot_root(root.guid); } + if prune { + self.prune_unpersisted_orphans(64); + } if wake { self.wake_checkpointer(); } - // Persistent detachments deliberately keep their cache and dirty - // bookkeeping. A clean checkpoint may reclaim their exact FIFO GUIDs; - // a full GC additionally handles crash leftovers by reachability. + // Persisted detachments retain their cache/dirty bookkeeping until a + // clean checkpoint or full reachability GC proves durable detachment. + } + + /// Discard only unreachable versions that have never entered the inner + /// store. No data/manifest write or delete is performed. Checkpoint claims + /// and pins are fail-closed; their candidates return to the normal FIFO. + /// The COW successor inherits dirty WAL debt in `fork_frame` before the + /// predecessor becomes eligible here. + fn prune_unpersisted_orphans(&self, limit: usize) -> usize { + let _capture = self.cow_prune_capture.lock().unwrap(); + if self.flushing_count() != 0 { + return 0; + } + let candidates = self.take_retired_orphans_bounded(limit); + if candidates.is_empty() { + return 0; + } + let _epoch = self.begin_gc_epoch(); + let mut retry = Vec::new(); + let mut freed = 0; + for guid in candidates { + { + let mut state = self.mutation_shard(guid).lock().unwrap(); + if state.checkpoint_owned_or_pending(&guid) { + retry.push(guid); + continue; + } + // `snapshot_dirty` leaves fenced entries unclaimed. This + // closes the has_blob -> cache removal window against I/O. + state.gc_deleting.insert(guid); + self.delete_fence_total.fetch_add(1, Ordering::AcqRel); + } + let mut removed = false; + if matches!(self.store.has_blob(guid), Ok(false)) { + self.invalidate_indexed_reads(guid); + if let Some((_, entry)) = self.cache.remove_if(&guid, |_, entry| { + if Arc::strong_count(entry) > 1 { + return false; + } + let mut state = self.mutation_shard(guid).lock().unwrap(); + if state.flushing.contains_key(&guid) || state.has_logical_delete_fence(&guid) { + return false; + } + // Cache -> mutation is the normal eviction lock order. + // Remove dirty intent before releasing the fence; no + // checkpoint can claim bytes that are about to disappear. + state.remove_unclaimed_dirty(&guid); + let hints = state.remove_maintenance_candidates(&guid); + self.decrement_candidate_totals(hints); + true + }) { + entry.clear_dirty_hint(); + self.route_resident.remove(guid); + removed = true; + } + } + { + let mut state = self.mutation_shard(guid).lock().unwrap(); + if state.gc_deleting.remove(&guid) { + self.delete_fence_total.fetch_sub(1, Ordering::AcqRel); + } + } + if removed { + freed += 1; + self.remove_read_index_token_if_unreachable(guid); + } else { + retry.push(guid); + } + } + self.restore_retired_orphans(retry); + freed } /// Discard an ephemeral snapshot root from process-local state. @@ -977,22 +1153,35 @@ impl BufferManager { /// GC either omits an already-retired root or owns a strong pin for the /// full reachability walk. pub(crate) fn snapshot_roots_pinned(self: &Arc) -> Result> { - self.snapshots - .lock() - .expect("snapshot registry poisoned") + let mut snapshots = self.snapshots.lock().expect("snapshot registry poisoned"); + // First upgrade all pins without changing the registry so an error + // cannot leak a partial set of epoch holders. + let roots = snapshots .live - .values() - .map(|root| { + .iter() + .map(|(&epoch, root)| { root.pin .upgrade() - .map(|pin| PinnedSnapshotRoot { - store: Arc::clone(self), - guid: root.guid, - pin: Some(pin), - }) + .map(|pin| (epoch, root.guid, pin)) .ok_or(Error::Internal("live snapshot registry lost its root pin")) }) - .collect() + .collect::>>()?; + Ok(roots + .into_iter() + .map(|(epoch, guid, pin)| { + snapshots + .live + .get_mut(&epoch) + .expect("registered root") + .holders += 1; + PinnedSnapshotRoot { + store: Arc::clone(self), + epoch, + guid, + pin: Some(pin), + } + }) + .collect()) } /// Free every persisted frame not in `reachable`, returning the count @@ -1170,25 +1359,14 @@ impl BufferManager { #[must_use] pub(crate) fn gc_orphan_backlog_count(&self) -> usize { let snapshots = self.snapshots.lock().expect("snapshot registry poisoned"); - snapshots.cow_pending.values().map(Vec::len).sum::() - + snapshots.orphans.len() - + snapshots - .structural_pending - .values() - .map(HashSet::len) - .sum::() - + snapshots.structural_orphans.len() + snapshots.pending.values().map(Vec::len).sum::() + + snapshots.waiting.values().map(Vec::len).sum::() + snapshots.retired_orphans.len() } pub(crate) fn orphan_staging_count(&self) -> usize { let snapshots = self.snapshots.lock().expect("snapshot registry poisoned"); - snapshots.cow_pending.values().map(Vec::len).sum::() - + snapshots - .structural_pending - .values() - .map(HashSet::len) - .sum::() + snapshots.pending.values().map(Vec::len).sum::() } pub(crate) fn register_checkpoint_waker(&self, thread: std::thread::Thread) { @@ -1283,8 +1461,11 @@ impl BufferManager { clock: AtomicU64::new(1), telemetry: Telemetry::default(), current_epoch: AtomicU64::new(1), + snapshot_epoch_root: Mutex::new(None), + staged_snapshot_epoch: AtomicU64::new(1), fork_barrier: AtomicU64::new(0), snapshots: Mutex::new(SnapshotState::default()), + cow_prune_capture: Mutex::new(()), gc_epoch: AtomicU64::new(0), physical_gc: Mutex::new(()), gc_reclaimed_count: AtomicU64::new(0), @@ -2324,11 +2505,30 @@ impl BufferManager { } pub(crate) fn flush_write_deltas_for_tree(&self, tree_id: u64) -> Result<()> { + if !self.write_delta.has_any() { + return Ok(()); + } let _flush = self.write_delta_flush.lock().unwrap(); let ops = self.write_delta.begin_flush_tree(tree_id); self.apply_write_delta_ops(ops) } + /// Flush a frozen set of named trees with one flush admission. The caller + /// holds their mutation gates, so an empty overlay is stable for them. + pub(crate) fn flush_write_deltas_for_trees( + &self, + trees: impl IntoIterator, + ) -> Result<()> { + if !self.write_delta.has_any() { + return Ok(()); + } + let _flush = self.write_delta_flush.lock().unwrap(); + for tree in trees { + self.apply_write_delta_ops(self.write_delta.begin_flush_tree(tree))?; + } + Ok(()) + } + fn apply_write_delta_ops(&self, ops: Vec) -> Result<()> { if ops.is_empty() { return Ok(()); @@ -2497,6 +2697,22 @@ impl BufferManager { /// tree-wide stable for WAL trimming. #[must_use] pub fn snapshot_dirty(&self) -> HashMap { + let _capture = self.cow_prune_capture.lock().unwrap(); + // Freeze the retirement decision for the entire capture. Looking it + // up independently per shard could capture an old parent, then omit + // its child when their final lease drops between the two shards. + // Earlier epochs can still own parent images that reference a + // currently retired child. Keep that child's dirty image available + // to their dependency planner until the complete pipeline is idle. + let retired = if self.flushing_count() == 0 { + self.snapshots + .lock() + .expect("snapshot registry poisoned") + .retired_orphan_set + .clone() + } else { + HashSet::new() + }; let mut out = HashMap::new(); for shard in &self.mutation { let (claimed, logically_deleted) = { @@ -2505,7 +2721,7 @@ impl BufferManager { let mut claimed = Vec::new(); let mut logically_deleted = Vec::new(); for (guid, seq) in snap { - if state.has_logical_delete_fence(&guid) { + if state.has_logical_delete_fence(&guid) || retired.contains(&guid) { logically_deleted.push(guid); } else if state.gc_deleting.contains(&guid) { // Transient physical GC may still defer on a pin. @@ -5369,6 +5585,223 @@ mod tests { assert_eq!(bm.current_epoch(), u64::MAX - 1); } + #[test] + fn later_snapshots_do_not_retain_detached_versions() { + let inner: Arc = Arc::new(MemoryBlobStore::new()); + let root_guid = [0x71; 16]; + let cow = [0x72; 16]; + let structural = [0x73; 16]; + for guid in [root_guid, cow, structural] { + inner.write_blob(guid, &make_buf(1)).unwrap(); + } + let bm = BufferManager::new(inner, 8); + let root = bm.pin(root_guid).unwrap(); + let first = bm.register_snapshot(root_guid, &root).unwrap(); + bm.stage_cow_reclaim(root_guid, cow, first); + bm.stage_structural_reclaim(root_guid, structural); + // Register before dirty publication: the visibility upper bound must + // be captured at detachment, not at promotion or retirement. + let later = bm.register_snapshot(root_guid, &root).unwrap(); + bm.mark_dirty_cached(root_guid, 7, &root); + bm.retire_snapshot(first); + let snapshots = bm.snapshots.lock().unwrap(); + assert!(snapshots.waiting.is_empty()); + assert!(snapshots.retired_orphan_set.contains(&cow)); + assert!(snapshots.retired_orphan_set.contains(&structural)); + drop(snapshots); + assert_eq!(bm.fork_barrier(), later); + bm.retire_snapshot(later); + } + + #[test] + fn detached_version_waits_only_for_epochs_inside_visibility_interval() { + let inner: Arc = Arc::new(MemoryBlobStore::new()); + let root_guid = [0x74; 16]; + inner.write_blob(root_guid, &make_buf(1)).unwrap(); + let bm = BufferManager::new(inner, 4); + let root = bm.pin(root_guid).unwrap(); + let older = bm.register_snapshot(root_guid, &root).unwrap(); + let first = bm.register_snapshot(root_guid, &root).unwrap(); + let last = bm.register_snapshot(root_guid, &root).unwrap(); + let child = [0x75; 16]; + bm.stage_cow_reclaim(root_guid, child, first); + bm.mark_dirty_cached(root_guid, 9, &root); + bm.retire_snapshot(last); + assert!(bm.snapshots.lock().unwrap().waiting.contains_key(&first)); + assert!(!bm + .snapshots + .lock() + .unwrap() + .retired_orphan_set + .contains(&child)); + bm.retire_snapshot(first); + assert!(bm + .snapshots + .lock() + .unwrap() + .retired_orphan_set + .contains(&child)); + assert_eq!(bm.fork_barrier(), older); + bm.retire_snapshot(older); + } + + #[test] + fn gc_root_pin_retains_epoch_after_public_lease_retires() { + let inner: Arc = Arc::new(MemoryBlobStore::new()); + let root_guid = [0x76; 16]; + inner.write_blob(root_guid, &make_buf(1)).unwrap(); + let bm = Arc::new(BufferManager::new(inner, 4)); + let root = bm.pin(root_guid).unwrap(); + let epoch = bm.register_snapshot(root_guid, &root).unwrap(); + let gc_roots = bm.snapshot_roots_pinned().unwrap(); + let child = [0x77; 16]; + bm.stage_cow_reclaim(root_guid, child, 0); + bm.mark_dirty_cached(root_guid, 9, &root); + bm.retire_snapshot(epoch); + assert_eq!(bm.fork_barrier(), epoch); + assert_eq!(bm.take_retired_orphans_bounded(10).len(), 0); + drop(gc_roots); + assert_eq!(bm.fork_barrier(), 0); + assert_eq!(bm.take_retired_orphans_bounded(10), vec![child]); + } + + #[test] + fn batch_snapshot_epoch_exhaustion_is_all_or_nothing() { + let inner: Arc = Arc::new(MemoryBlobStore::new()); + let guid = [0x78; 16]; + inner.write_blob(guid, &make_buf(1)).unwrap(); + let bm = BufferManager::new(inner, 4); + let root = bm.pin(guid).unwrap(); + bm.set_current_epoch(u64::MAX - 2); + let roots = [(guid, Arc::clone(&root)), ([0x79; 16], root)]; + assert!(matches!( + bm.register_snapshots(&roots), + Err(Error::SnapshotEpochExhausted) + )); + assert_eq!(bm.current_epoch(), u64::MAX - 2); + assert_eq!(bm.fork_barrier(), 0); + assert!(bm.snapshots.lock().unwrap().live.is_empty()); + } + + #[test] + fn unpersisted_cow_pruning_preserves_successor_wal_debt_and_checkpoint_claims() { + let inner: Arc = Arc::new(MemoryBlobStore::new()); + let bm = BufferManager::new(inner.clone(), 4); + let source_guid = [0x7A; 16]; + let successor_guid = [0x7B; 16]; + let mut bytes = make_buf(1); + crate::layout::set_frame_blob_guid(bytes.as_mut_slice(), source_guid); + bm.install_new_blob(source_guid, bytes, 7); + let source = bm.pin(source_guid).unwrap(); + let successor = bm + .fork_frame(source.read().as_slice(), successor_guid, STRUCTURAL_SEQ) + .unwrap(); + bm.mark_dirty_cached(successor_guid, 11, &successor); + assert_eq!( + bm.mutation_shard(successor_guid).lock().unwrap().dirty[&successor_guid], + 7 + ); + bm.restore_retired_orphans([source_guid]); + assert_eq!( + bm.prune_unpersisted_orphans(8), + 0, + "pin must protect the image" + ); + drop(source); + let captured = bm.snapshot_dirty(); + assert_eq!( + bm.prune_unpersisted_orphans(8), + 0, + "checkpoint owns captured bytes" + ); + bm.restore_dirty(captured); + assert_eq!(bm.prune_unpersisted_orphans(8), 1); + assert!(bm + .get_cached_with_access(source_guid, PinAccess::Silent) + .is_none()); + assert!(!inner.has_blob(source_guid).unwrap()); + assert_eq!( + bm.mutation_shard(successor_guid).lock().unwrap().dirty[&successor_guid], + 7 + ); + + // An inner-store image must survive even if it is no longer pinned. + inner.write_blob(successor_guid, &successor.read()).unwrap(); + drop(successor); + bm.restore_retired_orphans([successor_guid]); + assert_eq!(bm.prune_unpersisted_orphans(8), 0); + assert!(inner.has_blob(successor_guid).unwrap()); + } + + #[test] + fn checkpoint_omits_retired_parent_images_after_transient_child_pruning() { + let inner: Arc = Arc::new(MemoryBlobStore::new()); + let bm = BufferManager::new(inner, 8); + let parent = [0x7C; 16]; + let child = [0x7D; 16]; + let live = [0x7E; 16]; + bm.install_new_blob(parent, make_buf(1), 5); + bm.install_new_blob(child, make_buf(2), 7); + bm.install_new_blob(live, make_buf(3), 5); + // The retired parent's cache image may still reference `child` even + // though neither image is reachable by a live tree or snapshot. + bm.restore_retired_orphans([child, parent]); + assert_eq!(bm.prune_unpersisted_orphans(1), 1); + let captured = bm.snapshot_dirty(); + assert!( + !captured.contains_key(&parent), + "do not write a dead image with an elided dependency" + ); + assert!(!captured.contains_key(&child)); + assert_eq!(captured[&live], 5); + // An unrelated older checkpoint may own an ancestor image not named + // by a candidate. Pruning waits for the complete flushing frontier. + assert_eq!(bm.prune_unpersisted_orphans(8), 0); + bm.restore_dirty(captured); + assert_eq!(bm.prune_unpersisted_orphans(8), 1); + } + + #[test] + fn structural_promotion_inherits_child_wal_debt_before_dirty_cancellation() { + let inner: Arc = Arc::new(MemoryBlobStore::new()); + let bm = BufferManager::new(inner, 4); + let parent_guid = [0x7F; 16]; + let child_guid = [0x80; 16]; + bm.install_new_blob(parent_guid, make_buf(1), STRUCTURAL_SEQ); + bm.install_new_blob(child_guid, make_buf(2), 7); + let parent = bm.pin(parent_guid).unwrap(); + bm.stage_structural_reclaim(parent_guid, child_guid); + assert_eq!(bm.orphan_staging_count(), 1); + bm.mark_dirty_cached(parent_guid, STRUCTURAL_SEQ, &parent); + let captured = bm.snapshot_dirty(); + assert_eq!(captured[&parent_guid], 7); + assert!(!captured.contains_key(&child_guid)); + bm.restore_dirty(captured); + } + + #[test] + fn later_checkpoint_keeps_retired_child_while_an_older_parent_epoch_is_live() { + let inner: Arc = Arc::new(MemoryBlobStore::new()); + let bm = BufferManager::new(inner, 4); + let parent = [0x81; 16]; + let child = [0x82; 16]; + bm.install_new_blob(parent, make_buf(1), 5); + let older = bm.snapshot_dirty(); + bm.install_new_blob(child, make_buf(2), 7); + bm.restore_retired_orphans([child]); + let later = bm.snapshot_dirty(); + assert_eq!( + later[&child], 7, + "older in-flight parents may still depend on this image" + ); + assert_eq!(bm.prune_unpersisted_orphans(8), 0); + bm.restore_dirty(later); + bm.restore_dirty(older); + let idle = bm.snapshot_dirty(); + assert!(!idle.contains_key(&child)); + bm.restore_dirty(idle); + } + #[test] fn structural_last_lease_drop_before_parent_dirty_stays_staged() { let inner: Arc = Arc::new(MemoryBlobStore::new()); diff --git a/tests/snapshot.rs b/tests/snapshot.rs index 15fde08..93b41ce 100644 --- a/tests/snapshot.rs +++ b/tests/snapshot.rs @@ -578,6 +578,149 @@ fn run_crash_session(child_test: &str, dir: &std::path::Path) { assert!(status.success(), "crash-session child {child_test} failed"); } +/// Exit without dropping the DB after many pruned, WAL-acknowledged COW +/// generations. Recovery must replay every generation from the old durable +/// topology; no intermediate cache image may be required for correctness. +#[test] +#[ignore = "child-process body for db_pruned_cow_versions_recover_without_final_checkpoint"] +fn pruned_cow_wal_session() { + let Some(dir) = std::env::var_os(CRASH_LEAK_DIR_ENV) else { + return; + }; + let dir = std::path::PathBuf::from(dir); + let db = DB::open(crash_leak_cfg(&dir)).unwrap(); + let left = db.create_tree("left").unwrap(); + let right = db.create_tree("right").unwrap(); + for start in (0..1800u32).step_by(100) { + assert!(db + .atomic(|batch| { + for index in start..start + 100 { + let key = format!("k{index:08}"); + batch.put("left", key.as_bytes(), &[0; 512]); + batch.put("right", key.as_bytes(), &[0; 512]); + } + }) + .unwrap()); + } + db.checkpoint().unwrap(); + assert!(left.stats().unwrap().blob_count > 1); + assert!(right.stats().unwrap().blob_count > 1); + for generation in 1..=96u8 { + let (old_left, old_right) = db + .view(&[("left", b""), ("right", b"")], |view| { + Ok(( + view.tree("left").unwrap().clone(), + view.tree("right").unwrap().clone(), + )) + }) + .unwrap(); + assert!(db + .atomic(|batch| { + for index in (0..1800u32).step_by(100) { + let key = format!("k{index:08}"); + batch.put("left", key.as_bytes(), &[generation; 512]); + batch.put("right", key.as_bytes(), &[generation; 512]); + } + }) + .unwrap()); + for view in [&old_left, &old_right] { + assert_eq!( + view.get(b"k00000400").unwrap().unwrap(), + vec![generation - 1; 512] + ); + } + drop((old_left, old_right)); + assert!( + db.stats().bm_gc_orphan_backlog_count < 80, + "transient generations must be pruned without a checkpoint" + ); + } + std::process::exit(0); +} + +#[test] +fn db_pruned_cow_versions_recover_without_final_checkpoint() { + let dir = tempdir().unwrap(); + run_crash_session("pruned_cow_wal_session", dir.path()); + let db = DB::open(crash_leak_cfg(dir.path())).unwrap(); + for family in ["left", "right"] { + let tree = db.open_tree(family).unwrap(); + for index in 0..1800u32 { + let expected = if index % 100 == 0 { 96 } else { 0 }; + assert_eq!( + tree.get(format!("k{index:08}").as_bytes()) + .unwrap() + .unwrap(), + vec![expected; 512] + ); + } + } + db.checkpoint().unwrap(); +} + +#[test] +fn memory_db_cow_pruning_with_foreign_tree_snapshots_and_concurrent_checkpoints() { + let mut cfg = TreeConfig::memory(); + cfg.memory_flush_on_write = false; + let db = DB::open(cfg).unwrap(); + let data = db.create_tree("data").unwrap(); + let other = db.create_tree("other").unwrap(); + other.put(b"anchor", b"value").unwrap(); + for index in 0..2400u32 { + data.put(format!("k{index:08}").as_bytes(), &[0; 512]) + .unwrap(); + } + db.checkpoint().unwrap(); + let start = Arc::new(std::sync::Barrier::new(4)); + let mut workers = Vec::new(); + for worker in 0..2u32 { + let tree = data.clone(); + let start = Arc::clone(&start); + workers.push(std::thread::spawn(move || { + start.wait(); + for generation in 1..=40u8 { + for index in (worker * 1200..worker * 1200 + 1200).step_by(60) { + tree.put(format!("k{index:08}").as_bytes(), &[generation; 512]) + .unwrap(); + } + } + })); + } + let snapshot_start = Arc::clone(&start); + let snapshots = std::thread::spawn(move || { + snapshot_start.wait(); + for _ in 0..1600 { + let snapshot = other.snapshot(b"").unwrap(); + assert_eq!( + snapshot.get(b"anchor").unwrap().as_deref(), + Some(b"value".as_slice()) + ); + } + }); + let checkpoint_db = db.clone(); + let checkpoints = std::thread::spawn(move || { + start.wait(); + for _ in 0..80 { + checkpoint_db.checkpoint().unwrap(); + } + }); + for worker in workers { + worker.join().unwrap(); + } + snapshots.join().unwrap(); + checkpoints.join().unwrap(); + db.checkpoint().unwrap(); + for index in 0..2400u32 { + let expected = if index % 60 == 0 { 40 } else { 0 }; + assert_eq!( + data.get(format!("k{index:08}").as_bytes()) + .unwrap() + .unwrap(), + vec![expected; 512] + ); + } +} + /// Child body for [`crash_leaked_tree_snapshot_does_not_leave_persistent_garbage`]: /// snapshot + writes + checkpoint, then "crash" — forget the snapshot /// so it never retires. Snapshot roots are in-memory only, so reopen From d42da835af3f198415172c8e529b4c53d698c675 Mon Sep 17 00:00:00 2001 From: "Guocheng(Eric) Song" <132118805+feichai0017@users.noreply.github.com> Date: Thu, 15 Oct 2026 14:25:29 +0800 Subject: [PATCH 2/2] test: add reproducible agent workspace regression benchmark Run the same multi-family handler against Holt and SQLite with balanced frozen-binary A/B order, full domain fingerprints and reopen verification. Keep durability and checkpoint differences explicit. Record bounded macOS results, sync replication and raw-bundle hashes. Keep large raw samples and unredacted desktop environment logs local. --- benches/agent_workspace/.gitignore | 5 + benches/agent_workspace/Cargo.lock | 455 ++++ benches/agent_workspace/Cargo.toml | 19 + benches/agent_workspace/README.md | 90 + benches/agent_workspace/RESULTS_2026-10-15.md | 112 + .../RESULTS_OPTIMIZATION_2026-10-15.md | 113 + benches/agent_workspace/results/README.md | 50 + benches/agent_workspace/results/SHA256SUMS | 2 + .../optimization-2026-10-15/RESULTS.md | 33 + .../optimization-2026-10-15/summary.json | 2136 +++++++++++++++ .../sync-followup-summary.json | 196 ++ ...mization-evidence-2026-10-15.tar.gz.sha256 | 1 + benches/agent_workspace/results/summary.json | 2335 +++++++++++++++++ benches/agent_workspace/run_ab.py | 156 ++ benches/agent_workspace/run_comparison.py | 230 ++ benches/agent_workspace/src/main.rs | 1165 ++++++++ 16 files changed, 7098 insertions(+) create mode 100644 benches/agent_workspace/.gitignore create mode 100644 benches/agent_workspace/Cargo.lock create mode 100644 benches/agent_workspace/Cargo.toml create mode 100644 benches/agent_workspace/README.md create mode 100644 benches/agent_workspace/RESULTS_2026-10-15.md create mode 100644 benches/agent_workspace/RESULTS_OPTIMIZATION_2026-10-15.md create mode 100644 benches/agent_workspace/results/README.md create mode 100644 benches/agent_workspace/results/SHA256SUMS create mode 100644 benches/agent_workspace/results/optimization-2026-10-15/RESULTS.md create mode 100644 benches/agent_workspace/results/optimization-2026-10-15/summary.json create mode 100644 benches/agent_workspace/results/optimization-2026-10-15/sync-followup-summary.json create mode 100644 benches/agent_workspace/results/optimization-evidence-2026-10-15.tar.gz.sha256 create mode 100644 benches/agent_workspace/results/summary.json create mode 100644 benches/agent_workspace/run_ab.py create mode 100644 benches/agent_workspace/run_comparison.py create mode 100644 benches/agent_workspace/src/main.rs diff --git a/benches/agent_workspace/.gitignore b/benches/agent_workspace/.gitignore new file mode 100644 index 0000000..57299dd --- /dev/null +++ b/benches/agent_workspace/.gitignore @@ -0,0 +1,5 @@ +__pycache__/ +results/*.tar.gz +results/optimization-2026-10-15/environment.json +results/optimization-2026-10-15/regression-final.log +results/optimization-2026-10-15/soak-*.jsonl diff --git a/benches/agent_workspace/Cargo.lock b/benches/agent_workspace/Cargo.lock new file mode 100644 index 0000000..9da12c4 --- /dev/null +++ b/benches/agent_workspace/Cargo.lock @@ -0,0 +1,455 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 4 + +[[package]] +name = "bitflags" +version = "2.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b588b76d00fde79687d7646a9b5bdf3cc0f655e0bbd080335a95d7e96f3587da" + +[[package]] +name = "bumpalo" +version = "3.20.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "72f5acc6cb2ba439de613abc23857ec3d78374d8ed5ac84e9d11336e87da8649" + +[[package]] +name = "cc" +version = "1.4.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5d262e149917187838d5b42777c8253bcb64500067342904e7d429499a6f277e" +dependencies = [ + "find-msvc-tools", + "shlex", +] + +[[package]] +name = "cfg-if" +version = "1.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801" + +[[package]] +name = "crc32fast" +version = "1.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9481c1c90cbf2ac953f07c8d4a58aa3945c425b7185c9154d67a65e4230da511" +dependencies = [ + "cfg-if", +] + +[[package]] +name = "crossbeam-channel" +version = "0.5.16" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d85363c37faeca707aef026efa9f3b34d077bce547e48f770770625c6013679e" +dependencies = [ + "crossbeam-utils", +] + +[[package]] +name = "crossbeam-utils" +version = "0.8.22" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "61803da095bee82a81bb1a452ecc25d3b2f1416d1897eb86430c6159ef717c17" + +[[package]] +name = "dashmap" +version = "6.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e6361d5c062261c78a176addb82d4c821ae42bed6089de0e12603cd25de2059c" +dependencies = [ + "cfg-if", + "crossbeam-utils", + "hashbrown 0.14.5", + "lock_api", + "once_cell", + "parking_lot_core", +] + +[[package]] +name = "fallible-iterator" +version = "0.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2acce4a10f12dc2fb14a218589d4f1f62ef011b2d0cc4b3cb1bba8e94da14649" + +[[package]] +name = "fallible-streaming-iterator" +version = "0.1.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7360491ce676a36bf9bb3c56c1aa791658183a54d2744120f27285738d90465a" + +[[package]] +name = "find-msvc-tools" +version = "0.1.10" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "26b73573e6edcd2af0cdf47bd6cb58f0b3839491263c314eaad1ccf24430e1de" + +[[package]] +name = "foldhash" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "77ce24cb58228fbb8aa041425bb1050850ac19177686ea6e0f41a70416f56fdb" + +[[package]] +name = "hashbrown" +version = "0.14.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e5274423e17b7c9fc20b6e7e208532f9b19825d82dfd615708b70edd83df41f1" + +[[package]] +name = "hashbrown" +version = "0.16.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "841d1cc9bed7f9236f321df977030373f4a4163ae1a7dbfe1a51a2c1a51d9100" +dependencies = [ + "foldhash", +] + +[[package]] +name = "hashlink" +version = "0.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ea0b22561a9c04a7cb1a302c013e0259cd3b4bb619f145b32f72b8b4bcbed230" +dependencies = [ + "hashbrown 0.16.1", +] + +[[package]] +name = "holt" +version = "0.9.2" +dependencies = [ + "crc32fast", + "crossbeam-channel", + "dashmap", + "libc", + "smallvec", +] + +[[package]] +name = "holt-agent-workspace-bench" +version = "0.0.0" +dependencies = [ + "crc32fast", + "holt", + "rusqlite", + "serde", + "serde_json", +] + +[[package]] +name = "itoa" +version = "1.0.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682" + +[[package]] +name = "js-sys" +version = "0.3.99" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "142bc4740e452c1e57ade0cbc129f139c9093e354346f0872ef985f4f5cf5f11" +dependencies = [ + "cfg-if", + "once_cell", + "wasm-bindgen", +] + +[[package]] +name = "libc" +version = "0.2.189" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3eaf3ede3fee6db1a4c2ee091bf8a8b4dccdc6d17f656fb07896ee72867612f2" + +[[package]] +name = "libsqlite3-sys" +version = "0.37.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b1f111c8c41e7c61a49cd34e44c7619462967221a6443b0ec299e0ac30cfb9b1" +dependencies = [ + "cc", + "pkg-config", + "vcpkg", +] + +[[package]] +name = "lock_api" +version = "0.4.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "224399e74b87b5f3557511d98dff8b14089b3dadafcab6bb93eab67d3aace965" +dependencies = [ + "scopeguard", +] + +[[package]] +name = "memchr" +version = "2.8.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cf8baf1c55e62ffcace7a9f06f4bd9cd3f0c4beb022d3b367256b91b87513d98" + +[[package]] +name = "once_cell" +version = "1.21.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50" + +[[package]] +name = "parking_lot_core" +version = "0.9.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2621685985a2ebf1c516881c026032ac7deafcda1a2c9b7850dc81e3dfcb64c1" +dependencies = [ + "cfg-if", + "libc", + "redox_syscall", + "smallvec", + "windows-link", +] + +[[package]] +name = "pkg-config" +version = "0.3.33" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "19f132c84eca552bf34cab8ec81f1c1dcc229b811638f9d283dceabe58c5569e" + +[[package]] +name = "proc-macro2" +version = "1.0.107" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "985e7ec9bb745e6ce6535b544d84d6cd6f7ad8bd711c398938ae983b91a766d9" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "quote" +version = "1.0.47" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1fbf4db142a473a8d80c26bbf18454ed458bf8d26c8219c331daecfdbd079001" +dependencies = [ + "proc-macro2", +] + +[[package]] +name = "redox_syscall" +version = "0.5.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ed2bf2547551a7053d6fdfafda3f938979645c44812fbfcda098faae3f1a362d" +dependencies = [ + "bitflags", +] + +[[package]] +name = "rsqlite-vfs" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c51c9ae4df8a7fba42103df5c621fa3c37eccf3a3c650879e90fc48b11cc192c" +dependencies = [ + "hashbrown 0.16.1", + "thiserror", +] + +[[package]] +name = "rusqlite" +version = "0.39.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a0d2b0146dd9661bf67bb107c0bb2a55064d556eeb3fc314151b957f313bcd4e" +dependencies = [ + "bitflags", + "fallible-iterator", + "fallible-streaming-iterator", + "hashlink", + "libsqlite3-sys", + "smallvec", + "sqlite-wasm-rs", +] + +[[package]] +name = "rustversion" +version = "1.0.22" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b39cdef0fa800fc44525c84ccb54a029961a8215f9619753635a9c0d2538d46d" + +[[package]] +name = "scopeguard" +version = "1.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "94143f37725109f92c262ed2cf5e59bce7498c01bcc1502d7b9afe439a4e9f49" + +[[package]] +name = "serde" +version = "1.0.229" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4148590afebada386688f18773da617792bf2ef03ffc1e4cbd2b1d45b023e0ba" +dependencies = [ + "serde_core", + "serde_derive", +] + +[[package]] +name = "serde_core" +version = "1.0.229" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "67dca2c9c51e58a4791a4b1ed58308b39c64224d349a935ab5039aa360942a48" +dependencies = [ + "serde_derive", +] + +[[package]] +name = "serde_derive" +version = "1.0.229" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e7a5d71263a5a7d47b41f6b3f06ba276f10cc18b0931f1799f710578e2309348" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.3", +] + +[[package]] +name = "serde_json" +version = "1.0.151" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c841b55ecdae098c80dcae9cf767f6f8a0c2cdb3416bbef72181df4d0fe73f14" +dependencies = [ + "itoa", + "memchr", + "serde", + "serde_core", + "zmij", +] + +[[package]] +name = "shlex" +version = "2.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f8fadd59c855ef2080decdef8ff161eb6661b86933c9d82e5ba29dc602a55aba" + +[[package]] +name = "smallvec" +version = "1.15.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8ed6a63f02c8539c91a8685a86f4099661ba3da017932f6ebbea6de3f0fa7c90" + +[[package]] +name = "sqlite-wasm-rs" +version = "0.5.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cdd578e94101503d97e2b286bbf8db2135035ca24b2ce4cbf3f9e2fb2bbf1eee" +dependencies = [ + "cc", + "js-sys", + "rsqlite-vfs", + "wasm-bindgen", +] + +[[package]] +name = "syn" +version = "2.0.119" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "872831b642d1a07999a962a351ed35b955ea2cfc8f3862091e2a240a84f17297" +dependencies = [ + "proc-macro2", + "quote", + "unicode-ident", +] + +[[package]] +name = "syn" +version = "3.0.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "53e9bae58849f64dfa4f5d5ae372c8341f7305f82a3868709269343628b659a3" +dependencies = [ + "proc-macro2", + "quote", + "unicode-ident", +] + +[[package]] +name = "thiserror" +version = "2.0.20" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ec86235f5fcc2a73650310756d2ac5b138a5780bbbdfae3eeccec992c435ba4f" +dependencies = [ + "thiserror-impl", +] + +[[package]] +name = "thiserror-impl" +version = "2.0.20" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bc04cd3e1236dd4a98afca4569f2deb3f120e5422a4023be2cb683f8486292af" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.3", +] + +[[package]] +name = "unicode-ident" +version = "1.0.24" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75" + +[[package]] +name = "vcpkg" +version = "0.2.15" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "accd4ea62f7bb7a82fe23066fb0957d48ef677f6eeb8215f372f52e48bb32426" + +[[package]] +name = "wasm-bindgen" +version = "0.2.122" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3ed04576f974d2b2fba0f38c51dbc5518011e38c36bf1143164be765528fd409" +dependencies = [ + "cfg-if", + "once_cell", + "rustversion", + "wasm-bindgen-macro", + "wasm-bindgen-shared", +] + +[[package]] +name = "wasm-bindgen-macro" +version = "0.2.122" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "916151b09da36bd82f6615cbf3a419e2f0ba23a03c6160e8e92eb6bd4aa1dec6" +dependencies = [ + "quote", + "wasm-bindgen-macro-support", +] + +[[package]] +name = "wasm-bindgen-macro-support" +version = "0.2.122" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "299047362ccbfce148b67ab7e73349f77748e00c8296f9542adfad2ad82c5c5e" +dependencies = [ + "bumpalo", + "proc-macro2", + "quote", + "syn 2.0.119", + "wasm-bindgen-shared", +] + +[[package]] +name = "wasm-bindgen-shared" +version = "0.2.122" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9a929b2c61f11ba3e9bc35b50c1f25cb38e0e892c0c231ae2b8cf78d5dad4437" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "windows-link" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5" + +[[package]] +name = "zmij" +version = "1.0.23" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "29666d0abbfad1e3dc4dcf6144730dd3a3ab225bbbdac83319345b1b44ccfc1b" diff --git a/benches/agent_workspace/Cargo.toml b/benches/agent_workspace/Cargo.toml new file mode 100644 index 0000000..af3cc28 --- /dev/null +++ b/benches/agent_workspace/Cargo.toml @@ -0,0 +1,19 @@ +[package] +name = "holt-agent-workspace-bench" +version = "0.0.0" +edition = "2021" +publish = false + +[workspace] + +[dependencies] +holt = { path = "../..", default-features = false } +rusqlite = { version = "=0.39.0", features = ["bundled"] } +serde = { version = "1", features = ["derive"] } +serde_json = "1" +crc32fast = "1" + +[profile.release] +opt-level = 3 +lto = "thin" +codegen-units = 1 diff --git a/benches/agent_workspace/README.md b/benches/agent_workspace/README.md new file mode 100644 index 0000000..43f0f5b --- /dev/null +++ b/benches/agent_workspace/README.md @@ -0,0 +1,90 @@ +# Agent workspace workflow: Holt vs SQLite + +This independent, unpublished package runs the same application handler against +Holt and SQLite. It measures request/response JSON, consistent metadata reads, +guarded multi-family mutations, and optional 4 KiB local text-content I/O. +It does not run the NoKV SDK/RPC server or a remote object provider. + +The eight metadata families are workspace marker, current path, revision, +object reference, operation, command dedupe, history, and change event. Holt +uses named trees and DB atomic/view; SQLite uses identically encoded values +in eight `WITHOUT ROWID` tables, prepared statements, and read/write +transactions. A publish touches all eight families. Both engines execute +the same condition checks, exact-replay rules, content verification, and +JSON handler. Rename is a roundtrip and remove recreates the original path; +each of these workflows commits twice. + +The default mix is 40% read, 15% stat, 15% list, 20% publish, 5% rename +roundtrip, 3% remove+recreate, and 2% bounded consistent snapshot across all +families. Dataset: 8 workspaces × 512 paths. One client uses one workspace; +four clients use independent workspaces sharing one DB and its families. + +```sh +cargo build --release --locked --manifest-path benches/agent_workspace/Cargo.toml +python3 benches/agent_workspace/run_comparison.py \ + --binary benches/agent_workspace/target/release/holt-agent-workspace-bench \ + --out /tmp/holt-sqlite-workspace-results +``` + +The output directory must not already contain same-named run stores. Raw +JSON, per-operation latency samples, exact commands, environment, resource +accounting, domain fingerprints, and a generated report are preserved. Validated +temporary databases are removed between runs; add `--keep-stores` to retain +them. A full suite with retained COW pressure stores can use tens of GiB. Eight +rounds run H,S,S,H / S,H,H,S, giving four independent processes per engine. +Use `--cases metadata_async_1,content_sync_1` for a subset; `--rounds 2` is a +smoke test, not the published comparison. + +Holt is the local path dependency; the report records its commit. SQLite is +bundled through the pinned rusqlite dependency, with its runtime version in +each result. Holt has a shared 64 MiB cache target; SQLite has a 64 MiB target +per connection. Total RSS is not matched by these knobs. Results retain actual +maximum RSS, including Holt's protected dirty/COW frames above its cache target. + +Async uses Holt `sync=false` and SQLite `synchronous=OFF`. Sync uses Holt +`sync=true` and SQLite `synchronous=FULL`, `fullfsync=ON`, and +`checkpoint_fullfsync=ON`. On macOS, Rust `File::sync_data()` uses +`F_FULLFSYNC`, so SQLite's default `fullfsync=OFF` is not the sync comparison. +Sync content publications sync the content file and parent directory on +both engines. These are configured durability boundaries, not a power-cut +qualification. Base cases disable background checkpointing. Cases marked +`background` enable Holt's native 500 ms/512-dirty-blob policy and SQLite's +1000-page WAL autocheckpoint. `_normal` cases additionally set SQLite WAL +`synchronous=NORMAL`, syncing at checkpoint/log reuse. `OFF` and `NORMAL` +results are kept separate; they differ in crash consistency. Each timed burst +ends with an explicit durable checkpoint, reported separately and also +included in drain-inclusive throughput. Setup and warmup are excluded. + +Attached mode writes canonical mutation bytes with the metadata commit: +Holt's attached WAL envelope versus SQLite's transactional recovery table. +The final checkpoint expires this local recovery material. Ordinary and +attached results remain separate. The diagnostic `--read-mode point` omits +cross-tree snapshots for read/stat and must not be presented as having the +default view's consistency semantics. + +Every run verifies exact replay causes no second commit, verifies all table +contents on clean reopen, and the runner rejects any cross-engine or +cross-repetition difference in complete ordered domain content. Objects, +history, and receipts are retained; no vacuum is included. This is a bounded +workspace workload, not indefinite retention or maintenance steady-state +evidence. + +## Baseline / patched / SQLite regression comparison + +`run_ab.py` uses the exact same driver and cases with two frozen binaries. +Twelve forward/reverse matched runs give four processes per variant and +balance each pair's order. It preserves full evidence and rejects mismatched +fingerprints or reopen failures before removing the generated store. + +```sh +python3 benches/agent_workspace/run_ab.py \ + --baseline /absolute/path/to/baseline-binary \ + --patched /absolute/path/to/patched-binary \ + --out /tmp/holt-workspace-ab \ + --cases metadata_async_1,metadata_async_4,metadata_async_4_background_normal +``` + +The output directory must not exist. See +`RESULTS_OPTIMIZATION_2026-10-15.md` for the DB-view/COW optimization results, +sync follow-up, validation boundaries, and remaining costs. The original +Holt/SQLite comparison remains separately preserved. diff --git a/benches/agent_workspace/RESULTS_2026-10-15.md b/benches/agent_workspace/RESULTS_2026-10-15.md new file mode 100644 index 0000000..27187f6 --- /dev/null +++ b/benches/agent_workspace/RESULTS_2026-10-15.md @@ -0,0 +1,112 @@ +# Holt 与 SQLite:agent workspace 本地端到端工作流 + +测量日期:2026-10-15。结论:Holt 的多 Tree 一致性读取和 COW/回收成本确实明显;但整体吞吐的胜负还取决于持久化和 checkpoint 策略,不能用 SQLite `synchronous=OFF` 的结果替代生产配置的结论。 + +## 测量边界 + +这是同一个应用 handler、两种存储适配器的工作流比较: + +```text +JSON 请求编解码 → workspace 领域逻辑 → 多 Tree/多表读写事务 + → 可选的本地内容文件读写与 CRC 校验 → JSON 响应编解码 +``` + +不包含 NoKV SDK/RPC、网络或远端对象服务,也不包含故障注入、断电验证或无限期保留测试。它不能被称为完整 NoKV 产品的性能结果。 + +- Holt:本地 `0.9.2` 源码,commit `f080daf2bc9d8019bb134141dc5ee31fd36f2c03`。本轮未改引擎实现。 +- SQLite:rusqlite `0.39.0`,bundled SQLite `3.51.3`。 +- 硬件:Apple M3 Pro,12 logical CPUs,36 GiB RAM;macOS 26.6.2(Darwin 25.6.0),Apple ARM64。 +- 编译:Rust `1.97.1`,release,opt-level 3,thin LTO,codegen-units 1;两边在同一个可执行文件中。 +- 初始数据:8 个 workspace,每个 512 个文件,共 4,096 个路径。 +- 元数据族:workspace、path、revision、object_ref、operation、dedupe、history、event。Holt 为 8 棵 named Tree,SQLite 为 8 张 `WITHOUT ROWID` 表;值的编码相同。 +- 混合操作:40% read、15% stat、15% list(最多 64 条,返回名称及元数据)、20% publish、5% rename roundtrip、3% remove+recreate、2% 全 8 族一致性 snapshot。实际计数随固定 seed 采样,原始数据逐项保留。 +- publish 一次原子更新 8 个元数据族。rename roundtrip 与 remove+recreate 分别包含两次提交。历史、receipt 和 immutable revision/content 保留;未运行 vacuum。 +- 4 客户端各自使用不同 workspace,共享同一个 DB 和元数据族。全部测量的 CAS retry 均为 0。 +- Holt 的 cache target 为共享 64 MiB;SQLite 为每连接 64 MiB。它们不等于相同的总内存限制:本次 SQLite 最大 RSS 小于 20 MiB,Holt 的受保护 dirty/COW 帧则能远超 cache target。下表另报实际峰值 RSS。 + +每个场景运行 8 个独立进程,顺序为 H,S,S,H / S,H,H,S,每引擎 4 次。下面是每进程指标的中位数;p95 也是 4 个进程 p95 的中位数,不是池化分位数。共 128 次正式运行全部通过完整有序领域数据校验、跨引擎/重复运行一致性检查和关闭重开检查;另有不计入统计的 smoke/runner QA。 + +## 持久化配置 + +必须分开看这些行: + +| Profile | Holt | SQLite | 用途与限制 | +|---|---|---|---| +| 异步诊断 OFF | WAL `sync=false` | WAL `synchronous=OFF` | 两者无逐操作同步,但 buffering 和 crash consistency 不相同;不作为严格持久性等价比较。 | +| 周期 checkpoint / NORMAL | WAL `sync=false`,默认后台 checkpoint 开启 | WAL `NORMAL`,autocheckpoint=1000 pages | Holt 默认 500 ms/512 dirty blobs,SQLite 为 1000 WAL pages;同步周期和每轮字节数不同,按各自策略报告。 | +| 同步 FULL | WAL `sync=true` | WAL `FULL` | SQLite 同时开启 `fullfsync=ON` 和 `checkpoint_fullfsync=ON`,与此 macOS Rust `File::sync_data()` 的 `F_FULLFSYNC` 路径对齐。 | + +4 KiB 内容的同步发布在两边都同步内容文件和父目录。基准未做断电证明。每个 burst 都有最终显式 durable checkpoint,耗时单独报告,也保留包含该 drain 的吞吐。 + +attached 场景还将相同结构的 canonical mutation 与领域数据同事务保存:Holt 使用 attached WAL envelope,SQLite 使用事务内 recovery table。最终 checkpoint 退休这些本地恢复材料。它不是共享日志、远程复制或永久审计日志的比较。 + +## 更接近实际服务配置的结果 + +| 工作流 | 客户端 | 请求数/进程 | Holt ops/s | SQLite ops/s | H/S | Holt p95 | SQLite p95 | +|---|---:|---:|---:|---:|---:|---:|---:| +| 纯元数据,后台 checkpoint / NORMAL | 4 | 4,000 | 1,975 | 1,558 | 1.27× | 7.39 ms | 9.26 ms | +| 4 KiB 内容,后台 checkpoint / NORMAL | 4 | 4,000 | 1,755 | 1,680 | 1.04× | 4.68 ms | 7.53 ms | +| 纯元数据,同步 FULL,清理后复测 | 1 | 4,000 | 698 | 656 | 1.06× | 6.28 ms | 7.21 ms | +| 4 KiB 内容,同步 FULL,清理后复测 | 1 | 1,600 | 400 | 313 | 1.28× | 9.10 ms | 11.67 ms | +| canonical recovery + 4 KiB 内容,同步 FULL | 1 | 1,600 | 429 | 315 | 1.36× | 9.09 ms | 11.53 ms | + +Holt 在这些混合工作流中没有普遍输给 SQLite。4 客户端含内容的 NORMAL 行只有约 4% 差距,低于本机跨进程波动,按接近处理。 + +同步测试对设备状态敏感。第一组在大体积 COW 压力组之后,普通纯元数据 FULL 的中位数为 H 553 / S 753 ops/s,普通内容 FULL 为 H 323 / S 366;恢复磁盘可用空间后,另做完整平衡顺序复测得到上表。两组均保留,不用单个同步数字宣称可移植排名,也不能把 ordinary 与 attached 不同时间窗口的差异解释成 attached 本身加速。 + +## 多 Tree 读取成本被总吞吐掩盖 + +单客户端、纯元数据、异步 OFF 诊断中,整个 mix 是 H 20,124 / S 16,216 ops/s;但逐操作分布不同: + +| 操作 | Holt p50 | SQLite p50 | +|---|---:|---:| +| read:3 族一致性读 | 37.7 µs | 4.0 µs | +| stat:3 族一致性读 | 37.9 µs | 4.0 µs | +| list:2 族一致性读、64 条 | 73.3 µs | 47.8 µs | +| snapshot:8 族、每族最多 32 条 | 155.0 µs | 73.3 µs | +| publish:8 族原子提交 | 37.0 µs | 134.3 µs | + +这组负载里,Holt 的便宜写入拉高了混合吞吐,同时一致性 point read 约慢 9–10 倍。原始 stage timer 的 H view+reads 为约 492 ms,apply 为约 52 ms;S 分别为约 113 ms 与 568 ms。stage timer 含捕获、等待、实际读取/回调和释放,不能进一步当成纯锁时间或纯 memcpy 时间。 + +诊断中仅将 read/stat 改成独立 point lookup 后,H 的整个 mix 从 20,124 升到 33,571 ops/s,p50 约 2.2 µs;S 是 16,299 ops/s。其余操作仍使用原来的事务/视图。这个诊断弱化了跨 Tree 一致性,不能作为可直接发布的优化。 + +## 并发 COW、内存和空间放大 + +| 场景 | H ops/s | H 峰值 RSS 中位数 | H checkpoint 后元数据分配空间 | H orphan backlog 中位数 | +|---|---:|---:|---:|---:| +| 4 客户端,4,000 请求,关闭后台 checkpoint | 1,000 | 1,391 MiB | 1,323 MiB | 1,071 | +| 4 客户端,12,000 请求,关闭后台 checkpoint | 230 | 6,639 MiB | 8,372 MiB | 8,062 | +| 4 客户端,4,000 请求,后台 checkpoint / NORMAL 对照 | 1,975 | 454 MiB | 1,421 MiB | 936 | +| 4 客户端,4 KiB 内容,后台 checkpoint / NORMAL 对照 | 1,755 | 385 MiB | 1,047 MiB | 750 | + +以上分配空间不含内容文件,也不是领域数据净大小。NORMAL 对照中,SQLite 对应峰值 RSS 约 11 MiB、元数据文件约 2.8–3.1 MiB。Holt 长压力组最高单次 RSS 约 8 GiB。压力组最终业务可达拓扑约 30 个 blobs,store 则还保留数千个待回收槽位。 + +关闭后台 checkpoint 的 OFF 对照中,SQLite 4,000 请求约 14,960 ops/s,12,000 请求约 14,776 ops/s;开启自动 checkpoint 但仍为 OFF 时,SQLite 约 20,950 ops/s。这些数字比 NORMAL 高很多,证明持久性/同步策略是决定性变量,不能拿 OFF 行宣称生产 SQLite 比 Holt 快 10–60 倍。 + +后台 checkpoint 明显降低了 Holt 内存压力并改善吞吐,但没有消除固定粒度 COW 和回收积压。它也不等于物理文件立刻缩小。这里观察的是一次有界 burst 和最终 checkpoint,不是永久泄漏证明,也不是所有维护收敛后稳态容量。 + +## 与代码路径的联系及优化顺序 + +当前 `DB::view` 为每个 scope 分配、复制完整 512 KiB root。一次 read/stat 捕获 3 族,光 root memcpy 就是 1.5 MiB,当前还有完整清零、epoch 注册及 live-root dirty publication。它与 SQLite 建立 read transaction 的成本差异一致。 + +并发 view 存活时,512 KiB child frame COW 与 dirty/orphan 保护会让 cache target 失去硬上限;当前全 DB fork barrier 还会保守影响未捕获 Tree。即使改成 per-Tree barrier,同一 metadata family 内不同 workspace 仍可能共享一个物理 frame,因此它只会消除一部分保守 fork。 + +基于本次结果,优先级是: + +1. 先降低 DBView 的 root capture 成本,保留一致性与 escaped-view lifetime 语义。 +2. 缩小 snapshot/COW domain,完善 dirty/orphan 的有界回收、内存背压和观测。 +3. 再缩短 atomic 持 gate 的窗口、批量处理 delta flush、复用 tree resolution。单客户端 async 的 apply 已很便宜,不能预期仅削减 batch 包装就解决并发放大。 +4. 后续再评估 immutable root descriptor、较小 COW 单元或版本化读。point-read 诊断给出的是方向和上限,不是正确性等价的替代方案。 + +本轮没有修改 Holt 的 atomic/view/COW 实现。 + +## 复现与证据 + +- [方法和运行命令](README.md) +- [应用 handler 与两种存储适配器](src/main.rs) +- [交错执行、状态校验与报告脚本](run_comparison.py) +- [汇总数据和原始证据说明](results/README.md) + +原始 3 组实验保留完整的每操作延迟、命令、逐轮环境、`time -l`、领域 fingerprint、二进制 SHA、对应 source archive 和详细报告。128 次正式运行没有失败或 CAS retry。额外 runner QA 验证了自动清理只作用于本次生成且已通过校验的测试库。 + +机器是带桌面后台负载的 macOS 开发机,未绑核、未控制 SSD 状态,不是 Linux 生产节点或冷缓存/严格同 RSS 测试。容量、同步、周期 checkpoint 和 OFF 诊断各自保留;不合并成一个总分。 diff --git a/benches/agent_workspace/RESULTS_OPTIMIZATION_2026-10-15.md b/benches/agent_workspace/RESULTS_OPTIMIZATION_2026-10-15.md new file mode 100644 index 0000000..ae9a73a --- /dev/null +++ b/benches/agent_workspace/RESULTS_OPTIMIZATION_2026-10-15.md @@ -0,0 +1,113 @@ +# DB::view 捕获与 COW 回收优化:最终复测 + +本次改动解决跨 Tree view 的重复捕获工作,以及短命 COW 版本不断进入 +checkpoint/store 的放大。没有改变公开 API、磁盘格式、跨 Tree 一致性、 +escaped view/cursor 生命周期或 WAL 同步确认边界。 + +基线:`f080daf2bc9d8019bb134141dc5ee31fd36f2c03`(0.9.2)。 +修改版:测量时为同一基线上的本地补丁(`git diff --binary`),补丁 SHA256 +`53a9b55f9a0dd8df8763229d1892c46863b1463680c40b4ffd222f29412157f1`。 +两版使用完全相同的 Rust workspace handler、Cargo.lock 和 release 设置。 +SQLite 为同一驱动中的 bundled 3.51.3。 + +提交 PR 时另整合了 main 的 `36fb295`(manifest dual-writer 防护)及依赖更新; +下列正式性能数字仍对应上述测量版本,不冒充整合后 main 的完整复测结果。 + +## 测量边界 + +Apple M3 Pro、36 GiB RAM、12 logical CPUs,macOS,Rust 1.97.1。 +8 workspaces × 512 预置 paths;原有八类 metadata、请求比例、JSON 编解码、 +guarded mutation、内容 CRC、4 KiB 内容读写和 checkpoint 策略均未变。 +这不是 NoKV SDK/RPC 或远端对象存储测试。 + +11 个场景,原版 / 修改版 / SQLite 各 4 次独立进程,共 132 次;另为 +attached metadata 同步档补测 12 次。采用正反配对顺序,保留每次测量、 +全部延迟样本、环境/磁盘余量、峰值 RSS、命令和二进制校验值。 +144 次均通过完整有序域指纹(CRC32、各表行数/字节数)及重开校验;CAS 重试为 0。 +测量时没有并行运行编译器、测试或其他本任务基准。桌面背景负载仍然存在。 + +以下吞吐不含最后一次 checkpoint;完整报告同时给出含最终 drain 的吞吐。 +异步 OFF 是诊断档,不能当作 Holt/SQLite 崩溃语义等价的生产对比。 +`background_normal` 保留各自原生策略:Holt 500 ms / 512 dirty blobs, +SQLite NORMAL / 1000-page autocheckpoint,并非相同刷盘节奏。同步档使用 +Holt sync=true、SQLite FULL,并打开 macOS fullfsync/checkpoint_fullfsync。 +未额外调用 vacuum;后台维护保持原配置。 + +## 请求吞吐:每组进程中位数 + +| 场景 | 原版 Holt ops/s | 修改版 Holt ops/s | SQLite ops/s | 修改版/原版 | +|---|---:|---:|---:|---:| +| metadata async,1 client,12k ops | 19,674 | 23,188 | 14,952 | 1.18× | +| metadata async,4 clients,4k ops,无后台 | 1,003 | 7,102 | 15,133 | 7.08× | +| metadata async,4 clients,12k ops,无后台 | 226 | 5,826 | 13,725 | 25.76× | +| metadata async,4 clients,4k ops,原生后台/NORMAL | 1,971 | 22,098 | 1,877 | 11.21× | +| content async,1 client,6k ops | 2,217 | 2,190 | 1,707 | 0.99× | +| content async,4 clients,6k ops,无后台 | 761 | 4,593 | 4,128 | 6.03× | +| content async,4 clients,4k ops,原生后台/NORMAL | 1,746 | 3,596 | 1,860 | 2.06× | +| metadata sync,1 client,4k ops | 774 | 772 | 793 | 1.00× | +| content sync,1 client,1.6k ops | 390 | 402 | 351 | 1.03× | +| attached metadata sync,1 client,4k ops | 804 | 780 | 838 | 0.97× | +| attached content sync,1 client,1.6k ops | 401 | 409 | 340 | 1.02× | + +新版本四并发 metadata 有明显运行波动:4k 无后台为 6,425–21,126 ops/s, +12k 无后台为 5,586–12,546 ops/s,不能只引用最快一次。 +短后台负载可能在首个周期 checkpoint 前结束,因此 11.21× 不代表长程稳态。 +另外完成的 100k 后台 metadata 正确性压力测试约为 3,523 ops/s;该次与测试 +并行运行,不纳入正式性能矩阵,也不是严格稳态资格测试。 + +## 放大与最终 drain + +| 场景 | 峰值 RSS MiB,旧 → 新 | metadata allocated MiB,旧 → 新 | 最终 checkpoint ms,旧 → 新 | checkpoint 后 COW backlog,旧 → 新 | +|---|---:|---:|---:|---:| +| metadata 4 clients / 4k / 无后台 | 1,388 → 87.6 | 1,325 → 21.2 | 1,561 → 40.3 | 1,068.5 → 0 | +| metadata 4 clients / 12k / 无后台 | 6,800 → 107.0 | 8,160 → 40.7 | 12,620 → 70.1 | 7,849.5 → 0 | +| metadata 4 clients / 原生后台 | 423 → 79.9 | 1,430 → 26.2 | 159.7 → 38.0 | 897.5 → 0 | +| content 4 clients / 无后台 | 1,824 → 75.4 | 1,774 → 28.7 | 2,033 → 54.8 | 1,498 → 0 | +| content 4 clients / 原生后台 | 406 → 72.3 | 1,111 → 27.4 | 32.3 → 39.6 | 572 → 0 | + +12k metadata 包含最后 drain 的吞吐为 183 → 5,644 ops/s(30.9×); +请求 p95 为 61.3 ms → 2.61 ms。Holt 的 64 MiB 仍只是 cache target, +不是严格 RSS 上限;SQLite 实测 RSS 更低,不能声称两者总内存相等。 + +单线程 metadata read p50:38.54 → 29.83 µs;stat:38.63 → 29.98 µs; +8-family snapshot:154.10 → 134.69 µs。view+reads 阶段总计 +501.9 → 411.4 ms;这是包含实际查询的阶段计时,不是纯 memcpy/capture 微基准。 + +## 同步档的补充确认 + +attached metadata 首轮 804.2 → 779.9 ops/s(-3.0%),补测 +796.9 → 795.0 ops/s(-0.24%)。首轮新旧范围为 773–845 / 774–809, +补测为 777–818 / 758–806,均重叠。两轮每个 Holt 进程 WAL sync 数均为 +1,492;view+reads 阶段均下降,apply/fsync 阶段有波动。 +没有复现稳定的同步退化,也不能据此承诺所有负载绝对零回退。 + +## 实现与验证要点 + +- 多根复制后批量注册独立 epoch lease;完整覆盖的 buffer 不再先清零。 + named-tree 捕获不再逐次脏写根,DB checkpoint 在 catalog 根汇总 epoch。 +- orphan 仅等待创建至脱离之间真正可能引用它的 epoch;按阻塞 epoch 索引, + 后来的快照不能延长旧版本寿命。GC 根 pin 也保留 epoch 保护。 +- 未进入底层 store 的退役版本可被有界丢弃;新副本/合并父页继承 WAL 低水位。 + 子页在释放写 latch 前发布 dirty debt。已持久化 GUID 仍等待安全回收边界。 +- dirty capture 与临时版本回收互斥;只有旧 flushing claims 全部退出,才整体 + 取消退役图像,避免旧 checkpoint 父图像依赖已丢弃的子图像。 + 回收队列非空时不再每次 view drop 都唤醒后台线程。 + +迭代先导测试确实抓到上述父子依赖竞态,相关失败日志保留但不计入正式结果。 +最终版本通过 484 个 all-features 单元测试、整套 lib/integration/example 测试、 +Clippy、doctest,以及额外的 No-WAL 并发快照/写入/checkpoint 测试。 +最终源码还通过 100k 操作的 DB oracle soak,以及 30 秒、95 轮 +SIGKILL/reopen 的已确认跨 Tree 写入恢复 smoke(非长期资格测试)。 +新增进程退出测试验证多代 COW 被丢弃后、不做最终 checkpoint,sync WAL +仍能恢复全部已确认数据。另完成 100k metadata、40k 不同 seed metadata、 +12k content 的后台正确性压力测试,均重开成功且无后台错误日志。 +40k 扩展测试在一次有界清理后仍有 76 个待回收 GUID,不是无限积压已被全面 +证明消失;本轮也没有做 Linux/io_uring、断电或长时间稳态资格测试。 + +仍有每个 Tree 一次 512 KiB 根帧复制,以及全局 COW/GC epoch 的保守开销。 +SQLite OFF 在四并发 metadata 的吞吐和 p95 上仍领先;本次不是所有维度的 +SQLite 胜负结论,也未修改 NoKV 的 OCC、业务历史或 outbox 保留策略。 + +聚合数据见 `results/optimization-2026-10-15/`。原始证据与精确源码包保留在本地 +`results/optimization-evidence-2026-10-15.tar.gz`,不提交大体积原始日志及桌面进程 +环境快照;SHA256 与内容说明见 `results/README.md`。可通过同一驱动重新生成。 diff --git a/benches/agent_workspace/results/README.md b/benches/agent_workspace/results/README.md new file mode 100644 index 0000000..f1b2be7 --- /dev/null +++ b/benches/agent_workspace/results/README.md @@ -0,0 +1,50 @@ +# 2026-10-15 evidence + +`summary.json` contains per-case medians, per-run ranges, per-operation latency, +checkpoint/drain, allocated metadata space, and measured process maximum RSS. + +The full raw archives are retained locally and intentionally not committed: +they include per-process desktop environment snapshots and multi-megabyte raw +latency samples. The checked-in reports and summary JSON contain the aggregate +results; the runners regenerate the complete evidence on a fresh run. Archive +checksums remain in `SHA256SUMS` and the corresponding `.sha256` file. A checksum +is provenance for the retained local bundle, not a public download link. + +The local `evidence-2026-10-15.tar.gz` contains three groups: + +- `comparison`: original 72-process matrix, including the checkpoint-disabled + pressure reproduction, FULL and attached modes, and the point-read diagnostic. +- `followup`: 40 processes for equal-length checkpoint off/on comparisons and + the balanced FULL replication after restoring free disk space. +- `native-checkpoint`: 16 processes with SQLite WAL NORMAL, fullfsync enabled, + native WAL autocheckpoint, and Holt background checkpoint enabled. + +Each group includes exact commands, per-operation latency samples in raw JSON, +complete ordered-domain CRC/count/byte fingerprints, resource accounting, +environment snapshots, and generated detailed reports. Binary hashes are in +each environment record; source archives capture the driver used for the +corresponding phase. `SHA256SUMS` checks the evidence files. + +Generated test databases are removed after their data/reopen checks and +statistics have been captured. Their bytes are reproducible by running the +preserved command with the associated source and Holt commit. They are not +included in the archive. The current runner removes validated databases by +default; use `--keep-stores` when inspecting their physical topology. + +The separate runner QA is excluded from the 128-process result set. + +## DB::view and COW optimization + +`optimization-2026-10-15/summary.json` and `RESULTS.md` summarize 132 balanced +baseline/patched/SQLite processes. `sync-followup-summary.json` retains the +additional 12-process attached-metadata sync replication. See +`../RESULTS_OPTIMIZATION_2026-10-15.md` for the implementation, caveats, validation +and source provenance. + +The local `optimization-evidence-2026-10-15.tar.gz` also includes the raw runs, +frozen baseline source, exact driver, measured library patch, test/soak logs, +and explicitly separated failed preflight and correctness-only stress runs. +Its SHA256 is +`9fef9fed20e86412dee9a741742698868a0114c3b334f69e446ff947a4d9a62e`. +The unredacted environment and validation logs beside the summaries are likewise +local-only; no generated database is committed or included in the archive. diff --git a/benches/agent_workspace/results/SHA256SUMS b/benches/agent_workspace/results/SHA256SUMS new file mode 100644 index 0000000..17e799f --- /dev/null +++ b/benches/agent_workspace/results/SHA256SUMS @@ -0,0 +1,2 @@ +4c8008385deb34295819d42fa4442ae09e8f3c5a368ef0f532449b07f81ca2bc summary.json +6b76df2c534c735d22513d80278a12b154cda2278ebbd4ed6f12392ba96a26de evidence-2026-10-15.tar.gz diff --git a/benches/agent_workspace/results/optimization-2026-10-15/RESULTS.md b/benches/agent_workspace/results/optimization-2026-10-15/RESULTS.md new file mode 100644 index 0000000..9608457 --- /dev/null +++ b/benches/agent_workspace/results/optimization-2026-10-15/RESULTS.md @@ -0,0 +1,33 @@ +# Agent workspace: baseline / patched Holt / SQLite + +Unchanged shared workspace handler, workload, dataset, consistency, durability and checkpoint policies. Four independent processes per variant in balanced order; medians of per-process metrics. No concurrent compiler/tests/benchmarks during these runs. Each database is fresh and removed only after full domain fingerprint and clean-reopen validation. Raw samples, commands, resource logs and environment remain alongside this report. + +Async OFF is diagnostic, not crash-equivalent to Holt. Background `_normal` uses native checkpoint policies (Holt 500 ms/512 blobs, SQLite NORMAL/1000 pages), not matched checkpoint cadence. Sync uses Holt sync=true and SQLite FULL with macOS fullfsync/checkpoint_fullfsync. These are bounded desktop workloads, not long-running maintenance or power-cut qualification. + +| Case | Old H ops/s | New H ops/s | SQLite ops/s | New/old | New/SQLite | Old/new p95 us | +|---|---:|---:|---:|---:|---:|---:| +| attached_content_sync_1 | 401 | 409 | 340 | 1.02x | 1.20x | 9834.4 / 9736.7 | +| attached_metadata_sync_1 | 804 | 780 | 838 | 0.97x | 0.93x | 5711.4 / 5921.2 | +| content_async_1 | 2,217 | 2,191 | 1,707 | 0.99x | 1.28x | 664.4 / 562.7 | +| content_async_4 | 761 | 4,593 | 4,128 | 6.03x | 1.11x | 12930.8 / 959.6 | +| content_async_4_background_normal | 1,746 | 3,596 | 1,860 | 2.06x | 1.93x | 5328.4 / 930.6 | +| content_sync_1 | 390 | 402 | 351 | 1.03x | 1.15x | 9695.6 / 9963.8 | +| metadata_async_1 | 19,674 | 23,188 | 14,952 | 1.18x | 1.55x | 79.9 / 73.4 | +| metadata_async_4 | 226 | 5,826 | 13,725 | 25.76x | 0.42x | 61299.6 / 2611.4 | +| metadata_async_4_background_normal | 1,971 | 22,098 | 1,877 | 11.21x | 11.78x | 7300.4 / 613.8 | +| metadata_async_4_short | 1,003 | 7,102 | 15,133 | 7.08x | 0.47x | 14277.2 / 2194.9 | +| metadata_sync_1 | 774 | 772 | 793 | 1.00x | 0.97x | 5965.5 / 5945.1 | + +| Case | Old/new RSS MiB | Old/new metadata MiB | Old/new final checkpoint ms | Old/new post-drain orphan debt | +|---|---:|---:|---:|---:| +| attached_content_sync_1 | 47.6 / 46.3 | 25.1 / 21.4 | 27.7 / 25.9 | 0 / 0 | +| attached_metadata_sync_1 | 47.6 / 48.2 | 20.7 / 21.4 | 46.2 / 46.8 | 0 / 0 | +| content_async_1 | 52.0 / 49.1 | 26.3 / 24.1 | 50.6 / 46.3 | 0 / 0 | +| content_async_4 | 1824.1 / 75.4 | 1774.2 / 28.7 | 2033.3 / 54.8 | 1498 / 0 | +| content_async_4_background_normal | 405.9 / 72.3 | 1110.6 / 27.4 | 32.3 / 39.6 | 572 / 0 | +| content_sync_1 | 47.8 / 45.7 | 23.6 / 21.6 | 18.4 / 15.9 | 0 / 0 | +| metadata_async_1 | 62.6 / 63.9 | 30.2 / 29.2 | 49.6 / 48.2 | 0 / 0 | +| metadata_async_4 | 6800.3 / 107.0 | 8160.1 / 40.7 | 12619.7 / 70.1 | 7850 / 0 | +| metadata_async_4_background_normal | 423.4 / 79.9 | 1430.2 / 26.2 | 159.7 / 38.0 | 898 / 0 | +| metadata_async_4_short | 1388.2 / 87.6 | 1325.4 / 21.2 | 1561.3 / 40.3 | 1068 / 0 | +| metadata_sync_1 | 49.2 / 49.5 | 20.6 / 20.6 | 37.4 / 40.1 | 0 / 0 | diff --git a/benches/agent_workspace/results/optimization-2026-10-15/summary.json b/benches/agent_workspace/results/optimization-2026-10-15/summary.json new file mode 100644 index 0000000..233e080 --- /dev/null +++ b/benches/agent_workspace/results/optimization-2026-10-15/summary.json @@ -0,0 +1,2136 @@ +{ + "attached_content_sync_1": { + "baseline": { + "runs": 4, + "ops_per_second": 401.1923550971288, + "ops_range": [ + 363.93101335083765, + 411.28219910214654 + ], + "drain_ops_per_second": 398.42023954114296, + "p50_us": 529.0415, + "p95_us": 9834.4375, + "p99_us": 11985.8955, + "rss_mib": 47.578125, + "metadata_mib": 25.12890625, + "checkpoint_ms": 27.7389165, + "view_ms": 81.86060599999999, + "apply_ms": 1644.7502275, + "cas_retries": 0, + "orphan_backlog": 0.0, + "live_slots": 15.0, + "per_operation": { + "list": { + "p50_us": 87.1665, + "p95_us": 112.4165, + "p99_us": 135.541, + "mean_us": 88.5337991631799 + }, + "publish": { + "p50_us": 6922.3955, + "p95_us": 10847.4375, + "p99_us": 11989.833, + "mean_us": 7272.853043103449 + }, + "read": { + "p50_us": 552.125, + "p95_us": 1512.0625, + "p99_us": 3086.208, + "mean_us": 741.183556726094 + }, + "remove_recreate": { + "p50_us": 9815.5, + "p95_us": 14220.8335, + "p99_us": 19219.479, + "mean_us": 9790.654075471699 + }, + "rename_roundtrip": { + "p50_us": 5861.583500000001, + "p95_us": 7675.353999999999, + "p99_us": 8002.4585, + "mean_us": 5627.206966666667 + }, + "snapshot": { + "p50_us": 170.3125, + "p95_us": 225.958, + "p99_us": 292.5415, + "mean_us": 178.78578 + }, + "stat": { + "p50_us": 43.0835, + "p95_us": 66.8335, + "p99_us": 101.771, + "mean_us": 46.38331069958848 + } + } + }, + "patched": { + "runs": 4, + "ops_per_second": 409.31923408046464, + "ops_range": [ + 390.5953310614315, + 437.77223319483886 + ], + "drain_ops_per_second": 406.5285822954589, + "p50_us": 481.4165, + "p95_us": 9736.7495, + "p99_us": 12002.729, + "rss_mib": 46.3359375, + "metadata_mib": 21.423828125, + "checkpoint_ms": 25.894479, + "view_ms": 74.07381799999999, + "apply_ms": 1596.1660044999999, + "cas_retries": 0, + "orphan_backlog": 0.0, + "live_slots": 15.0, + "per_operation": { + "list": { + "p50_us": 82.979, + "p95_us": 110.729, + "p99_us": 170.70850000000002, + "mean_us": 85.33966108786612 + }, + "publish": { + "p50_us": 6802.833500000001, + "p95_us": 10707.3335, + "p99_us": 11962.8335, + "mean_us": 7139.013566091953 + }, + "read": { + "p50_us": 504.33299999999997, + "p95_us": 1558.167, + "p99_us": 4011.8545000000004, + "mean_us": 684.182641815235 + }, + "remove_recreate": { + "p50_us": 9596.707999999999, + "p95_us": 13898.563, + "p99_us": 14118.8125, + "mean_us": 9900.975226415094 + }, + "rename_roundtrip": { + "p50_us": 5735.853999999999, + "p95_us": 7230.9794999999995, + "p99_us": 9010.1245, + "mean_us": 5581.844713333334 + }, + "snapshot": { + "p50_us": 154.458, + "p95_us": 200.39600000000002, + "p99_us": 244.9375, + "mean_us": 161.56416000000002 + }, + "stat": { + "p50_us": 37.1665, + "p95_us": 57.021, + "p99_us": 94.8745, + "mean_us": 40.237430041152265 + } + } + }, + "sqlite": { + "runs": 4, + "ops_per_second": 340.18212177047747, + "ops_range": [ + 320.01405924166744, + 368.1851728748713 + ], + "drain_ops_per_second": 338.12252403542664, + "p50_us": 561.4795, + "p95_us": 11312.75, + "p99_us": 14020.479, + "rss_mib": 13.0390625, + "metadata_mib": 4.53125, + "checkpoint_ms": 31.478541999999997, + "view_ms": 83.24010949999999, + "apply_ms": 1947.185247, + "cas_retries": 0, + "orphan_backlog": 0.0, + "live_slots": 0.0, + "per_operation": { + "list": { + "p50_us": 147.416, + "p95_us": 361.041, + "p99_us": 455.9375, + "mean_us": 176.64507531380752 + }, + "publish": { + "p50_us": 8504.417, + "p95_us": 12600.0835, + "p99_us": 15301.875, + "mean_us": 8700.680808908048 + }, + "read": { + "p50_us": 586.7915, + "p95_us": 2138.9790000000003, + "p99_us": 4571.103999999999, + "mean_us": 833.984304700162 + }, + "remove_recreate": { + "p50_us": 11622.229, + "p95_us": 15909.667, + "p99_us": 17933.041, + "mean_us": 11599.129283018869 + }, + "rename_roundtrip": { + "p50_us": 6286.8544999999995, + "p95_us": 8355.1875, + "p99_us": 10376.9995, + "mean_us": 6411.62996 + }, + "snapshot": { + "p50_us": 247.20800000000003, + "p95_us": 434.64599999999996, + "p99_us": 502.1875, + "mean_us": 256.54254000000003 + }, + "stat": { + "p50_us": 17.124499999999998, + "p95_us": 75.5625, + "p99_us": 126.8755, + "mean_us": 27.005376543209877 + } + } + } + }, + "attached_metadata_sync_1": { + "baseline": { + "runs": 4, + "ops_per_second": 804.1743826208213, + "ops_range": [ + 773.1055706339329, + 845.1810326268668 + ], + "drain_ops_per_second": 796.4381374267338, + "p50_us": 182.833, + "p95_us": 5711.353999999999, + "p99_us": 7435.125, + "rss_mib": 47.59375, + "metadata_mib": 20.673828125, + "checkpoint_ms": 46.2080835, + "view_ms": 551.7672505, + "apply_ms": 4368.228413999999, + "cas_retries": 0, + "orphan_backlog": 0.0, + "live_slots": 14.0, + "per_operation": { + "list": { + "p50_us": 188.70850000000002, + "p95_us": 364.771, + "p99_us": 619.917, + "mean_us": 207.65113813559321 + }, + "publish": { + "p50_us": 3063.2709999999997, + "p95_us": 4031.125, + "p99_us": 4965.2715, + "mean_us": 3076.5877909836063 + }, + "read": { + "p50_us": 106.6665, + "p95_us": 288.583, + "p99_us": 449.95849999999996, + "mean_us": 131.82173801916935 + }, + "remove_recreate": { + "p50_us": 5972.9585, + "p95_us": 8041.8745, + "p99_us": 12491.5625, + "mean_us": 6082.188870833334 + }, + "rename_roundtrip": { + "p50_us": 5926.3330000000005, + "p95_us": 7938.3125, + "p99_us": 10006.229500000001, + "mean_us": 5956.871223618091 + }, + "snapshot": { + "p50_us": 491.0, + "p95_us": 915.0835, + "p99_us": 1102.9795, + "mean_us": 501.1663642857143 + }, + "stat": { + "p50_us": 102.6665, + "p95_us": 282.02099999999996, + "p99_us": 412.58349999999996, + "mean_us": 128.77903820598007 + } + } + }, + "patched": { + "runs": 4, + "ops_per_second": 779.9119880420842, + "ops_range": [ + 774.3648641235283, + 808.8013628425902 + ], + "drain_ops_per_second": 772.4353753127554, + "p50_us": 178.563, + "p95_us": 5921.2080000000005, + "p99_us": 7377.6665, + "rss_mib": 48.171875, + "metadata_mib": 21.412109375, + "checkpoint_ms": 46.757687000000004, + "view_ms": 520.9463860000001, + "apply_ms": 4536.077284999999, + "cas_retries": 0, + "orphan_backlog": 0.0, + "live_slots": 14.0, + "per_operation": { + "list": { + "p50_us": 197.375, + "p95_us": 347.10450000000003, + "p99_us": 524.417, + "mean_us": 212.96585762711865 + }, + "publish": { + "p50_us": 3129.8540000000003, + "p95_us": 4022.6040000000003, + "p99_us": 4841.8745, + "mean_us": 3184.2685515222483 + }, + "read": { + "p50_us": 106.2085, + "p95_us": 260.60400000000004, + "p99_us": 419.27099999999996, + "mean_us": 122.43777348242811 + }, + "remove_recreate": { + "p50_us": 6024.458500000001, + "p95_us": 7992.4169999999995, + "p99_us": 8307.604, + "mean_us": 6172.9558875 + }, + "rename_roundtrip": { + "p50_us": 6033.875, + "p95_us": 7955.125, + "p99_us": 11489.354, + "mean_us": 6121.352896984925 + }, + "snapshot": { + "p50_us": 507.10400000000004, + "p95_us": 916.5625, + "p99_us": 1168.979, + "mean_us": 543.0520428571429 + }, + "stat": { + "p50_us": 100.2085, + "p95_us": 242.95850000000002, + "p99_us": 366.72900000000004, + "mean_us": 118.80302657807309 + } + } + }, + "sqlite": { + "runs": 4, + "ops_per_second": 837.9735100209382, + "ops_range": [ + 810.6968682424083, + 857.3837054119614 + ], + "drain_ops_per_second": 827.068256241871, + "p50_us": 45.896, + "p95_us": 5680.292, + "p99_us": 7976.563, + "rss_mib": 18.34375, + "metadata_mib": 8.79296875, + "checkpoint_ms": 54.527812999999995, + "view_ms": 148.1776625, + "apply_ms": 4543.847986000001, + "cas_retries": 0, + "orphan_backlog": 0.0, + "live_slots": 0.0, + "per_operation": { + "list": { + "p50_us": 152.8745, + "p95_us": 290.979, + "p99_us": 377.9795, + "mean_us": 167.61043474576272 + }, + "publish": { + "p50_us": 3196.6665000000003, + "p95_us": 4603.875, + "p99_us": 5087.812, + "mean_us": 3106.8798448477755 + }, + "read": { + "p50_us": 13.875499999999999, + "p95_us": 53.3125, + "p99_us": 83.4375, + "mean_us": 19.91889297124601 + }, + "remove_recreate": { + "p50_us": 6010.8125, + "p95_us": 8613.667, + "p99_us": 9961.3335, + "mean_us": 6132.692183333334 + }, + "rename_roundtrip": { + "p50_us": 5968.7080000000005, + "p95_us": 8767.4585, + "p99_us": 10073.812, + "mean_us": 6103.093489949749 + }, + "snapshot": { + "p50_us": 239.6045, + "p95_us": 397.75, + "p99_us": 496.77099999999996, + "mean_us": 251.54702857142857 + }, + "stat": { + "p50_us": 13.75, + "p95_us": 55.0, + "p99_us": 83.312, + "mean_us": 19.882239202657807 + } + } + } + }, + "content_async_1": { + "baseline": { + "runs": 4, + "ops_per_second": 2217.2485134064955, + "ops_range": [ + 1555.6305138999476, + 2420.4782138807163 + ], + "drain_ops_per_second": 2171.880282910002, + "p50_us": 110.7915, + "p95_us": 664.4165, + "p99_us": 1632.042, + "rss_mib": 52.03125, + "metadata_mib": 26.287109375, + "checkpoint_ms": 50.5808755, + "view_ms": 311.24956199999997, + "apply_ms": 35.1298065, + "cas_retries": 0, + "orphan_backlog": 0.0, + "live_slots": 17.0, + "per_operation": { + "list": { + "p50_us": 83.312, + "p95_us": 121.1665, + "p99_us": 236.77100000000002, + "mean_us": 90.71186139169473 + }, + "publish": { + "p50_us": 338.104, + "p95_us": 705.4795, + "p99_us": 1534.875, + "mean_us": 394.8270183266932 + }, + "read": { + "p50_us": 325.792, + "p95_us": 875.0625, + "p99_us": 2993.625, + "mean_us": 842.1542684592403 + }, + "remove_recreate": { + "p50_us": 380.60450000000003, + "p95_us": 923.8545, + "p99_us": 1874.4375, + "mean_us": 456.015625 + }, + "rename_roundtrip": { + "p50_us": 80.208, + "p95_us": 161.792, + "p99_us": 306.25, + "mean_us": 93.98472278911565 + }, + "snapshot": { + "p50_us": 177.95850000000002, + "p95_us": 260.91650000000004, + "p99_us": 418.625, + "mean_us": 187.96154435483868 + }, + "stat": { + "p50_us": 41.875, + "p95_us": 69.4375, + "p99_us": 141.875, + "mean_us": 46.99237236403995 + } + } + }, + "patched": { + "runs": 4, + "ops_per_second": 2190.5191444146476, + "ops_range": [ + 2044.0448223258752, + 2388.4785764667686 + ], + "drain_ops_per_second": 2154.772403720812, + "p50_us": 90.0835, + "p95_us": 562.7080000000001, + "p99_us": 1162.7705, + "rss_mib": 49.0703125, + "metadata_mib": 24.0546875, + "checkpoint_ms": 46.329896000000005, + "view_ms": 245.744052, + "apply_ms": 31.2950615, + "cas_retries": 0, + "orphan_backlog": 0.0, + "live_slots": 17.0, + "per_operation": { + "list": { + "p50_us": 75.7085, + "p95_us": 101.4375, + "p99_us": 157.375, + "mean_us": 80.20946296296296 + }, + "publish": { + "p50_us": 319.958, + "p95_us": 561.7294999999999, + "p99_us": 932.0, + "mean_us": 368.71961394422306 + }, + "read": { + "p50_us": 311.75, + "p95_us": 761.5625, + "p99_us": 2350.4165000000003, + "mean_us": 877.0443495518566 + }, + "remove_recreate": { + "p50_us": 350.83349999999996, + "p95_us": 568.4585, + "p99_us": 1044.1665, + "mean_us": 382.39478125 + }, + "rename_roundtrip": { + "p50_us": 67.7295, + "p95_us": 97.125, + "p99_us": 133.39600000000002, + "mean_us": 72.02906802721088 + }, + "snapshot": { + "p50_us": 152.22899999999998, + "p95_us": 200.562, + "p99_us": 272.5625, + "mean_us": 159.42276612903225 + }, + "stat": { + "p50_us": 33.2505, + "p95_us": 47.6455, + "p99_us": 68.5835, + "mean_us": 35.74500665926748 + } + } + }, + "sqlite": { + "runs": 4, + "ops_per_second": 1707.1232991168745, + "ops_range": [ + 1513.151858604844, + 1765.258607667525 + ], + "drain_ops_per_second": 1697.2287269037865, + "p50_us": 181.125, + "p95_us": 858.292, + "p99_us": 1751.75, + "rss_mib": 11.8828125, + "metadata_mib": 3.6875, + "checkpoint_ms": 20.49875, + "view_ms": 116.38860550000001, + "apply_ms": 353.73398199999997, + "cas_retries": 0, + "orphan_backlog": 0.0, + "live_slots": 0.0, + "per_operation": { + "list": { + "p50_us": 73.2915, + "p95_us": 184.187, + "p99_us": 257.5, + "mean_us": 85.19603591470258 + }, + "publish": { + "p50_us": 467.16650000000004, + "p95_us": 1159.75, + "p99_us": 1934.0625, + "mean_us": 560.9918737051793 + }, + "read": { + "p50_us": 296.58299999999997, + "p95_us": 912.2085, + "p99_us": 3025.2084999999997, + "mean_us": 1065.7163053777208 + }, + "remove_recreate": { + "p50_us": 576.1455000000001, + "p95_us": 1371.9995, + "p99_us": 3348.1665000000003, + "mean_us": 695.7413125 + }, + "rename_roundtrip": { + "p50_us": 230.22899999999998, + "p95_us": 582.938, + "p99_us": 1041.563, + "mean_us": 291.05321768707483 + }, + "snapshot": { + "p50_us": 104.021, + "p95_us": 254.1245, + "p99_us": 343.4375, + "mean_us": 127.06954838709677 + }, + "stat": { + "p50_us": 6.6665, + "p95_us": 22.979, + "p99_us": 55.1665, + "mean_us": 9.174933407325195 + } + } + } + }, + "content_async_4": { + "baseline": { + "runs": 4, + "ops_per_second": 761.2296211931737, + "ops_range": [ + 648.9131129422935, + 858.159173090041 + ], + "drain_ops_per_second": 598.6980516890137, + "p50_us": 2649.3959999999997, + "p95_us": 12930.75, + "p99_us": 27848.666, + "rss_mib": 1824.0703125, + "metadata_mib": 1774.173828125, + "checkpoint_ms": 2033.3179169999999, + "view_ms": 19608.104466, + "apply_ms": 2479.0036215, + "cas_retries": 0, + "orphan_backlog": 1498.0, + "live_slots": 1514.5, + "per_operation": { + "list": { + "p50_us": 3903.1459999999997, + "p95_us": 23988.229, + "p99_us": 39842.854, + "mean_us": 7153.9287508214675 + }, + "publish": { + "p50_us": 2760.7084999999997, + "p95_us": 9857.5415, + "p99_us": 16227.0415, + "mean_us": 3711.5340333896875 + }, + "read": { + "p50_us": 2405.375, + "p95_us": 7548.5, + "p99_us": 16500.4165, + "mean_us": 4061.360824268644 + }, + "remove_recreate": { + "p50_us": 5790.875, + "p95_us": 17960.561999999998, + "p99_us": 31201.313000000002, + "mean_us": 7094.895619791667 + }, + "rename_roundtrip": { + "p50_us": 4549.1044999999995, + "p95_us": 15577.2915, + "p99_us": 22322.0, + "mean_us": 5891.346827464789 + }, + "snapshot": { + "p50_us": 7169.9794999999995, + "p95_us": 24983.104, + "p99_us": 37014.6455, + "mean_us": 9259.158140186915 + }, + "stat": { + "p50_us": 1887.5415, + "p95_us": 6596.8755, + "p99_us": 11410.6455, + "mean_us": 2411.479105145414 + } + } + }, + "patched": { + "runs": 4, + "ops_per_second": 4593.111881945666, + "ops_range": [ + 4220.89639176672, + 8317.526659461213 + ], + "drain_ops_per_second": 4409.38903836717, + "p50_us": 280.167, + "p95_us": 959.6254999999999, + "p99_us": 2021.8125, + "rss_mib": 75.359375, + "metadata_mib": 28.68359375, + "checkpoint_ms": 54.804958, + "view_ms": 413.637282, + "apply_ms": 65.462016, + "cas_retries": 0, + "orphan_backlog": 0.0, + "live_slots": 17.0, + "per_operation": { + "list": { + "p50_us": 80.7715, + "p95_us": 327.958, + "p99_us": 713.083, + "mean_us": 125.16624151150054 + }, + "publish": { + "p50_us": 319.14549999999997, + "p95_us": 746.9794999999999, + "p99_us": 1205.9165, + "mean_us": 388.90063651732885 + }, + "read": { + "p50_us": 384.52049999999997, + "p95_us": 1287.396, + "p99_us": 3856.1459999999997, + "mean_us": 1267.3335438813351 + }, + "remove_recreate": { + "p50_us": 386.45849999999996, + "p95_us": 947.7085, + "p99_us": 1550.3125, + "mean_us": 467.7088958333333 + }, + "rename_roundtrip": { + "p50_us": 81.771, + "p95_us": 338.79150000000004, + "p99_us": 608.604, + "mean_us": 127.01724999999999 + }, + "snapshot": { + "p50_us": 201.417, + "p95_us": 682.9585, + "p99_us": 1957.104, + "mean_us": 320.13845794392523 + }, + "stat": { + "p50_us": 36.0205, + "p95_us": 146.6455, + "p99_us": 340.04150000000004, + "mean_us": 58.250540268456376 + } + } + }, + "sqlite": { + "runs": 4, + "ops_per_second": 4128.026349523325, + "ops_range": [ + 3620.0800424313925, + 4500.222478623731 + ], + "drain_ops_per_second": 4051.37560637364, + "p50_us": 300.1875, + "p95_us": 1702.7085000000002, + "p99_us": 4787.4375, + "rss_mib": 11.984375, + "metadata_mib": 3.671875, + "checkpoint_ms": 25.9985835, + "view_ms": 126.0452655, + "apply_ms": 1621.435484, + "cas_retries": 0, + "orphan_backlog": 0.0, + "live_slots": 0.0, + "per_operation": { + "list": { + "p50_us": 67.1045, + "p95_us": 103.625, + "p99_us": 198.2505, + "mean_us": 74.34148576122672 + }, + "publish": { + "p50_us": 458.20849999999996, + "p95_us": 4159.6045, + "p99_us": 11435.5835, + "mean_us": 1232.093289518174 + }, + "read": { + "p50_us": 330.4375, + "p95_us": 934.2085, + "p99_us": 2995.8959999999997, + "mean_us": 1318.642737948084 + }, + "remove_recreate": { + "p50_us": 602.396, + "p95_us": 7350.0835, + "p99_us": 22814.124499999998, + "mean_us": 1870.5911640625 + }, + "rename_roundtrip": { + "p50_us": 241.10399999999998, + "p95_us": 1609.3755, + "p99_us": 19996.8335, + "mean_us": 845.9227376760564 + }, + "snapshot": { + "p50_us": 132.375, + "p95_us": 203.97899999999998, + "p99_us": 361.5835, + "mean_us": 141.23659345794394 + }, + "stat": { + "p50_us": 8.9375, + "p95_us": 22.833, + "p99_us": 41.5625, + "mean_us": 12.041587248322148 + } + } + } + }, + "content_async_4_background_normal": { + "baseline": { + "runs": 4, + "ops_per_second": 1745.8354264611244, + "ops_range": [ + 1454.758179866574, + 1952.5368793030175 + ], + "drain_ops_per_second": 1702.5357790476342, + "p50_us": 1132.4379999999999, + "p95_us": 5328.375, + "p99_us": 9702.1875, + "rss_mib": 405.921875, + "metadata_mib": 1110.576171875, + "checkpoint_ms": 32.2603335, + "view_ms": 4885.587757, + "apply_ms": 640.1132505, + "cas_retries": 0, + "orphan_backlog": 572.0, + "live_slots": 588.0, + "per_operation": { + "list": { + "p50_us": 1582.8955, + "p95_us": 8905.1245, + "p99_us": 13604.4165, + "mean_us": 2718.4239170812607 + }, + "publish": { + "p50_us": 1785.5835, + "p95_us": 5190.333500000001, + "p99_us": 7733.25, + "mean_us": 2190.3483113087673 + }, + "read": { + "p50_us": 961.6875, + "p95_us": 3156.4790000000003, + "p99_us": 52163.0835, + "mean_us": 2378.953575627679 + }, + "remove_recreate": { + "p50_us": 2851.6665000000003, + "p95_us": 7425.1455, + "p99_us": 10180.1665, + "mean_us": 3336.564168 + }, + "rename_roundtrip": { + "p50_us": 1606.271, + "p95_us": 5661.354, + "p99_us": 8402.7085, + "mean_us": 2180.760926229508 + }, + "snapshot": { + "p50_us": 2571.875, + "p95_us": 10003.396, + "p99_us": 13746.917, + "mean_us": 3617.545190140845 + }, + "stat": { + "p50_us": 626.1875, + "p95_us": 2341.4165000000003, + "p99_us": 3678.5625, + "mean_us": 852.9286580267558 + } + } + }, + "patched": { + "runs": 4, + "ops_per_second": 3596.4952947590336, + "ops_range": [ + 3078.252744771388, + 4426.780361142276 + ], + "drain_ops_per_second": 3532.146820796316, + "p50_us": 286.39599999999996, + "p95_us": 930.6044999999999, + "p99_us": 2702.2915000000003, + "rss_mib": 72.2578125, + "metadata_mib": 27.37890625, + "checkpoint_ms": 39.577083, + "view_ms": 296.9416295, + "apply_ms": 43.249202, + "cas_retries": 0, + "orphan_backlog": 0.0, + "live_slots": 16.0, + "per_operation": { + "list": { + "p50_us": 75.4585, + "p95_us": 384.979, + "p99_us": 657.4580000000001, + "mean_us": 136.6710497512438 + }, + "publish": { + "p50_us": 325.5, + "p95_us": 755.7085, + "p99_us": 2728.875, + "mean_us": 408.5509428208386 + }, + "read": { + "p50_us": 397.2715, + "p95_us": 1154.1664999999998, + "p99_us": 4200.188, + "mean_us": 1613.9767924066136 + }, + "remove_recreate": { + "p50_us": 378.5625, + "p95_us": 753.604, + "p99_us": 1284.479, + "mean_us": 436.104508 + }, + "rename_roundtrip": { + "p50_us": 71.521, + "p95_us": 315.08349999999996, + "p99_us": 636.875, + "mean_us": 124.01321584699454 + }, + "snapshot": { + "p50_us": 164.72899999999998, + "p95_us": 1110.083, + "p99_us": 3570.4165000000003, + "mean_us": 416.0469718309859 + }, + "stat": { + "p50_us": 34.833, + "p95_us": 139.3955, + "p99_us": 333.91700000000003, + "mean_us": 52.21384448160535 + } + } + }, + "sqlite": { + "runs": 4, + "ops_per_second": 1859.9600374039305, + "ops_range": [ + 1693.4798327455812, + 2022.81311743352 + ], + "drain_ops_per_second": 1849.215459167795, + "p50_us": 396.58349999999996, + "p95_us": 7564.271, + "p99_us": 20286.9165, + "rss_mib": 11.3359375, + "metadata_mib": 3.529296875, + "checkpoint_ms": 10.672583, + "view_ms": 157.15418, + "apply_ms": 4186.01679, + "cas_retries": 0, + "orphan_backlog": 0.0, + "live_slots": 0.0, + "per_operation": { + "list": { + "p50_us": 93.2705, + "p95_us": 248.1875, + "p99_us": 426.29150000000004, + "mean_us": 120.68778192371477 + }, + "publish": { + "p50_us": 2649.125, + "p95_us": 11290.729, + "p99_us": 39541.667, + "mean_us": 4692.063106734435 + }, + "read": { + "p50_us": 456.813, + "p95_us": 1266.792, + "p99_us": 9505.0415, + "mean_us": 1272.7028505817516 + }, + "remove_recreate": { + "p50_us": 4613.0835, + "p95_us": 16553.6455, + "p99_us": 38459.5, + "mean_us": 6240.770004 + }, + "rename_roundtrip": { + "p50_us": 514.2705000000001, + "p95_us": 14029.646, + "p99_us": 65274.2505, + "mean_us": 4551.097448087432 + }, + "snapshot": { + "p50_us": 157.4375, + "p95_us": 498.41650000000004, + "p99_us": 1051.5625, + "mean_us": 236.30221830985914 + }, + "stat": { + "p50_us": 12.229, + "p95_us": 65.9375, + "p99_us": 141.3125, + "mean_us": 22.709413879598664 + } + } + } + }, + "content_sync_1": { + "baseline": { + "runs": 4, + "ops_per_second": 389.7392020985212, + "ops_range": [ + 359.01235588704117, + 414.7949001429524 + ], + "drain_ops_per_second": 388.28743041850294, + "p50_us": 504.45799999999997, + "p95_us": 9695.5625, + "p99_us": 12084.6455, + "rss_mib": 47.765625, + "metadata_mib": 23.568359375, + "checkpoint_ms": 18.379021, + "view_ms": 87.86924350000001, + "apply_ms": 1623.834456, + "cas_retries": 0, + "orphan_backlog": 0.0, + "live_slots": 15.0, + "per_operation": { + "list": { + "p50_us": 87.875, + "p95_us": 123.021, + "p99_us": 190.3955, + "mean_us": 98.32418828451884 + }, + "publish": { + "p50_us": 6945.2295, + "p95_us": 10666.3125, + "p99_us": 12209.729, + "mean_us": 7308.846318965517 + }, + "read": { + "p50_us": 522.812, + "p95_us": 1633.333, + "p99_us": 3011.7084999999997, + "mean_us": 976.4323411669368 + }, + "remove_recreate": { + "p50_us": 9783.8745, + "p95_us": 13415.271, + "p99_us": 18822.5835, + "mean_us": 9959.725575471697 + }, + "rename_roundtrip": { + "p50_us": 5539.3125, + "p95_us": 7398.6455000000005, + "p99_us": 9675.021, + "mean_us": 5476.734773333334 + }, + "snapshot": { + "p50_us": 164.563, + "p95_us": 285.4375, + "p99_us": 372.0625, + "mean_us": 178.79162 + }, + "stat": { + "p50_us": 43.5, + "p95_us": 67.1875, + "p99_us": 101.25, + "mean_us": 46.93939094650206 + } + } + }, + "patched": { + "runs": 4, + "ops_per_second": 402.1760099344121, + "ops_range": [ + 385.7715284397552, + 427.37990195238336 + ], + "drain_ops_per_second": 400.68115175306195, + "p50_us": 473.0, + "p95_us": 9963.7915, + "p99_us": 12058.4375, + "rss_mib": 45.6953125, + "metadata_mib": 21.6015625, + "checkpoint_ms": 15.8527705, + "view_ms": 74.37643, + "apply_ms": 1662.0523685, + "cas_retries": 0, + "orphan_backlog": 0.0, + "live_slots": 15.0, + "per_operation": { + "list": { + "p50_us": 82.9375, + "p95_us": 110.958, + "p99_us": 150.167, + "mean_us": 85.07865271966527 + }, + "publish": { + "p50_us": 7149.3955, + "p95_us": 10804.9375, + "p99_us": 12817.979, + "mean_us": 7534.430364942529 + }, + "read": { + "p50_us": 510.229, + "p95_us": 1563.771, + "p99_us": 2794.8955, + "mean_us": 578.238393841167 + }, + "remove_recreate": { + "p50_us": 9966.9585, + "p95_us": 13752.896, + "p99_us": 15878.271, + "mean_us": 10113.083396226415 + }, + "rename_roundtrip": { + "p50_us": 5880.041499999999, + "p95_us": 7336.125, + "p99_us": 7976.2295, + "mean_us": 5711.810806666666 + }, + "snapshot": { + "p50_us": 154.917, + "p95_us": 205.10399999999998, + "p99_us": 253.75, + "mean_us": 164.44666 + }, + "stat": { + "p50_us": 36.125, + "p95_us": 57.167, + "p99_us": 81.8125, + "mean_us": 39.97017695473251 + } + } + }, + "sqlite": { + "runs": 4, + "ops_per_second": 350.93311509551035, + "ops_range": [ + 299.2719181842948, + 369.51271809369547 + ], + "drain_ops_per_second": 350.27400837503626, + "p50_us": 495.4375, + "p95_us": 10791.1875, + "p99_us": 13286.313, + "rss_mib": 9.9765625, + "metadata_mib": 2.46875, + "checkpoint_ms": 8.587396000000002, + "view_ms": 72.9912485, + "apply_ms": 1812.8227305, + "cas_retries": 0, + "orphan_backlog": 0.0, + "live_slots": 0.0, + "per_operation": { + "list": { + "p50_us": 148.042, + "p95_us": 291.08299999999997, + "p99_us": 408.7505, + "mean_us": 157.0228430962343 + }, + "publish": { + "p50_us": 8354.207999999999, + "p95_us": 11956.3335, + "p99_us": 15257.771, + "mean_us": 8492.423176724138 + }, + "read": { + "p50_us": 514.396, + "p95_us": 1813.229, + "p99_us": 4414.166499999999, + "mean_us": 734.5737957860615 + }, + "remove_recreate": { + "p50_us": 10503.9585, + "p95_us": 15616.292, + "p99_us": 21324.646, + "mean_us": 10953.977990566038 + }, + "rename_roundtrip": { + "p50_us": 5997.291499999999, + "p95_us": 8357.3755, + "p99_us": 9617.979500000001, + "mean_us": 6014.497813333333 + }, + "snapshot": { + "p50_us": 190.167, + "p95_us": 355.25, + "p99_us": 396.52099999999996, + "mean_us": 205.57332 + }, + "stat": { + "p50_us": 16.686999999999998, + "p95_us": 63.9795, + "p99_us": 102.083, + "mean_us": 24.019380658436212 + } + } + } + }, + "metadata_async_1": { + "baseline": { + "runs": 4, + "ops_per_second": 19673.61296130564, + "ops_range": [ + 17621.78904846194, + 20170.89650870727 + ], + "drain_ops_per_second": 18184.721912812303, + "p50_us": 39.9795, + "p95_us": 79.916, + "p99_us": 160.625, + "rss_mib": 62.5546875, + "metadata_mib": 30.185546875, + "checkpoint_ms": 49.64247949999999, + "view_ms": 501.9311565, + "apply_ms": 53.4701405, + "cas_retries": 0, + "orphan_backlog": 0.0, + "live_slots": 31.0, + "per_operation": { + "list": { + "p50_us": 73.917, + "p95_us": 83.9375, + "p99_us": 101.0625, + "mean_us": 75.28613449972663 + }, + "publish": { + "p50_us": 37.708, + "p95_us": 45.4995, + "p99_us": 156.5205, + "mean_us": 41.30946132037262 + }, + "read": { + "p50_us": 38.5415, + "p95_us": 45.104, + "p99_us": 53.0835, + "mean_us": 38.76555762355415 + }, + "remove_recreate": { + "p50_us": 72.4165, + "p95_us": 93.3955, + "p99_us": 363.75, + "mean_us": 80.5497293956044 + }, + "rename_roundtrip": { + "p50_us": 72.4795, + "p95_us": 85.4165, + "p99_us": 346.9795, + "mean_us": 79.73778764805414 + }, + "snapshot": { + "p50_us": 154.1045, + "p95_us": 200.60399999999998, + "p99_us": 239.16649999999998, + "mean_us": 159.21302643171805 + }, + "stat": { + "p50_us": 38.6255, + "p95_us": 45.229, + "p99_us": 57.1255, + "mean_us": 38.977190084985835 + } + } + }, + "patched": { + "runs": 4, + "ops_per_second": 23188.22978586259, + "ops_range": [ + 20816.140141194144, + 23586.427698861313 + ], + "drain_ops_per_second": 21254.70355615512, + "p50_us": 32.1665, + "p95_us": 73.396, + "p99_us": 143.417, + "rss_mib": 63.921875, + "metadata_mib": 29.21875, + "checkpoint_ms": 48.2109165, + "view_ms": 411.39797350000003, + "apply_ms": 52.1596585, + "cas_retries": 0, + "orphan_backlog": 0.0, + "live_slots": 31.0, + "per_operation": { + "list": { + "p50_us": 68.417, + "p95_us": 78.5205, + "p99_us": 95.5415, + "mean_us": 69.65812465828321 + }, + "publish": { + "p50_us": 31.917, + "p95_us": 39.125, + "p99_us": 144.85399999999998, + "mean_us": 35.77607391656541 + }, + "read": { + "p50_us": 29.8335, + "p95_us": 36.1665, + "p99_us": 46.542, + "mean_us": 30.605220925341747 + }, + "remove_recreate": { + "p50_us": 60.875, + "p95_us": 78.4585, + "p99_us": 349.0, + "mean_us": 68.86304532967034 + }, + "rename_roundtrip": { + "p50_us": 61.396, + "p95_us": 75.3125, + "p99_us": 347.541, + "mean_us": 67.82178934010152 + }, + "snapshot": { + "p50_us": 134.687, + "p95_us": 202.0, + "p99_us": 261.7295, + "mean_us": 143.41885682819384 + }, + "stat": { + "p50_us": 29.979, + "p95_us": 36.542, + "p99_us": 45.9585, + "mean_us": 30.796894900849857 + } + } + }, + "sqlite": { + "runs": 4, + "ops_per_second": 14951.966051289974, + "ops_range": [ + 13267.961600038021, + 15546.441201052028 + ], + "drain_ops_per_second": 14480.879165011938, + "p50_us": 5.833, + "p95_us": 249.72899999999998, + "p99_us": 369.70849999999996, + "rss_mib": 13.625, + "metadata_mib": 4.97265625, + "checkpoint_ms": 26.544687, + "view_ms": 120.26008350000001, + "apply_ms": 620.118076, + "cas_retries": 0, + "orphan_backlog": 0.0, + "live_slots": 0.0, + "per_operation": { + "list": { + "p50_us": 47.7705, + "p95_us": 69.5, + "p99_us": 121.354, + "mean_us": 52.08659814106069 + }, + "publish": { + "p50_us": 145.187, + "p95_us": 257.91650000000004, + "p99_us": 483.938, + "mean_us": 159.6736089509923 + }, + "read": { + "p50_us": 4.1455, + "p95_us": 6.375, + "p99_us": 12.1035, + "mean_us": 4.644904521556256 + }, + "remove_recreate": { + "p50_us": 260.1045, + "p95_us": 404.27099999999996, + "p99_us": 882.4995, + "mean_us": 280.9316785714285 + }, + "rename_roundtrip": { + "p50_us": 239.1045, + "p95_us": 446.14599999999996, + "p99_us": 1060.604, + "mean_us": 269.2974780033841 + }, + "snapshot": { + "p50_us": 72.3335, + "p95_us": 101.458, + "p99_us": 164.375, + "mean_us": 77.67555947136563 + }, + "stat": { + "p50_us": 4.1255, + "p95_us": 6.354, + "p99_us": 12.2705, + "mean_us": 4.704116997167139 + } + } + } + }, + "metadata_async_4": { + "baseline": { + "runs": 4, + "ops_per_second": 226.15146465988403, + "ops_range": [ + 215.29827215175047, + 229.90097877467952 + ], + "drain_ops_per_second": 182.69514865802967, + "p50_us": 8467.833, + "p95_us": 61299.646, + "p99_us": 140883.7505, + "rss_mib": 6800.265625, + "metadata_mib": 8160.14453125, + "checkpoint_ms": 12619.732021, + "view_ms": 183643.62052350002, + "apply_ms": 21180.583109, + "cas_retries": 0, + "orphan_backlog": 7849.5, + "live_slots": 7880.5, + "per_operation": { + "list": { + "p50_us": 23247.396, + "p95_us": 142802.4165, + "p99_us": 249251.5, + "mean_us": 40671.464463394346 + }, + "publish": { + "p50_us": 8906.7915, + "p95_us": 42545.375, + "p99_us": 72980.58350000001, + "mean_us": 13328.856775712515 + }, + "read": { + "p50_us": 6306.146000000001, + "p95_us": 29155.875, + "p99_us": 50098.0, + "mean_us": 9549.616034990599 + }, + "remove_recreate": { + "p50_us": 16484.8545, + "p95_us": 73919.6245, + "p99_us": 120560.25, + "mean_us": 24126.150233516484 + }, + "rename_roundtrip": { + "p50_us": 16890.7705, + "p95_us": 71977.0, + "p99_us": 116244.6245, + "mean_us": 24468.00305139373 + }, + "snapshot": { + "p50_us": 29638.7085, + "p95_us": 159314.6875, + "p99_us": 250097.0625, + "mean_us": 47571.7913018018 + }, + "stat": { + "p50_us": 6696.937, + "p95_us": 29649.604, + "p99_us": 50949.3125, + "mean_us": 10012.28552323674 + } + } + }, + "patched": { + "runs": 4, + "ops_per_second": 5825.9820620181135, + "ops_range": [ + 5586.187234809988, + 12545.901441298982 + ], + "drain_ops_per_second": 5643.675611250126, + "p50_us": 369.542, + "p95_us": 2611.375, + "p99_us": 5082.875, + "rss_mib": 107.0078125, + "metadata_mib": 40.658203125, + "checkpoint_ms": 70.147479, + "view_ms": 7246.963444, + "apply_ms": 831.5918855, + "cas_retries": 0, + "orphan_backlog": 0.0, + "live_slots": 31.0, + "per_operation": { + "list": { + "p50_us": 1476.667, + "p95_us": 5327.625, + "p99_us": 7810.042, + "mean_us": 1968.8466292290625 + }, + "publish": { + "p50_us": 369.5, + "p95_us": 1184.5835, + "p99_us": 1605.021, + "mean_us": 476.8574310202396 + }, + "read": { + "p50_us": 277.95849999999996, + "p95_us": 765.229, + "p99_us": 1196.8125, + "mean_us": 323.6795992270733 + }, + "remove_recreate": { + "p50_us": 769.1455, + "p95_us": 1938.875, + "p99_us": 3002.2290000000003, + "mean_us": 865.2965714285715 + }, + "rename_roundtrip": { + "p50_us": 774.0415, + "p95_us": 1968.334, + "p99_us": 2619.5205, + "mean_us": 869.4341716027875 + }, + "snapshot": { + "p50_us": 1632.6455, + "p95_us": 4804.353999999999, + "p99_us": 6512.625, + "mean_us": 1997.1111396396395 + }, + "stat": { + "p50_us": 290.75, + "p95_us": 743.021, + "p99_us": 1169.562, + "mean_us": 326.1560757244396 + } + } + }, + "sqlite": { + "runs": 4, + "ops_per_second": 13725.455838029797, + "ops_range": [ + 11594.73829539383, + 15566.63608025908 + ], + "drain_ops_per_second": 13199.664451408604, + "p50_us": 6.604, + "p95_us": 249.0, + "p99_us": 428.08349999999996, + "rss_mib": 14.0546875, + "metadata_mib": 5.0390625, + "checkpoint_ms": 30.0071455, + "view_ms": 127.409395, + "apply_ms": 2416.8967375, + "cas_retries": 0, + "orphan_backlog": 0.0, + "live_slots": 0.0, + "per_operation": { + "list": { + "p50_us": 50.2495, + "p95_us": 66.5625, + "p99_us": 116.2915, + "mean_us": 53.70387742651137 + }, + "publish": { + "p50_us": 145.16649999999998, + "p95_us": 283.95849999999996, + "p99_us": 2121.6035, + "mean_us": 698.5480675340768 + }, + "read": { + "p50_us": 4.354, + "p95_us": 7.5625, + "p99_us": 14.1045, + "mean_us": 5.021038332985167 + }, + "remove_recreate": { + "p50_us": 251.062, + "p95_us": 467.35400000000004, + "p99_us": 1742.4375, + "mean_us": 414.3677458791209 + }, + "rename_roundtrip": { + "p50_us": 229.562, + "p95_us": 448.08349999999996, + "p99_us": 4104.2705, + "mean_us": 1061.4254651567944 + }, + "snapshot": { + "p50_us": 82.0415, + "p95_us": 116.812, + "p99_us": 193.1245, + "mean_us": 87.16263963963964 + }, + "stat": { + "p50_us": 4.417, + "p95_us": 7.542, + "p99_us": 14.8335, + "mean_us": 5.18594095133953 + } + } + } + }, + "metadata_async_4_background_normal": { + "baseline": { + "runs": 4, + "ops_per_second": 1971.3102172399053, + "ops_range": [ + 1945.1409079549446, + 1994.0063081625842 + ], + "drain_ops_per_second": 1829.9711002271954, + "p50_us": 1099.959, + "p95_us": 7300.416499999999, + "p99_us": 14519.708, + "rss_mib": 423.375, + "metadata_mib": 1430.1875, + "checkpoint_ms": 159.695292, + "view_ms": 7221.4041225, + "apply_ms": 810.6684, + "cas_retries": 0, + "orphan_backlog": 897.5, + "live_slots": 913.5, + "per_operation": { + "list": { + "p50_us": 4127.0005, + "p95_us": 14832.645499999999, + "p99_us": 22257.7085, + "mean_us": 5384.761172470979 + }, + "publish": { + "p50_us": 1114.375, + "p95_us": 4008.7915000000003, + "p99_us": 6165.228999999999, + "mean_us": 1480.5025705209657 + }, + "read": { + "p50_us": 835.458, + "p95_us": 2533.7915000000003, + "p99_us": 4381.896000000001, + "mean_us": 1039.2355492957747 + }, + "remove_recreate": { + "p50_us": 2301.125, + "p95_us": 6817.6455, + "p99_us": 9336.021, + "mean_us": 2806.3451880000002 + }, + "rename_roundtrip": { + "p50_us": 2316.2295, + "p95_us": 6588.416499999999, + "p99_us": 8719.832999999999, + "mean_us": 2910.0769644808743 + }, + "snapshot": { + "p50_us": 4633.25, + "p95_us": 15196.5005, + "p99_us": 22107.646, + "mean_us": 5965.600612676057 + }, + "stat": { + "p50_us": 860.2915, + "p95_us": 2794.1040000000003, + "p99_us": 4666.875, + "mean_us": 1068.619912207358 + } + } + }, + "patched": { + "runs": 4, + "ops_per_second": 22098.379022323286, + "ops_range": [ + 8585.016653708944, + 23579.385081233457 + ], + "drain_ops_per_second": 18266.28849201852, + "p50_us": 76.375, + "p95_us": 613.8335, + "p99_us": 1906.9795, + "rss_mib": 79.8828125, + "metadata_mib": 26.19140625, + "checkpoint_ms": 38.023521, + "view_ms": 611.7685815, + "apply_ms": 88.64454599999999, + "cas_retries": 0, + "orphan_backlog": 0.0, + "live_slots": 16.0, + "per_operation": { + "list": { + "p50_us": 369.5625, + "p95_us": 1441.0835, + "p99_us": 3750.812, + "mean_us": 524.6831940298507 + }, + "publish": { + "p50_us": 97.812, + "p95_us": 268.396, + "p99_us": 990.229, + "mean_us": 139.51803049555275 + }, + "read": { + "p50_us": 51.729, + "p95_us": 163.0625, + "p99_us": 313.60400000000004, + "mean_us": 73.53368891610533 + }, + "remove_recreate": { + "p50_us": 184.14600000000002, + "p95_us": 439.8125, + "p99_us": 2120.7495, + "mean_us": 241.036524 + }, + "rename_roundtrip": { + "p50_us": 191.16649999999998, + "p95_us": 433.14549999999997, + "p99_us": 2515.313, + "mean_us": 266.1419644808743 + }, + "snapshot": { + "p50_us": 496.792, + "p95_us": 2013.4589999999998, + "p99_us": 5116.353999999999, + "mean_us": 715.0956408450704 + }, + "stat": { + "p50_us": 51.7915, + "p95_us": 163.60399999999998, + "p99_us": 634.104, + "mean_us": 79.4990016722408 + } + } + }, + "sqlite": { + "runs": 4, + "ops_per_second": 1876.5968759864145, + "ops_range": [ + 1610.9883096116146, + 2431.1173899498904 + ], + "drain_ops_per_second": 1869.3490498999852, + "p50_us": 47.8745, + "p95_us": 7430.958, + "p99_us": 12031.582999999999, + "rss_mib": 11.0078125, + "metadata_mib": 2.896484375, + "checkpoint_ms": 8.2763125, + "view_ms": 140.77099199999998, + "apply_ms": 7980.1026505, + "cas_retries": 0, + "orphan_backlog": 0.0, + "live_slots": 0.0, + "per_operation": { + "list": { + "p50_us": 95.75, + "p95_us": 303.58349999999996, + "p99_us": 399.75, + "mean_us": 127.89037810945274 + }, + "publish": { + "p50_us": 2588.562, + "p95_us": 9013.3335, + "p99_us": 44452.333, + "mean_us": 5841.448796696315 + }, + "read": { + "p50_us": 10.604, + "p95_us": 90.0415, + "p99_us": 181.3125, + "mean_us": 22.200568279240663 + }, + "remove_recreate": { + "p50_us": 5957.291499999999, + "p95_us": 13038.271, + "p99_us": 116745.29149999999, + "mean_us": 8819.692824 + }, + "rename_roundtrip": { + "p50_us": 5187.521000000001, + "p95_us": 14373.375, + "p99_us": 153801.5835, + "mean_us": 8927.784587431695 + }, + "snapshot": { + "p50_us": 173.3125, + "p95_us": 606.2085, + "p99_us": 875.0415, + "mean_us": 245.95185211267605 + }, + "stat": { + "p50_us": 9.9375, + "p95_us": 78.7495, + "p99_us": 168.79149999999998, + "mean_us": 21.225290133779264 + } + } + } + }, + "metadata_async_4_short": { + "baseline": { + "runs": 4, + "ops_per_second": 1003.3620631233744, + "ops_range": [ + 1002.2620427891975, + 1012.3926777613284 + ], + "drain_ops_per_second": 723.5761212875759, + "p50_us": 2040.8960000000002, + "p95_us": 14277.167000000001, + "p99_us": 30984.5415, + "rss_mib": 1388.1953125, + "metadata_mib": 1325.404296875, + "checkpoint_ms": 1561.2780834999999, + "view_ms": 14145.347947999999, + "apply_ms": 1417.7584535, + "cas_retries": 0, + "orphan_backlog": 1068.5, + "live_slots": 1084.0, + "per_operation": { + "list": { + "p50_us": 7511.7919999999995, + "p95_us": 32340.5625, + "p99_us": 46980.979, + "mean_us": 10592.052304311776 + }, + "publish": { + "p50_us": 2057.7290000000003, + "p95_us": 8124.9165, + "p99_us": 11434.146, + "mean_us": 2731.7911905972046 + }, + "read": { + "p50_us": 1682.25, + "p95_us": 5181.4375, + "p99_us": 8384.875, + "mean_us": 2032.118900489896 + }, + "remove_recreate": { + "p50_us": 4277.896000000001, + "p95_us": 11779.375, + "p99_us": 16968.0415, + "mean_us": 5101.829496 + }, + "rename_roundtrip": { + "p50_us": 4308.603999999999, + "p95_us": 13211.229, + "p99_us": 19579.1875, + "mean_us": 5491.266592896174 + }, + "snapshot": { + "p50_us": 9124.313, + "p95_us": 30795.8335, + "p99_us": 51941.625, + "mean_us": 11171.850626760563 + }, + "stat": { + "p50_us": 1766.2505, + "p95_us": 5261.562, + "p99_us": 8824.1875, + "mean_us": 2131.886227424749 + } + } + }, + "patched": { + "runs": 4, + "ops_per_second": 7101.61374328817, + "ops_range": [ + 6424.604085258936, + 21126.016689553184 + ], + "drain_ops_per_second": 6639.926039911363, + "p50_us": 265.3755, + "p95_us": 2194.875, + "p99_us": 4834.1035, + "rss_mib": 87.5625, + "metadata_mib": 21.1875, + "checkpoint_ms": 40.3168125, + "view_ms": 2004.6453065, + "apply_ms": 227.969338, + "cas_retries": 0, + "orphan_backlog": 0.0, + "live_slots": 16.0, + "per_operation": { + "list": { + "p50_us": 981.229, + "p95_us": 4623.2295, + "p99_us": 6926.603999999999, + "mean_us": 1569.2530240464343 + }, + "publish": { + "p50_us": 244.6875, + "p95_us": 1176.1665, + "p99_us": 2291.5209999999997, + "mean_us": 402.0983996188056 + }, + "read": { + "p50_us": 187.4375, + "p95_us": 811.75, + "p99_us": 1728.75, + "mean_us": 280.960833435395 + }, + "remove_recreate": { + "p50_us": 452.0415, + "p95_us": 2025.333, + "p99_us": 3346.2285, + "mean_us": 725.6798120000001 + }, + "rename_roundtrip": { + "p50_us": 560.104, + "p95_us": 1719.062, + "p99_us": 4273.7505, + "mean_us": 734.9815628415301 + }, + "snapshot": { + "p50_us": 1351.875, + "p95_us": 4684.9785, + "p99_us": 6210.896000000001, + "mean_us": 1721.7740211267605 + }, + "stat": { + "p50_us": 182.0625, + "p95_us": 806.583, + "p99_us": 1578.4585, + "mean_us": 270.7866806020067 + } + } + }, + "sqlite": { + "runs": 4, + "ops_per_second": 15133.104006656009, + "ops_range": [ + 12779.76025406189, + 15688.222740978583 + ], + "drain_ops_per_second": 14451.161617246154, + "p50_us": 5.3125, + "p95_us": 238.35399999999998, + "p99_us": 345.3755, + "rss_mib": 10.8046875, + "metadata_mib": 2.57421875, + "checkpoint_ms": 12.7076875, + "view_ms": 37.88678, + "apply_ms": 688.654315, + "cas_retries": 0, + "orphan_backlog": 0.0, + "live_slots": 0.0, + "per_operation": { + "list": { + "p50_us": 48.1875, + "p95_us": 55.771, + "p99_us": 70.5415, + "mean_us": 49.47500995024876 + }, + "publish": { + "p50_us": 142.7295, + "p95_us": 244.875, + "p99_us": 1467.479, + "mean_us": 729.6650743329097 + }, + "read": { + "p50_us": 4.0005, + "p95_us": 6.229, + "p99_us": 8.7705, + "mean_us": 4.4118622167789345 + }, + "remove_recreate": { + "p50_us": 250.812, + "p95_us": 383.6875, + "p99_us": 854.229, + "mean_us": 402.02448 + }, + "rename_roundtrip": { + "p50_us": 231.60399999999998, + "p95_us": 341.333, + "p99_us": 773.8544999999999, + "mean_us": 505.1600901639344 + }, + "snapshot": { + "p50_us": 75.4585, + "p95_us": 103.646, + "p99_us": 145.52100000000002, + "mean_us": 78.86563380281689 + }, + "stat": { + "p50_us": 4.0005, + "p95_us": 6.1875, + "p99_us": 8.604, + "mean_us": 4.372726588628762 + } + } + } + }, + "metadata_sync_1": { + "baseline": { + "runs": 4, + "ops_per_second": 774.4463262276743, + "ops_range": [ + 756.5456447393102, + 812.3257985348521 + ], + "drain_ops_per_second": 769.3473479343147, + "p50_us": 216.54149999999998, + "p95_us": 5965.521, + "p99_us": 7215.8745, + "rss_mib": 49.1640625, + "metadata_mib": 20.625, + "checkpoint_ms": 37.437666500000006, + "view_ms": 700.3249225, + "apply_ms": 4412.173954, + "cas_retries": 0, + "orphan_backlog": 0.0, + "live_slots": 14.0, + "per_operation": { + "list": { + "p50_us": 249.27100000000002, + "p95_us": 416.60400000000004, + "p99_us": 569.875, + "mean_us": 252.12992118644067 + }, + "publish": { + "p50_us": 3072.3125, + "p95_us": 3959.1665000000003, + "p99_us": 5292.333, + "mean_us": 3093.045380562061 + }, + "read": { + "p50_us": 153.16649999999998, + "p95_us": 303.562, + "p99_us": 479.1255, + "mean_us": 166.56086325878593 + }, + "remove_recreate": { + "p50_us": 6062.228999999999, + "p95_us": 7943.666499999999, + "p99_us": 9404.207999999999, + "mean_us": 6217.4690875 + }, + "rename_roundtrip": { + "p50_us": 6107.188, + "p95_us": 7750.291, + "p99_us": 8969.229, + "mean_us": 6138.700814070352 + }, + "snapshot": { + "p50_us": 589.1665, + "p95_us": 968.771, + "p99_us": 1331.9375, + "mean_us": 585.5318428571429 + }, + "stat": { + "p50_us": 149.208, + "p95_us": 299.54150000000004, + "p99_us": 446.16650000000004, + "mean_us": 160.36507558139533 + } + } + }, + "patched": { + "runs": 4, + "ops_per_second": 771.8085173940791, + "ops_range": [ + 753.998394627711, + 789.8569597539976 + ], + "drain_ops_per_second": 766.4698762221396, + "p50_us": 201.5625, + "p95_us": 5945.083500000001, + "p99_us": 7350.7705000000005, + "rss_mib": 49.46875, + "metadata_mib": 20.576171875, + "checkpoint_ms": 40.0964585, + "view_ms": 604.2076715000001, + "apply_ms": 4467.313945, + "cas_retries": 0, + "orphan_backlog": 0.0, + "live_slots": 14.0, + "per_operation": { + "list": { + "p50_us": 242.1045, + "p95_us": 456.875, + "p99_us": 631.5, + "mean_us": 255.66694999999999 + }, + "publish": { + "p50_us": 3112.0834999999997, + "p95_us": 3987.9584999999997, + "p99_us": 5566.271000000001, + "mean_us": 3158.5567980093674 + }, + "read": { + "p50_us": 129.0625, + "p95_us": 278.39599999999996, + "p99_us": 398.54200000000003, + "mean_us": 140.38603642172524 + }, + "remove_recreate": { + "p50_us": 6037.146000000001, + "p95_us": 7830.353999999999, + "p99_us": 8266.729, + "mean_us": 6032.742737500001 + }, + "rename_roundtrip": { + "p50_us": 6048.312, + "p95_us": 7826.0, + "p99_us": 10949.354, + "mean_us": 6105.771361809046 + }, + "snapshot": { + "p50_us": 582.0, + "p95_us": 921.25, + "p99_us": 1242.833, + "mean_us": 573.7036 + }, + "stat": { + "p50_us": 131.583, + "p95_us": 269.97950000000003, + "p99_us": 432.29150000000004, + "mean_us": 142.552719269103 + } + } + }, + "sqlite": { + "runs": 4, + "ops_per_second": 793.4489489923458, + "ops_range": [ + 759.7116252273996, + 848.2801099881094 + ], + "drain_ops_per_second": 791.3101004092042, + "p50_us": 55.104, + "p95_us": 6011.104, + "p99_us": 7945.0205, + "rss_mib": 10.7421875, + "metadata_mib": 3.53515625, + "checkpoint_ms": 13.4875, + "view_ms": 175.7028495, + "apply_ms": 4768.6304585, + "cas_retries": 0, + "orphan_backlog": 0.0, + "live_slots": 0.0, + "per_operation": { + "list": { + "p50_us": 177.83350000000002, + "p95_us": 357.08299999999997, + "p99_us": 493.1245, + "mean_us": 191.88785932203393 + }, + "publish": { + "p50_us": 3333.5625, + "p95_us": 4351.603999999999, + "p99_us": 5036.5205, + "mean_us": 3270.204281030445 + }, + "read": { + "p50_us": 15.896, + "p95_us": 68.563, + "p99_us": 110.1255, + "mean_us": 24.144567412140574 + }, + "remove_recreate": { + "p50_us": 6373.9375, + "p95_us": 8702.208, + "p99_us": 9940.479, + "mean_us": 6524.158016666666 + }, + "rename_roundtrip": { + "p50_us": 6225.604, + "p95_us": 8179.2495, + "p99_us": 9899.4375, + "mean_us": 6392.253653266332 + }, + "snapshot": { + "p50_us": 269.562, + "p95_us": 597.979, + "p99_us": 711.8955, + "mean_us": 286.95001428571436 + }, + "stat": { + "p50_us": 15.896, + "p95_us": 64.479, + "p99_us": 100.729, + "mean_us": 23.950260797342192 + } + } + } + } +} diff --git a/benches/agent_workspace/results/optimization-2026-10-15/sync-followup-summary.json b/benches/agent_workspace/results/optimization-2026-10-15/sync-followup-summary.json new file mode 100644 index 0000000..64abad1 --- /dev/null +++ b/benches/agent_workspace/results/optimization-2026-10-15/sync-followup-summary.json @@ -0,0 +1,196 @@ +{ + "attached_metadata_sync_1": { + "baseline": { + "runs": 4, + "ops_per_second": 796.9146452650455, + "ops_range": [ + 776.7446345939584, + 817.5298984178204 + ], + "drain_ops_per_second": 790.1850592289754, + "p50_us": 193.72899999999998, + "p95_us": 5861.3125, + "p99_us": 7219.5625, + "rss_mib": 49.2578125, + "metadata_mib": 21.119140625, + "checkpoint_ms": 46.533311999999995, + "view_ms": 601.1776795000001, + "apply_ms": 4364.226430500001, + "cas_retries": 0, + "orphan_backlog": 0.0, + "live_slots": 14.0, + "per_operation": { + "list": { + "p50_us": 207.83350000000002, + "p95_us": 395.60400000000004, + "p99_us": 625.4585, + "mean_us": 225.47948559322032 + }, + "publish": { + "p50_us": 3071.0625, + "p95_us": 4011.375, + "p99_us": 4991.125, + "mean_us": 3043.0363149882905 + }, + "read": { + "p50_us": 122.687, + "p95_us": 296.5205, + "p99_us": 453.4375, + "mean_us": 143.10020479233228 + }, + "remove_recreate": { + "p50_us": 5958.896, + "p95_us": 7896.813, + "p99_us": 9201.1665, + "mean_us": 5964.941845833333 + }, + "rename_roundtrip": { + "p50_us": 6000.396, + "p95_us": 7969.521000000001, + "p99_us": 11727.6455, + "mean_us": 6132.000934673366 + }, + "snapshot": { + "p50_us": 550.4795, + "p95_us": 950.1455000000001, + "p99_us": 1245.4995, + "mean_us": 548.23725 + }, + "stat": { + "p50_us": 127.0005, + "p95_us": 296.458, + "p99_us": 458.79200000000003, + "mean_us": 141.94677657807307 + } + } + }, + "patched": { + "runs": 4, + "ops_per_second": 795.0223172238667, + "ops_range": [ + 758.1886626438123, + 805.7816105302051 + ], + "drain_ops_per_second": 787.5770623651442, + "p50_us": 188.08350000000002, + "p95_us": 5867.2705, + "p99_us": 7464.625, + "rss_mib": 48.984375, + "metadata_mib": 21.64453125, + "checkpoint_ms": 50.605125, + "view_ms": 559.6837304999999, + "apply_ms": 4393.625425, + "cas_retries": 0, + "orphan_backlog": 0.0, + "live_slots": 14.0, + "per_operation": { + "list": { + "p50_us": 210.8755, + "p95_us": 399.2915, + "p99_us": 526.542, + "mean_us": 222.79783050847456 + }, + "publish": { + "p50_us": 3084.167, + "p95_us": 4018.417, + "p99_us": 5704.9375, + "mean_us": 3095.6283296252927 + }, + "read": { + "p50_us": 117.479, + "p95_us": 269.08349999999996, + "p99_us": 396.229, + "mean_us": 131.76933674121403 + }, + "remove_recreate": { + "p50_us": 5994.896000000001, + "p95_us": 7952.9585, + "p99_us": 9566.896, + "mean_us": 6065.9691041666665 + }, + "rename_roundtrip": { + "p50_us": 6007.271000000001, + "p95_us": 7850.896000000001, + "p99_us": 10321.4995, + "mean_us": 6060.16425879397 + }, + "snapshot": { + "p50_us": 515.7505, + "p95_us": 907.375, + "p99_us": 1127.8335, + "mean_us": 514.0892571428572 + }, + "stat": { + "p50_us": 116.1455, + "p95_us": 266.021, + "p99_us": 380.04200000000003, + "mean_us": 130.38026910299004 + } + } + }, + "sqlite": { + "runs": 4, + "ops_per_second": 798.9155624366012, + "ops_range": [ + 786.1176342497304, + 817.694945774022 + ], + "drain_ops_per_second": 790.6047253720205, + "p50_us": 53.4165, + "p95_us": 5959.146000000001, + "p99_us": 8008.5, + "rss_mib": 18.2890625, + "metadata_mib": 8.79296875, + "checkpoint_ms": 50.1754585, + "view_ms": 159.483521, + "apply_ms": 4748.8799045, + "cas_retries": 0, + "orphan_backlog": 0.0, + "live_slots": 0.0, + "per_operation": { + "list": { + "p50_us": 162.625, + "p95_us": 329.1455, + "p99_us": 517.5835, + "mean_us": 176.53460593220336 + }, + "publish": { + "p50_us": 3419.0625, + "p95_us": 4599.3745, + "p99_us": 5950.0625, + "mean_us": 3251.4388307962527 + }, + "read": { + "p50_us": 14.2915, + "p95_us": 63.4375, + "p99_us": 92.2915, + "mean_us": 21.77513354632588 + }, + "remove_recreate": { + "p50_us": 6587.146, + "p95_us": 8636.354, + "p99_us": 11258.021, + "mean_us": 6537.1691083333335 + }, + "rename_roundtrip": { + "p50_us": 6101.9375, + "p95_us": 8463.3545, + "p99_us": 10511.5, + "mean_us": 6313.189708542714 + }, + "snapshot": { + "p50_us": 268.72900000000004, + "p95_us": 497.83299999999997, + "p99_us": 769.6875, + "mean_us": 278.73720000000003 + }, + "stat": { + "p50_us": 14.8535, + "p95_us": 64.354, + "p99_us": 91.25, + "mean_us": 22.006814784053155 + } + } + } + } +} diff --git a/benches/agent_workspace/results/optimization-evidence-2026-10-15.tar.gz.sha256 b/benches/agent_workspace/results/optimization-evidence-2026-10-15.tar.gz.sha256 new file mode 100644 index 0000000..c1b6358 --- /dev/null +++ b/benches/agent_workspace/results/optimization-evidence-2026-10-15.tar.gz.sha256 @@ -0,0 +1 @@ +9fef9fed20e86412dee9a741742698868a0114c3b334f69e446ff947a4d9a62e optimization-evidence-2026-10-15.tar.gz diff --git a/benches/agent_workspace/results/summary.json b/benches/agent_workspace/results/summary.json new file mode 100644 index 0000000..b540de1 --- /dev/null +++ b/benches/agent_workspace/results/summary.json @@ -0,0 +1,2335 @@ +{ + "date": "2026-10-15", + "holt_commit": "f080daf2bc9d8019bb134141dc5ee31fd36f2c03", + "holt_version": "0.9.2", + "sqlite_version": "3.51.3", + "formal_processes": 128, + "groups": { + "comparison": { + "attached_content_sync_1": { + "holt": { + "runs": 4, + "ops_per_second": 428.7319279175029, + "ops_min": 423.23580907565804, + "ops_max": 434.51580843247797, + "p50_us": 480.0, + "p95_us": 9093.896, + "p99_us": 11566.457999999999, + "checkpoint_ms": 20.516854000000002, + "reopen_ms": 22.5384165, + "drain_ops_per_second": 426.512346610313, + "metadata_allocated_mib": 26.275390625, + "metadata_logical_mib": 28.013936042785645, + "view_ms": 75.1718305, + "apply_ms": 1582.678409, + "object_ms": 2054.3198294999997, + "commits": 604.0, + "cas_retries": 0, + "max_rss_mib": 47.171875, + "per_operation": { + "list": { + "mean_us": 79.45727615062762, + "p50_us": 77.7295, + "p95_us": 98.438, + "p99_us": 115.7705, + "count": 239.0 + }, + "publish": { + "mean_us": 6853.186363505747, + "p50_us": 6540.3535, + "p95_us": 10095.082999999999, + "p99_us": 12147.646, + "count": 348.0 + }, + "read": { + "mean_us": 672.6087957860616, + "p50_us": 512.0835, + "p95_us": 1419.021, + "p99_us": 2346.0209999999997, + "count": 617.0 + }, + "remove_recreate": { + "mean_us": 9548.342754716981, + "p50_us": 9331.792, + "p95_us": 13548.396, + "p99_us": 15266.4995, + "count": 53.0 + }, + "rename_roundtrip": { + "mean_us": 5411.917773333334, + "p50_us": 5708.9995, + "p95_us": 7032.1669999999995, + "p99_us": 7952.3955000000005, + "count": 75.0 + }, + "snapshot": { + "mean_us": 175.28166, + "p50_us": 170.60399999999998, + "p95_us": 211.54149999999998, + "p99_us": 244.95850000000002, + "count": 25.0 + }, + "stat": { + "mean_us": 42.98059053497942, + "p50_us": 41.021, + "p95_us": 57.021, + "p99_us": 67.167, + "count": 243.0 + } + } + }, + "sqlite": { + "runs": 4, + "ops_per_second": 315.28253659364725, + "ops_min": 297.2728766719408, + "ops_max": 366.85225436375885, + "p50_us": 831.375, + "p95_us": 11533.5415, + "p99_us": 14599.5, + "checkpoint_ms": 25.6078125, + "reopen_ms": 3.3715415, + "drain_ops_per_second": 313.7306025411923, + "metadata_allocated_mib": 4.53125, + "metadata_logical_mib": 4.4375, + "view_ms": 128.1126955, + "apply_ms": 2110.5757245, + "object_ms": 2752.7536945, + "commits": 604.0, + "cas_retries": 0, + "max_rss_mib": 13.1796875, + "per_operation": { + "list": { + "mean_us": 278.77486820083686, + "p50_us": 271.52099999999996, + "p95_us": 512.1875, + "p99_us": 647.4169999999999, + "count": 239.0 + }, + "publish": { + "mean_us": 9150.298005747127, + "p50_us": 8966.832999999999, + "p95_us": 12812.854, + "p99_us": 14830.3335, + "count": 348.0 + }, + "read": { + "mean_us": 952.953645056726, + "p50_us": 873.4585, + "p95_us": 2197.6040000000003, + "p99_us": 3699.3540000000003, + "count": 617.0 + }, + "remove_recreate": { + "mean_us": 12591.752009433963, + "p50_us": 12524.417000000001, + "p95_us": 16971.854, + "p99_us": 18183.792, + "count": 53.0 + }, + "rename_roundtrip": { + "mean_us": 6760.148613333333, + "p50_us": 6815.042, + "p95_us": 9112.104, + "p99_us": 9630.062, + "count": 75.0 + }, + "snapshot": { + "mean_us": 371.27418, + "p50_us": 351.60450000000003, + "p95_us": 576.1669999999999, + "p99_us": 713.938, + "count": 25.0 + }, + "stat": { + "mean_us": 44.94472427983539, + "p50_us": 39.979, + "p95_us": 83.8955, + "p99_us": 121.8545, + "count": 243.0 + } + } + }, + "holt_vs_sqlite_throughput_ratio": 1.3598340477388229 + }, + "attached_metadata_sync_1": { + "holt": { + "runs": 4, + "ops_per_second": 723.3349948880514, + "ops_min": 669.1202144005161, + "ops_max": 758.204088052692, + "p50_us": 243.85449999999997, + "p95_us": 5898.958500000001, + "p99_us": 7970.1669999999995, + "checkpoint_ms": 48.2050005, + "reopen_ms": 26.981167, + "drain_ops_per_second": 716.8512830465045, + "metadata_allocated_mib": 21.34375, + "metadata_logical_mib": 22.133214950561523, + "view_ms": 750.9582045, + "apply_ms": 4584.309546500001, + "object_ms": 0.0, + "commits": 1492.0, + "cas_retries": 0, + "max_rss_mib": 50.0, + "per_operation": { + "list": { + "mean_us": 291.0400177966102, + "p50_us": 278.47900000000004, + "p95_us": 534.396, + "p99_us": 641.687, + "count": 590.0 + }, + "publish": { + "mean_us": 3285.617837822014, + "p50_us": 3373.9790000000003, + "p95_us": 4326.8125, + "p99_us": 5319.938, + "count": 854.0 + }, + "read": { + "mean_us": 176.96464025559106, + "p50_us": 173.6455, + "p95_us": 316.8745, + "p99_us": 401.72950000000003, + "count": 1565.0 + }, + "remove_recreate": { + "mean_us": 6318.3255, + "p50_us": 6055.3125, + "p95_us": 8140.916499999999, + "p99_us": 8649.479, + "count": 120.0 + }, + "rename_roundtrip": { + "mean_us": 6424.522000000001, + "p50_us": 6108.146, + "p95_us": 8684.271, + "p99_us": 14366.1665, + "count": 199.0 + }, + "snapshot": { + "mean_us": 698.073242857143, + "p50_us": 695.2915, + "p95_us": 1072.8335, + "p99_us": 1174.292, + "count": 70.0 + }, + "stat": { + "mean_us": 179.75059468438536, + "p50_us": 176.27100000000002, + "p95_us": 310.14599999999996, + "p99_us": 388.91650000000004, + "count": 602.0 + } + } + }, + "sqlite": { + "runs": 4, + "ops_per_second": 637.2445443482059, + "ops_min": 584.3929427466398, + "ops_max": 690.0446857446985, + "p50_us": 65.5625, + "p95_us": 7664.7705000000005, + "p99_us": 9871.854, + "checkpoint_ms": 58.901104000000004, + "reopen_ms": 5.569917, + "drain_ops_per_second": 631.5291091333788, + "metadata_allocated_mib": 8.79296875, + "metadata_logical_mib": 8.48046875, + "view_ms": 241.9070165, + "apply_ms": 5893.591682, + "object_ms": 0.0, + "commits": 1492.0, + "cas_retries": 0, + "max_rss_mib": 18.46875, + "per_operation": { + "list": { + "mean_us": 274.1916728813559, + "p50_us": 266.958, + "p95_us": 487.667, + "p99_us": 546.5419999999999, + "count": 590.0 + }, + "publish": { + "mean_us": 4054.468347189696, + "p50_us": 3970.3334999999997, + "p95_us": 5464.875, + "p99_us": 11486.2085, + "count": 854.0 + }, + "read": { + "mean_us": 31.508402236421723, + "p50_us": 22.854, + "p95_us": 75.125, + "p99_us": 107.5205, + "count": 1565.0 + }, + "remove_recreate": { + "mean_us": 7809.617858333333, + "p50_us": 7771.5830000000005, + "p95_us": 10522.520499999999, + "p99_us": 19430.354, + "count": 120.0 + }, + "rename_roundtrip": { + "mean_us": 8018.213532663317, + "p50_us": 7916.9169999999995, + "p95_us": 10545.7085, + "p99_us": 24657.8125, + "count": 199.0 + }, + "snapshot": { + "mean_us": 415.80386428571427, + "p50_us": 409.5, + "p95_us": 728.625, + "p99_us": 1118.708, + "count": 70.0 + }, + "stat": { + "mean_us": 32.375119601328905, + "p50_us": 23.125, + "p95_us": 74.2085, + "p99_us": 98.8335, + "count": 602.0 + } + } + }, + "holt_vs_sqlite_throughput_ratio": 1.1350979797369651 + }, + "content_async_1": { + "holt": { + "runs": 4, + "ops_per_second": 2126.1031435543755, + "ops_min": 1879.819852815626, + "ops_max": 2324.5853753876336, + "p50_us": 97.6875, + "p95_us": 601.5830000000001, + "p99_us": 1146.4375, + "checkpoint_ms": 71.0602085, + "reopen_ms": 19.291167, + "drain_ops_per_second": 2069.3409485830875, + "metadata_allocated_mib": 25.818359375, + "metadata_logical_mib": 28.02133083343506, + "view_ms": 293.31370549999997, + "apply_ms": 30.5191315, + "object_ms": 2466.5735965, + "commits": 2227.0, + "cas_retries": 0, + "max_rss_mib": 48.71875, + "per_operation": { + "list": { + "mean_us": 83.80903367003367, + "p50_us": 81.0, + "p95_us": 103.917, + "p99_us": 135.27100000000002, + "count": 891.0 + }, + "publish": { + "mean_us": 386.29732031872516, + "p50_us": 344.3755, + "p95_us": 619.0835, + "p99_us": 1060.813, + "count": 1255.0 + }, + "read": { + "mean_us": 901.8181148100725, + "p50_us": 322.979, + "p95_us": 768.583, + "p99_us": 1819.292, + "count": 2343.0 + }, + "remove_recreate": { + "mean_us": 464.0320364583333, + "p50_us": 393.70849999999996, + "p95_us": 798.354, + "p99_us": 1937.7295, + "count": 192.0 + }, + "rename_roundtrip": { + "mean_us": 81.84955782312926, + "p50_us": 76.8955, + "p95_us": 110.7705, + "p99_us": 150.9375, + "count": 294.0 + }, + "snapshot": { + "mean_us": 184.8984072580645, + "p50_us": 170.708, + "p95_us": 277.33349999999996, + "p99_us": 321.91700000000003, + "count": 124.0 + }, + "stat": { + "mean_us": 44.32742841287458, + "p50_us": 40.75, + "p95_us": 62.583, + "p99_us": 110.792, + "count": 901.0 + } + } + }, + "sqlite": { + "runs": 4, + "ops_per_second": 2105.2495807516857, + "ops_min": 1465.4496433034508, + "ops_max": 2399.2398008690943, + "p50_us": 152.66649999999998, + "p95_us": 680.8544999999999, + "p99_us": 1215.229, + "checkpoint_ms": 24.909270999999997, + "reopen_ms": 1.7755830000000001, + "drain_ops_per_second": 2086.0696105842567, + "metadata_allocated_mib": 3.71875, + "metadata_logical_mib": 3.484375, + "view_ms": 92.434202, + "apply_ms": 300.7606015, + "object_ms": 2396.1586555000004, + "commits": 2227.0, + "cas_retries": 0, + "max_rss_mib": 11.9453125, + "per_operation": { + "list": { + "mean_us": 72.61557295173962, + "p50_us": 66.1045, + "p95_us": 120.2915, + "p99_us": 204.91649999999998, + "count": 891.0 + }, + "publish": { + "mean_us": 499.5214486055777, + "p50_us": 433.66700000000003, + "p95_us": 828.271, + "p99_us": 1456.8545, + "count": 1255.0 + }, + "read": { + "mean_us": 829.1077667520274, + "p50_us": 294.459, + "p95_us": 730.5835, + "p99_us": 1445.5, + "count": 2343.0 + }, + "remove_recreate": { + "mean_us": 617.4639713541667, + "p50_us": 546.4165, + "p95_us": 1096.7289999999998, + "p99_us": 1972.667, + "count": 192.0 + }, + "rename_roundtrip": { + "mean_us": 249.03505442176873, + "p50_us": 207.52100000000002, + "p95_us": 477.3745, + "p99_us": 1243.792, + "count": 294.0 + }, + "snapshot": { + "mean_us": 105.1604435483871, + "p50_us": 95.1875, + "p95_us": 191.27100000000002, + "p99_us": 308.77049999999997, + "count": 124.0 + }, + "stat": { + "mean_us": 7.21733795782464, + "p50_us": 6.0625, + "p95_us": 14.4375, + "p99_us": 29.458, + "count": 901.0 + } + } + }, + "holt_vs_sqlite_throughput_ratio": 1.009905506213303 + }, + "content_async_4": { + "holt": { + "runs": 4, + "ops_per_second": 802.8402858495122, + "ops_min": 576.520545012256, + "ops_max": 826.4037940473653, + "p50_us": 2760.1040000000003, + "p95_us": 13756.5, + "p99_us": 26752.854, + "checkpoint_ms": 2129.4105625, + "reopen_ms": 27.0653965, + "drain_ops_per_second": 626.1492324650585, + "metadata_allocated_mib": 1803.4765625, + "metadata_logical_mib": 1808.3290457725525, + "view_ms": 20286.3765315, + "apply_ms": 2676.2613075, + "object_ms": 2716.8183790000003, + "commits": 2135.0, + "cas_retries": 0, + "max_rss_mib": 1845.1875, + "per_operation": { + "list": { + "mean_us": 7618.577860898138, + "p50_us": 4679.021, + "p95_us": 25417.0835, + "p99_us": 38653.063, + "count": 913.0 + }, + "publish": { + "mean_us": 3880.231243871513, + "p50_us": 3001.125, + "p95_us": 10915.292, + "p99_us": 17182.75, + "count": 1183.0 + }, + "read": { + "mean_us": 3506.329586526576, + "p50_us": 2459.583, + "p95_us": 7960.8125, + "p99_us": 13224.4165, + "count": 2427.0 + }, + "remove_recreate": { + "mean_us": 7541.8132578125005, + "p50_us": 6640.8755, + "p95_us": 18149.896, + "p99_us": 26034.9585, + "count": 192.0 + }, + "rename_roundtrip": { + "mean_us": 6247.979357394366, + "p50_us": 4910.5, + "p95_us": 16712.000500000002, + "p99_us": 24178.208, + "count": 284.0 + }, + "snapshot": { + "mean_us": 10393.88728037383, + "p50_us": 8140.188, + "p95_us": 30768.542, + "p99_us": 39405.9375, + "count": 107.0 + }, + "stat": { + "mean_us": 2458.1579110738253, + "p50_us": 1996.7085000000002, + "p95_us": 6687.916499999999, + "p99_us": 11422.125, + "count": 894.0 + } + } + }, + "sqlite": { + "runs": 4, + "ops_per_second": 3338.8505953652934, + "ops_min": 2299.243766539193, + "ops_max": 5048.947263276283, + "p50_us": 306.854, + "p95_us": 1675.646, + "p99_us": 6323.728999999999, + "checkpoint_ms": 38.79375, + "reopen_ms": 1.919625, + "drain_ops_per_second": 3277.8503786649303, + "metadata_allocated_mib": 3.65625, + "metadata_logical_mib": 3.3828125, + "view_ms": 135.3697855, + "apply_ms": 1377.016537, + "object_ms": 4671.1366915, + "commits": 2135.0, + "cas_retries": 0, + "max_rss_mib": 12.015625, + "per_operation": { + "list": { + "mean_us": 82.27957721796275, + "p50_us": 71.5625, + "p95_us": 142.35399999999998, + "p99_us": 215.75, + "count": 913.0 + }, + "publish": { + "mean_us": 1156.6954535080304, + "p50_us": 460.41700000000003, + "p95_us": 4051.5625, + "p99_us": 10680.9375, + "count": 1183.0 + }, + "read": { + "mean_us": 1761.081730119489, + "p50_us": 348.187, + "p95_us": 1173.0625, + "p99_us": 42862.354, + "count": 2427.0 + }, + "remove_recreate": { + "mean_us": 1361.0706302083331, + "p50_us": 591.1875, + "p95_us": 4317.5415, + "p99_us": 11925.563, + "count": 192.0 + }, + "rename_roundtrip": { + "mean_us": 660.1366778169015, + "p50_us": 235.70850000000002, + "p95_us": 1843.3335000000002, + "p99_us": 10157.9785, + "count": 284.0 + }, + "snapshot": { + "mean_us": 146.8582523364486, + "p50_us": 129.35399999999998, + "p95_us": 255.3125, + "p99_us": 338.45799999999997, + "count": 107.0 + }, + "stat": { + "mean_us": 14.44010067114094, + "p50_us": 9.1045, + "p95_us": 32.146, + "p99_us": 70.9375, + "count": 894.0 + } + } + }, + "holt_vs_sqlite_throughput_ratio": 0.24045409128636852 + }, + "content_sync_1": { + "holt": { + "runs": 4, + "ops_per_second": 323.0148888211014, + "ops_min": 178.6304686526736, + "ops_max": 417.564807027584, + "p50_us": 506.1875, + "p95_us": 16633.792, + "p99_us": 25586.417, + "checkpoint_ms": 16.256104, + "reopen_ms": 14.579479500000001, + "drain_ops_per_second": 322.1130927646516, + "metadata_allocated_mib": 26.416015625, + "metadata_logical_mib": 28.041523933410645, + "view_ms": 90.737256, + "apply_ms": 2259.5423885, + "object_ms": 2852.9018225, + "commits": 604.0, + "cas_retries": 0, + "max_rss_mib": 47.375, + "per_operation": { + "list": { + "mean_us": 93.8928179916318, + "p50_us": 88.25, + "p95_us": 133.375, + "p99_us": 215.6255, + "count": 239.0 + }, + "publish": { + "mean_us": 9771.853632183907, + "p50_us": 7431.3125, + "p95_us": 22494.25, + "p99_us": 26068.4585, + "count": 348.0 + }, + "read": { + "mean_us": 570.3236507293354, + "p50_us": 525.646, + "p95_us": 1388.8125, + "p99_us": 2140.9584999999997, + "count": 617.0 + }, + "remove_recreate": { + "mean_us": 14002.142641509434, + "p50_us": 10051.854, + "p95_us": 30341.2705, + "p99_us": 42710.084, + "count": 53.0 + }, + "rename_roundtrip": { + "mean_us": 8252.65832, + "p50_us": 5973.771, + "p95_us": 14908.270499999999, + "p99_us": 16517.25, + "count": 75.0 + }, + "snapshot": { + "mean_us": 184.16836, + "p50_us": 170.3125, + "p95_us": 242.39600000000002, + "p99_us": 285.20849999999996, + "count": 25.0 + }, + "stat": { + "mean_us": 50.99741563786008, + "p50_us": 44.75, + "p95_us": 86.7495, + "p99_us": 159.47899999999998, + "count": 243.0 + } + } + }, + "sqlite": { + "runs": 4, + "ops_per_second": 365.4853129067483, + "ops_min": 327.4665400189823, + "ops_max": 382.0413883836104, + "p50_us": 549.125, + "p95_us": 10627.0, + "p99_us": 13167.854, + "checkpoint_ms": 10.9365205, + "reopen_ms": 3.934208, + "drain_ops_per_second": 364.6360138890059, + "metadata_allocated_mib": 2.46875, + "metadata_logical_mib": 2.0390625, + "view_ms": 79.14165249999999, + "apply_ms": 1837.7057625000002, + "object_ms": 2403.3224345, + "commits": 604.0, + "cas_retries": 0, + "max_rss_mib": 9.9921875, + "per_operation": { + "list": { + "mean_us": 165.21285146443515, + "p50_us": 150.0, + "p95_us": 315.91650000000004, + "p99_us": 424.58299999999997, + "count": 239.0 + }, + "publish": { + "mean_us": 8121.203537356321, + "p50_us": 7724.7505, + "p95_us": 11828.25, + "p99_us": 15006.25, + "count": 348.0 + }, + "read": { + "mean_us": 751.5714416531605, + "p50_us": 577.542, + "p95_us": 1766.4585, + "p99_us": 3587.375, + "count": 617.0 + }, + "remove_recreate": { + "mean_us": 11126.388358490565, + "p50_us": 10774.625, + "p95_us": 14622.9375, + "p99_us": 17011.0835, + "count": 53.0 + }, + "rename_roundtrip": { + "mean_us": 6223.595313333333, + "p50_us": 6204.125, + "p95_us": 8323.0, + "p99_us": 11759.854, + "count": 75.0 + }, + "snapshot": { + "mean_us": 203.81658, + "p50_us": 188.666, + "p95_us": 371.5005, + "p99_us": 431.47950000000003, + "count": 25.0 + }, + "stat": { + "mean_us": 26.668907407407403, + "p50_us": 17.75, + "p95_us": 64.6875, + "p99_us": 95.0415, + "count": 243.0 + } + } + }, + "holt_vs_sqlite_throughput_ratio": 0.8837971798432213 + }, + "metadata_async_1": { + "holt": { + "runs": 4, + "ops_per_second": 20123.846179696135, + "ops_min": 20094.865885851963, + "ops_max": 20227.838483824795, + "p50_us": 39.6045, + "p95_us": 76.4995, + "p99_us": 158.8545, + "checkpoint_ms": 60.649375000000006, + "reopen_ms": 27.1134165, + "drain_ops_per_second": 18266.043949261846, + "metadata_allocated_mib": 30.494140625, + "metadata_logical_mib": 37.12917613983154, + "view_ms": 491.8270685, + "apply_ms": 51.653818, + "object_ms": 0.0, + "commits": 4379.0, + "cas_retries": 0, + "max_rss_mib": 61.296875, + "per_operation": { + "list": { + "mean_us": 73.46477692728267, + "p50_us": 73.271, + "p95_us": 79.1875, + "p99_us": 84.2715, + "count": 1829.0 + }, + "publish": { + "mean_us": 40.199795261239366, + "p50_us": 36.9995, + "p95_us": 42.75, + "p99_us": 147.9995, + "count": 2469.0 + }, + "read": { + "mean_us": 37.93371808622503, + "p50_us": 37.7495, + "p95_us": 42.896, + "p99_us": 46.3125, + "count": 4755.0 + }, + "remove_recreate": { + "mean_us": 78.35263049450549, + "p50_us": 71.6665, + "p95_us": 80.6875, + "p99_us": 365.7085, + "count": 364.0 + }, + "rename_roundtrip": { + "mean_us": 77.60568358714045, + "p50_us": 72.104, + "p95_us": 79.0835, + "p99_us": 338.33299999999997, + "count": 591.0 + }, + "snapshot": { + "mean_us": 159.27934581497797, + "p50_us": 155.0, + "p95_us": 195.16649999999998, + "p99_us": 245.292, + "count": 227.0 + }, + "stat": { + "mean_us": 38.07197110481586, + "p50_us": 37.9375, + "p95_us": 43.0835, + "p99_us": 46.25, + "count": 1765.0 + } + } + }, + "sqlite": { + "runs": 4, + "ops_per_second": 16216.271500179577, + "ops_min": 15349.815895715214, + "ops_max": 16710.58417069026, + "p50_us": 5.646, + "p95_us": 222.1875, + "p99_us": 301.0, + "checkpoint_ms": 27.0126665, + "reopen_ms": 4.2226665, + "drain_ops_per_second": 15563.70192688785, + "metadata_allocated_mib": 4.97265625, + "metadata_logical_mib": 4.97265625, + "view_ms": 112.85381799999999, + "apply_ms": 567.638139, + "object_ms": 0.0, + "commits": 4379.0, + "cas_retries": 0, + "max_rss_mib": 13.75, + "per_operation": { + "list": { + "mean_us": 48.96286276653909, + "p50_us": 47.771, + "p95_us": 56.5205, + "p99_us": 62.959, + "count": 1829.0 + }, + "publish": { + "mean_us": 140.78616605913328, + "p50_us": 134.333, + "p95_us": 215.4375, + "p99_us": 279.3545, + "count": 2469.0 + }, + "read": { + "mean_us": 4.305696319663513, + "p50_us": 4.0, + "p95_us": 6.1665, + "p99_us": 6.8335, + "count": 4755.0 + }, + "remove_recreate": { + "mean_us": 262.4009423076923, + "p50_us": 238.77100000000002, + "p95_us": 353.3755, + "p99_us": 481.7495, + "count": 364.0 + }, + "rename_roundtrip": { + "mean_us": 270.20704653130286, + "p50_us": 219.29149999999998, + "p95_us": 316.313, + "p99_us": 812.2285, + "count": 591.0 + }, + "snapshot": { + "mean_us": 74.29775110132158, + "p50_us": 73.271, + "p95_us": 83.4585, + "p99_us": 91.6875, + "count": 227.0 + }, + "stat": { + "mean_us": 4.342175070821529, + "p50_us": 4.021, + "p95_us": 6.2085, + "p99_us": 6.9375, + "count": 1765.0 + } + } + }, + "holt_vs_sqlite_throughput_ratio": 1.2409662837399635 + }, + "metadata_async_4": { + "holt": { + "runs": 4, + "ops_per_second": 230.02036080745398, + "ops_min": 228.23169102729716, + "ops_max": 231.91042442957868, + "p50_us": 8323.6665, + "p95_us": 59715.0, + "p99_us": 139192.812, + "checkpoint_ms": 13346.489354, + "reopen_ms": 47.220146, + "drain_ops_per_second": 184.12831807673513, + "metadata_allocated_mib": 8371.92578125, + "metadata_logical_mib": 8369.56176996231, + "view_ms": 180760.329295, + "apply_ms": 20308.0812145, + "object_ms": 0.0, + "commits": 4297.0, + "cas_retries": 0, + "max_rss_mib": 6639.0, + "per_operation": { + "list": { + "mean_us": 41280.64599500832, + "p50_us": 23622.958, + "p95_us": 146614.646, + "p99_us": 247440.396, + "count": 1803.0 + }, + "publish": { + "mean_us": 12963.554251962, + "p50_us": 8535.896, + "p95_us": 40387.3335, + "p99_us": 72340.875, + "count": 2421.0 + }, + "read": { + "mean_us": 9356.887667328181, + "p50_us": 6155.2080000000005, + "p95_us": 27545.2705, + "p99_us": 49582.583, + "count": 4787.0 + }, + "remove_recreate": { + "mean_us": 23481.65550137363, + "p50_us": 15236.604, + "p95_us": 67607.85399999999, + "p99_us": 98908.16649999999, + "count": 364.0 + }, + "rename_roundtrip": { + "mean_us": 24337.713526132407, + "p50_us": 16654.25, + "p95_us": 70661.95850000001, + "p99_us": 101083.083, + "count": 574.0 + }, + "snapshot": { + "mean_us": 45127.54786936937, + "p50_us": 30384.9585, + "p95_us": 151530.6875, + "p99_us": 229110.125, + "count": 222.0 + }, + "stat": { + "mean_us": 9720.526473209404, + "p50_us": 6556.0, + "p95_us": 29000.124499999998, + "p99_us": 49529.6045, + "count": 1829.0 + } + } + }, + "sqlite": { + "runs": 4, + "ops_per_second": 14776.398824697011, + "ops_min": 13793.508406657153, + "ops_max": 15598.281849254305, + "p50_us": 5.2915, + "p95_us": 234.062, + "p99_us": 339.14599999999996, + "checkpoint_ms": 29.696645999999998, + "reopen_ms": 4.7042085, + "drain_ops_per_second": 14119.760160411028, + "metadata_allocated_mib": 4.994140625, + "metadata_logical_mib": 4.994140625, + "view_ms": 112.6849045, + "apply_ms": 2015.7975450000001, + "object_ms": 0.0, + "commits": 4297.0, + "cas_retries": 0, + "max_rss_mib": 14.125, + "per_operation": { + "list": { + "mean_us": 49.29449500831947, + "p50_us": 47.854, + "p95_us": 57.1045, + "p99_us": 66.146, + "count": 1803.0 + }, + "publish": { + "mean_us": 646.4797866584056, + "p50_us": 140.4375, + "p95_us": 238.4375, + "p99_us": 1236.104, + "count": 2421.0 + }, + "read": { + "mean_us": 4.310860142051389, + "p50_us": 3.9585, + "p95_us": 6.0, + "p99_us": 7.896, + "count": 4787.0 + }, + "remove_recreate": { + "mean_us": 968.2076675824176, + "p50_us": 249.959, + "p95_us": 387.7085, + "p99_us": 2768.6875, + "count": 364.0 + }, + "rename_roundtrip": { + "mean_us": 310.7524181184669, + "p50_us": 221.833, + "p95_us": 360.8335, + "p99_us": 2661.521, + "count": 574.0 + }, + "snapshot": { + "mean_us": 77.40804954954955, + "p50_us": 74.7915, + "p95_us": 92.5625, + "p99_us": 124.6245, + "count": 222.0 + }, + "stat": { + "mean_us": 4.2958102788408965, + "p50_us": 3.958, + "p95_us": 5.75, + "p99_us": 7.812, + "count": 1829.0 + } + } + }, + "holt_vs_sqlite_throughput_ratio": 0.015566740146658875 + }, + "metadata_point_async_1": { + "holt": { + "runs": 4, + "ops_per_second": 33571.28191041517, + "ops_min": 33183.439497019135, + "ops_max": 34114.19707000667, + "p50_us": 2.1875, + "p95_us": 75.4375, + "p99_us": 153.438, + "checkpoint_ms": 49.7075205, + "reopen_ms": 23.8631665, + "drain_ops_per_second": 29407.12932325436, + "metadata_allocated_mib": 30.51953125, + "metadata_logical_mib": 37.00905895233154, + "view_ms": 246.745408, + "apply_ms": 50.7077365, + "object_ms": 0.0, + "commits": 4379.0, + "cas_retries": 0, + "max_rss_mib": 61.296875, + "per_operation": { + "list": { + "mean_us": 72.98814953526517, + "p50_us": 73.0, + "p95_us": 77.4375, + "p99_us": 82.5, + "count": 1829.0 + }, + "publish": { + "mean_us": 40.11899189955447, + "p50_us": 37.0835, + "p95_us": 41.958, + "p99_us": 145.2705, + "count": 2469.0 + }, + "read": { + "mean_us": 1.7797006309148267, + "p50_us": 1.75, + "p95_us": 2.25, + "p99_us": 2.479, + "count": 4755.0 + }, + "remove_recreate": { + "mean_us": 77.63138324175824, + "p50_us": 71.313, + "p95_us": 77.2295, + "p99_us": 350.687, + "count": 364.0 + }, + "rename_roundtrip": { + "mean_us": 77.35695093062606, + "p50_us": 71.875, + "p95_us": 77.896, + "p99_us": 329.4585, + "count": 591.0 + }, + "snapshot": { + "mean_us": 153.27222687224668, + "p50_us": 150.3545, + "p95_us": 172.45850000000002, + "p99_us": 226.4375, + "count": 227.0 + }, + "stat": { + "mean_us": 1.8182691218130311, + "p50_us": 1.75, + "p95_us": 2.292, + "p99_us": 2.542, + "count": 1765.0 + } + } + }, + "sqlite": { + "runs": 4, + "ops_per_second": 16299.144108397997, + "ops_min": 15341.131635379015, + "ops_max": 16827.55779811058, + "p50_us": 6.167, + "p95_us": 216.8125, + "p99_us": 288.9795, + "checkpoint_ms": 35.7482295, + "reopen_ms": 5.39625, + "drain_ops_per_second": 15551.237977592184, + "metadata_allocated_mib": 4.97265625, + "metadata_logical_mib": 4.97265625, + "view_ms": 86.2001865, + "apply_ms": 565.7069395000001, + "object_ms": 0.0, + "commits": 4379.0, + "cas_retries": 0, + "max_rss_mib": 13.7578125, + "per_operation": { + "list": { + "mean_us": 47.54358583925642, + "p50_us": 46.709, + "p95_us": 53.8955, + "p99_us": 58.2085, + "count": 1829.0 + }, + "publish": { + "mean_us": 144.04598582422034, + "p50_us": 131.22899999999998, + "p95_us": 207.45850000000002, + "p99_us": 270.187, + "count": 2469.0 + }, + "read": { + "mean_us": 5.151709253417455, + "p50_us": 4.834, + "p95_us": 6.937, + "p99_us": 7.625, + "count": 4755.0 + }, + "remove_recreate": { + "mean_us": 270.01175274725273, + "p50_us": 233.3125, + "p95_us": 344.6255, + "p99_us": 930.75, + "count": 364.0 + }, + "rename_roundtrip": { + "mean_us": 242.03742808798648, + "p50_us": 215.2295, + "p95_us": 305.14549999999997, + "p99_us": 1059.8335000000002, + "count": 591.0 + }, + "snapshot": { + "mean_us": 71.2397268722467, + "p50_us": 69.9585, + "p95_us": 78.896, + "p99_us": 84.583, + "count": 227.0 + }, + "stat": { + "mean_us": 5.1865518413597735, + "p50_us": 4.875, + "p95_us": 6.6665, + "p99_us": 7.667, + "count": 1765.0 + } + } + }, + "holt_vs_sqlite_throughput_ratio": 2.0596959991977646 + }, + "metadata_sync_1": { + "holt": { + "runs": 4, + "ops_per_second": 553.3614400615238, + "ops_min": 286.09470654037386, + "ops_max": 807.6916883325755, + "p50_us": 211.8545, + "p95_us": 8271.0005, + "p99_us": 20068.061999999998, + "checkpoint_ms": 36.854625500000004, + "reopen_ms": 24.633542000000002, + "drain_ops_per_second": 550.5374632891195, + "metadata_allocated_mib": 21.05078125, + "metadata_logical_mib": 22.201330184936523, + "view_ms": 660.3522885, + "apply_ms": 6485.0799175, + "object_ms": 0.0, + "commits": 1492.0, + "cas_retries": 0, + "max_rss_mib": 48.0, + "per_operation": { + "list": { + "mean_us": 243.1041754237288, + "p50_us": 219.125, + "p95_us": 442.02099999999996, + "p99_us": 672.396, + "count": 590.0 + }, + "publish": { + "mean_us": 4397.851340163934, + "p50_us": 3363.8125, + "p95_us": 11093.271, + "p99_us": 12918.438, + "count": 854.0 + }, + "read": { + "mean_us": 157.8432530351438, + "p50_us": 133.52100000000002, + "p95_us": 329.33299999999997, + "p99_us": 549.146, + "count": 1565.0 + }, + "remove_recreate": { + "mean_us": 9242.2539875, + "p50_us": 6590.458500000001, + "p95_us": 22662.1045, + "p99_us": 27019.041, + "count": 120.0 + }, + "rename_roundtrip": { + "mean_us": 9200.443040201004, + "p50_us": 6512.5205, + "p95_us": 22238.2915, + "p99_us": 30009.688000000002, + "count": 199.0 + }, + "snapshot": { + "mean_us": 557.4782571428572, + "p50_us": 522.2080000000001, + "p95_us": 952.271, + "p99_us": 1523.812, + "count": 70.0 + }, + "stat": { + "mean_us": 151.76476910299004, + "p50_us": 128.813, + "p95_us": 327.0625, + "p99_us": 470.375, + "count": 602.0 + } + } + }, + "sqlite": { + "runs": 4, + "ops_per_second": 752.8109361795423, + "ops_min": 414.1661082433967, + "ops_max": 784.2910745488329, + "p50_us": 56.0835, + "p95_us": 6600.9785, + "p99_us": 7963.1255, + "checkpoint_ms": 11.435624999999998, + "reopen_ms": 5.879458, + "drain_ops_per_second": 750.6942516876042, + "metadata_allocated_mib": 3.53515625, + "metadata_logical_mib": 2.609375, + "view_ms": 173.91928000000001, + "apply_ms": 5042.362358, + "object_ms": 0.0, + "commits": 1492.0, + "cas_retries": 0, + "max_rss_mib": 10.859375, + "per_operation": { + "list": { + "mean_us": 189.7740254237288, + "p50_us": 167.9375, + "p95_us": 347.0625, + "p99_us": 520.0835, + "count": 590.0 + }, + "publish": { + "mean_us": 3445.059570257611, + "p50_us": 3642.3545, + "p95_us": 4313.1044999999995, + "p99_us": 5092.5205000000005, + "count": 854.0 + }, + "read": { + "mean_us": 23.20313003194888, + "p50_us": 14.0205, + "p95_us": 64.8955, + "p99_us": 100.604, + "count": 1565.0 + }, + "remove_recreate": { + "mean_us": 6868.949820833333, + "p50_us": 6900.271, + "p95_us": 8331.042, + "p99_us": 8953.292, + "count": 120.0 + }, + "rename_roundtrip": { + "mean_us": 6855.565977386935, + "p50_us": 6939.8125, + "p95_us": 8097.437, + "p99_us": 8874.438, + "count": 199.0 + }, + "snapshot": { + "mean_us": 279.2086071428571, + "p50_us": 253.39600000000002, + "p95_us": 583.4165, + "p99_us": 696.271, + "count": 70.0 + }, + "stat": { + "mean_us": 23.202145348837206, + "p50_us": 14.437000000000001, + "p95_us": 66.125, + "p99_us": 104.083, + "count": 602.0 + } + } + }, + "holt_vs_sqlite_throughput_ratio": 0.7350603099229545 + } + }, + "followup": { + "content_async_4_background": { + "holt": { + "runs": 4, + "ops_per_second": 1810.9449404524155, + "ops_min": 1492.9496388266123, + "ops_max": 2079.903343064626, + "p50_us": 1187.833, + "p95_us": 5341.125, + "p99_us": 10176.7705, + "checkpoint_ms": 46.456937499999995, + "reopen_ms": 26.014042, + "drain_ops_per_second": 1776.3778033913816, + "metadata_allocated_mib": 1152.607421875, + "metadata_logical_mib": 1170.541841506958, + "view_ms": 4873.378873, + "apply_ms": 662.792595, + "object_ms": 3226.3017895, + "commits": 1403.0, + "cas_retries": 0, + "max_rss_mib": 399.0703125, + "per_operation": { + "list": { + "mean_us": 2708.271014096186, + "p50_us": 1759.7295, + "p95_us": 8405.875, + "p99_us": 12263.188, + "count": 603.0 + }, + "publish": { + "mean_us": 2175.7325984752224, + "p50_us": 1796.896, + "p95_us": 5278.478999999999, + "p99_us": 7105.687, + "count": 787.0 + }, + "read": { + "mean_us": 2302.6335039804044, + "p50_us": 1036.583, + "p95_us": 3299.438, + "p99_us": 13355.5, + "count": 1633.0 + }, + "remove_recreate": { + "mean_us": 3352.876456, + "p50_us": 3049.6875, + "p95_us": 7655.375, + "p99_us": 10272.7505, + "count": 125.0 + }, + "rename_roundtrip": { + "mean_us": 2288.9205327868854, + "p50_us": 1781.2295, + "p95_us": 6282.353999999999, + "p99_us": 10333.25, + "count": 183.0 + }, + "snapshot": { + "mean_us": 3706.982105633803, + "p50_us": 3261.8125, + "p95_us": 9295.457999999999, + "p99_us": 13367.167000000001, + "count": 71.0 + }, + "stat": { + "mean_us": 868.7035677257525, + "p50_us": 649.7915, + "p95_us": 2504.0209999999997, + "p99_us": 4255.3330000000005, + "count": 598.0 + } + } + }, + "sqlite": { + "runs": 4, + "ops_per_second": 3935.389942304799, + "ops_min": 1880.7520149692964, + "ops_max": 4037.9372604529053, + "p50_us": 277.833, + "p95_us": 1483.833, + "p99_us": 4173.562, + "checkpoint_ms": 12.194166500000001, + "reopen_ms": 1.687125, + "drain_ops_per_second": 3876.9966340332116, + "metadata_allocated_mib": 2.76953125, + "metadata_logical_mib": 2.642578125, + "view_ms": 80.36440250000001, + "apply_ms": 722.15744, + "object_ms": 2395.805007, + "commits": 1403.0, + "cas_retries": 0, + "max_rss_mib": 11.0625, + "per_operation": { + "list": { + "mean_us": 78.95609701492538, + "p50_us": 63.8335, + "p95_us": 172.7295, + "p99_us": 242.271, + "count": 603.0 + }, + "publish": { + "mean_us": 908.9673265565439, + "p50_us": 384.1875, + "p95_us": 1977.146, + "p99_us": 7540.2919999999995, + "count": 787.0 + }, + "read": { + "mean_us": 1340.4138860992039, + "p50_us": 312.27099999999996, + "p95_us": 839.5415, + "p99_us": 3488.4584999999997, + "count": 1633.0 + }, + "remove_recreate": { + "mean_us": 1312.483148, + "p50_us": 513.5, + "p95_us": 4232.25, + "p99_us": 11121.604500000001, + "count": 125.0 + }, + "rename_roundtrip": { + "mean_us": 571.0405273224044, + "p50_us": 185.8545, + "p95_us": 1494.417, + "p99_us": 10224.8125, + "count": 183.0 + }, + "snapshot": { + "mean_us": 145.54400704225353, + "p50_us": 116.021, + "p95_us": 303.3125, + "p99_us": 486.896, + "count": 71.0 + }, + "stat": { + "mean_us": 11.44383695652174, + "p50_us": 7.9165, + "p95_us": 26.3125, + "p99_us": 45.9795, + "count": 598.0 + } + } + }, + "holt_vs_sqlite_throughput_ratio": 0.4601691235181178 + }, + "content_sync_1": { + "holt": { + "runs": 4, + "ops_per_second": 399.6768073447783, + "ops_min": 362.88165564711846, + "ops_max": 431.54121235435065, + "p50_us": 517.8335, + "p95_us": 9100.271, + "p99_us": 11420.832999999999, + "checkpoint_ms": 16.4101665, + "reopen_ms": 18.529229, + "drain_ops_per_second": 398.03542227262, + "metadata_allocated_mib": 23.54296875, + "metadata_logical_mib": 28.014424324035645, + "view_ms": 76.7804865, + "apply_ms": 1524.8035825, + "object_ms": 2364.38504, + "commits": 604.0, + "cas_retries": 0, + "max_rss_mib": 46.3359375, + "per_operation": { + "list": { + "mean_us": 79.15456485355648, + "p50_us": 77.1045, + "p95_us": 95.9585, + "p99_us": 112.521, + "count": 239.0 + }, + "publish": { + "mean_us": 6784.372025862069, + "p50_us": 6432.3955000000005, + "p95_us": 10071.583, + "p99_us": 11457.4585, + "count": 348.0 + }, + "read": { + "mean_us": 928.6126799027552, + "p50_us": 549.7919999999999, + "p95_us": 1474.229, + "p99_us": 3435.917, + "count": 617.0 + }, + "remove_recreate": { + "mean_us": 9234.021679245283, + "p50_us": 9180.021, + "p95_us": 13359.9375, + "p99_us": 15077.542000000001, + "count": 53.0 + }, + "rename_roundtrip": { + "mean_us": 5262.99216, + "p50_us": 5056.1875, + "p95_us": 7356.7295, + "p99_us": 13175.896, + "count": 75.0 + }, + "snapshot": { + "mean_us": 182.44416, + "p50_us": 172.125, + "p95_us": 271.8125, + "p99_us": 286.0205, + "count": 25.0 + }, + "stat": { + "mean_us": 44.63101646090534, + "p50_us": 43.5625, + "p95_us": 57.771, + "p99_us": 66.417, + "count": 243.0 + } + } + }, + "sqlite": { + "runs": 4, + "ops_per_second": 313.24449455564036, + "ops_min": 291.6918526249703, + "ops_max": 327.0722004216676, + "p50_us": 792.1669999999999, + "p95_us": 11668.417, + "p99_us": 14832.332999999999, + "checkpoint_ms": 9.855583, + "reopen_ms": 5.168979, + "drain_ops_per_second": 312.5859666822808, + "metadata_allocated_mib": 2.46875, + "metadata_logical_mib": 2.0390625, + "view_ms": 126.851636, + "apply_ms": 2051.8465850000002, + "object_ms": 2822.038461, + "commits": 604.0, + "cas_retries": 0, + "max_rss_mib": 9.875, + "per_operation": { + "list": { + "mean_us": 280.91606485355646, + "p50_us": 270.3965, + "p95_us": 514.9375, + "p99_us": 551.188, + "count": 239.0 + }, + "publish": { + "mean_us": 8956.82288505747, + "p50_us": 8831.334, + "p95_us": 12650.25, + "p99_us": 14776.6665, + "count": 348.0 + }, + "read": { + "mean_us": 1124.8957236628848, + "p50_us": 856.25, + "p95_us": 2239.417, + "p99_us": 3770.417, + "count": 617.0 + }, + "remove_recreate": { + "mean_us": 12753.655650943398, + "p50_us": 12503.104500000001, + "p95_us": 17413.146, + "p99_us": 20126.4375, + "count": 53.0 + }, + "rename_roundtrip": { + "mean_us": 6966.418573333333, + "p50_us": 7007.3544999999995, + "p95_us": 9762.6665, + "p99_us": 14477.625, + "count": 75.0 + }, + "snapshot": { + "mean_us": 391.93424, + "p50_us": 361.45849999999996, + "p95_us": 726.5205000000001, + "p99_us": 804.084, + "count": 25.0 + }, + "stat": { + "mean_us": 43.374751028806585, + "p50_us": 40.2495, + "p95_us": 87.3755, + "p99_us": 107.917, + "count": 243.0 + } + } + }, + "holt_vs_sqlite_throughput_ratio": 1.2759260395358212 + }, + "metadata_async_4_background": { + "holt": { + "runs": 4, + "ops_per_second": 2012.056112038577, + "ops_min": 1982.7628511534972, + "ops_max": 2033.2304402328134, + "p50_us": 1089.5835, + "p95_us": 7231.6875, + "p99_us": 14519.8335, + "checkpoint_ms": 177.5342915, + "reopen_ms": 23.986292, + "drain_ops_per_second": 1830.4130144571682, + "metadata_allocated_mib": 1468.384765625, + "metadata_logical_mib": 1489.2175254821777, + "view_ms": 7084.668917499999, + "apply_ms": 757.6100759999999, + "object_ms": 0.0, + "commits": 1403.0, + "cas_retries": 0, + "max_rss_mib": 446.5078125, + "per_operation": { + "list": { + "mean_us": 5412.1297985074625, + "p50_us": 4175.541499999999, + "p95_us": 15053.334, + "p99_us": 21419.4165, + "count": 603.0 + }, + "publish": { + "mean_us": 1425.0325146124524, + "p50_us": 1087.75, + "p95_us": 3862.7915000000003, + "p99_us": 5962.2495, + "count": 787.0 + }, + "read": { + "mean_us": 1027.9614846907532, + "p50_us": 819.9375, + "p95_us": 2544.8959999999997, + "p99_us": 4223.875, + "count": 1633.0 + }, + "remove_recreate": { + "mean_us": 2626.1441760000002, + "p50_us": 2229.583, + "p95_us": 6134.187, + "p99_us": 8035.083500000001, + "count": 125.0 + }, + "rename_roundtrip": { + "mean_us": 2778.4110655737704, + "p50_us": 2422.9795, + "p95_us": 6066.978999999999, + "p99_us": 8160.166499999999, + "count": 183.0 + }, + "snapshot": { + "mean_us": 5394.164260563381, + "p50_us": 4221.5835, + "p95_us": 13344.082999999999, + "p99_us": 19759.4585, + "count": 71.0 + }, + "stat": { + "mean_us": 1018.4302090301003, + "p50_us": 829.2085, + "p95_us": 2467.9790000000003, + "p99_us": 4348.521, + "count": 598.0 + } + } + }, + "sqlite": { + "runs": 4, + "ops_per_second": 20950.011803210735, + "ops_min": 20622.667096229947, + "ops_max": 22570.90088472458, + "p50_us": 4.9585, + "p95_us": 153.35399999999998, + "p99_us": 359.54150000000004, + "checkpoint_ms": 9.0308125, + "reopen_ms": 3.4627084999999997, + "drain_ops_per_second": 20002.604986753257, + "metadata_allocated_mib": 2.453125, + "metadata_logical_mib": 2.453125, + "view_ms": 35.7469355, + "apply_ms": 485.964854, + "object_ms": 0.0, + "commits": 1403.0, + "cas_retries": 0, + "max_rss_mib": 10.59375, + "per_operation": { + "list": { + "mean_us": 46.876847429519074, + "p50_us": 45.6665, + "p95_us": 52.3125, + "p99_us": 62.854, + "count": 603.0 + }, + "publish": { + "mean_us": 354.1286365946633, + "p50_us": 89.25, + "p95_us": 185.7705, + "p99_us": 7000.103999999999, + "count": 787.0 + }, + "read": { + "mean_us": 4.156046540110227, + "p50_us": 3.7705, + "p95_us": 5.6875, + "p99_us": 8.1665, + "count": 1633.0 + }, + "remove_recreate": { + "mean_us": 934.663368, + "p50_us": 155.5625, + "p95_us": 382.0, + "p99_us": 12203.5835, + "count": 125.0 + }, + "rename_roundtrip": { + "mean_us": 467.8992349726776, + "p50_us": 142.25, + "p95_us": 308.41700000000003, + "p99_us": 4176.104, + "count": 183.0 + }, + "snapshot": { + "mean_us": 73.9806338028169, + "p50_us": 69.7915, + "p95_us": 93.7085, + "p99_us": 139.875, + "count": 71.0 + }, + "stat": { + "mean_us": 4.125929765886288, + "p50_us": 3.7915, + "p95_us": 5.6665, + "p99_us": 8.396, + "count": 598.0 + } + } + }, + "holt_vs_sqlite_throughput_ratio": 0.09604081042714331 + }, + "metadata_async_4_short": { + "holt": { + "runs": 4, + "ops_per_second": 999.5402315344622, + "ops_min": 988.3852482297849, + "ops_max": 1046.751658103693, + "p50_us": 2039.9795, + "p95_us": 14262.75, + "p99_us": 30568.6455, + "checkpoint_ms": 1465.3116455, + "reopen_ms": 16.34875, + "drain_ops_per_second": 727.1387083243308, + "metadata_allocated_mib": 1323.310546875, + "metadata_logical_mib": 1359.8143892288208, + "view_ms": 14203.804248, + "apply_ms": 1430.089135, + "object_ms": 0.0, + "commits": 1403.0, + "cas_retries": 0, + "max_rss_mib": 1390.8984375, + "per_operation": { + "list": { + "mean_us": 10737.037242951908, + "p50_us": 7186.292, + "p95_us": 32893.8335, + "p99_us": 51683.666, + "count": 603.0 + }, + "publish": { + "mean_us": 2642.8969777636594, + "p50_us": 2122.875, + "p95_us": 7099.896000000001, + "p99_us": 11565.0, + "count": 787.0 + }, + "read": { + "mean_us": 2060.9414843845684, + "p50_us": 1693.2289999999998, + "p95_us": 5154.4375, + "p99_us": 8396.770499999999, + "count": 1633.0 + }, + "remove_recreate": { + "mean_us": 5156.978492, + "p50_us": 4251.687, + "p95_us": 12802.3125, + "p99_us": 20576.083, + "count": 125.0 + }, + "rename_roundtrip": { + "mean_us": 5642.02325136612, + "p50_us": 4174.646000000001, + "p95_us": 14897.9585, + "p99_us": 23336.6875, + "count": 183.0 + }, + "snapshot": { + "mean_us": 10212.916950704224, + "p50_us": 8275.4165, + "p95_us": 28892.416, + "p99_us": 41730.125, + "count": 71.0 + }, + "stat": { + "mean_us": 2138.3560610367895, + "p50_us": 1791.5205, + "p95_us": 5419.708, + "p99_us": 9193.666, + "count": 598.0 + } + } + }, + "sqlite": { + "runs": 4, + "ops_per_second": 14960.498785687803, + "ops_min": 14807.420263346785, + "ops_max": 16287.512645828412, + "p50_us": 4.9375, + "p95_us": 228.4375, + "p99_us": 326.66700000000003, + "checkpoint_ms": 15.158687500000001, + "reopen_ms": 5.597625, + "drain_ops_per_second": 14164.656001909327, + "metadata_allocated_mib": 2.5703125, + "metadata_logical_mib": 2.5703125, + "view_ms": 35.952162, + "apply_ms": 667.128095, + "object_ms": 0.0, + "commits": 1403.0, + "cas_retries": 0, + "max_rss_mib": 10.59375, + "per_operation": { + "list": { + "mean_us": 47.47044941956882, + "p50_us": 46.062, + "p95_us": 53.8335, + "p99_us": 61.5835, + "count": 603.0 + }, + "publish": { + "mean_us": 543.6776905972047, + "p50_us": 136.6255, + "p95_us": 233.667, + "p99_us": 1422.0415, + "count": 787.0 + }, + "read": { + "mean_us": 4.161485609308022, + "p50_us": 3.7705, + "p95_us": 5.5, + "p99_us": 7.4585, + "count": 1633.0 + }, + "remove_recreate": { + "mean_us": 361.20449599999995, + "p50_us": 250.9375, + "p95_us": 383.625, + "p99_us": 777.854, + "count": 125.0 + }, + "rename_roundtrip": { + "mean_us": 1021.6494918032787, + "p50_us": 219.39600000000002, + "p95_us": 346.1665, + "p99_us": 22778.646, + "count": 183.0 + }, + "snapshot": { + "mean_us": 73.99353521126761, + "p50_us": 70.312, + "p95_us": 93.875, + "p99_us": 127.833, + "count": 71.0 + }, + "stat": { + "mean_us": 4.061902173913044, + "p50_us": 3.7085, + "p95_us": 5.521, + "p99_us": 7.4795, + "count": 598.0 + } + } + }, + "holt_vs_sqlite_throughput_ratio": 0.06681195900304394 + }, + "metadata_sync_1": { + "holt": { + "runs": 4, + "ops_per_second": 697.8603267888208, + "ops_min": 683.3933354276423, + "ops_max": 718.53726825088, + "p50_us": 281.375, + "p95_us": 6275.125, + "p99_us": 7996.25, + "checkpoint_ms": 48.9023335, + "reopen_ms": 20.8060415, + "drain_ops_per_second": 692.7021480982319, + "metadata_allocated_mib": 21.638671875, + "metadata_logical_mib": 22.239660263061523, + "view_ms": 869.9241105000001, + "apply_ms": 4777.0094525, + "object_ms": 0.0, + "commits": 1492.0, + "cas_retries": 0, + "max_rss_mib": 50.0390625, + "per_operation": { + "list": { + "mean_us": 341.49720762711866, + "p50_us": 320.1875, + "p95_us": 553.125, + "p99_us": 603.521, + "count": 590.0 + }, + "publish": { + "mean_us": 3377.654838407494, + "p50_us": 3493.6040000000003, + "p95_us": 4381.7080000000005, + "p99_us": 5475.583500000001, + "count": 854.0 + }, + "read": { + "mean_us": 207.47789872204476, + "p50_us": 196.89600000000002, + "p95_us": 318.39599999999996, + "p99_us": 397.33299999999997, + "count": 1565.0 + }, + "remove_recreate": { + "mean_us": 6725.515966666666, + "p50_us": 6710.1875, + "p95_us": 8368.021, + "p99_us": 9920.375, + "count": 120.0 + }, + "rename_roundtrip": { + "mean_us": 6700.753231155779, + "p50_us": 6712.8745, + "p95_us": 8470.8335, + "p99_us": 11111.5625, + "count": 199.0 + }, + "snapshot": { + "mean_us": 784.1958571428572, + "p50_us": 754.8330000000001, + "p95_us": 1143.7295, + "p99_us": 1377.8125, + "count": 70.0 + }, + "stat": { + "mean_us": 207.50428820598006, + "p50_us": 193.542, + "p95_us": 321.2915, + "p99_us": 395.97950000000003, + "count": 602.0 + } + } + }, + "sqlite": { + "runs": 4, + "ops_per_second": 656.3313420403556, + "ops_min": 642.604957739693, + "ops_max": 670.3549642607885, + "p50_us": 67.6665, + "p95_us": 7207.103999999999, + "p99_us": 9645.9165, + "checkpoint_ms": 22.3865835, + "reopen_ms": 4.541396, + "drain_ops_per_second": 653.7784331565988, + "metadata_allocated_mib": 3.53515625, + "metadata_logical_mib": 2.609375, + "view_ms": 252.2310605, + "apply_ms": 5700.3037595000005, + "object_ms": 0.0, + "commits": 1492.0, + "cas_retries": 0, + "max_rss_mib": 10.625, + "per_operation": { + "list": { + "mean_us": 282.9503254237288, + "p50_us": 272.97950000000003, + "p95_us": 484.2715, + "p99_us": 560.417, + "count": 590.0 + }, + "publish": { + "mean_us": 3927.5171153395786, + "p50_us": 3944.7709999999997, + "p95_us": 5509.2080000000005, + "p99_us": 6661.0415, + "count": 854.0 + }, + "read": { + "mean_us": 33.79368338658146, + "p50_us": 26.708, + "p95_us": 74.0625, + "p99_us": 99.8335, + "count": 1565.0 + }, + "remove_recreate": { + "mean_us": 7637.9640083333325, + "p50_us": 7690.728999999999, + "p95_us": 10505.792000000001, + "p99_us": 12810.208, + "count": 120.0 + }, + "rename_roundtrip": { + "mean_us": 7741.9994472361805, + "p50_us": 7812.0, + "p95_us": 10496.729, + "p99_us": 13576.1875, + "count": 199.0 + }, + "snapshot": { + "mean_us": 435.02945714285715, + "p50_us": 426.4375, + "p95_us": 738.75, + "p99_us": 821.646, + "count": 70.0 + }, + "stat": { + "mean_us": 33.60180813953488, + "p50_us": 26.0205, + "p95_us": 75.9165, + "p99_us": 105.5625, + "count": 602.0 + } + } + }, + "holt_vs_sqlite_throughput_ratio": 1.0632744196237267 + } + }, + "native-checkpoint": { + "content_async_4_background_normal": { + "holt": { + "runs": 4, + "ops_per_second": 1754.4800984416083, + "ops_min": 1683.7979667462666, + "ops_max": 1928.5334907016663, + "p50_us": 1042.375, + "p95_us": 4682.541499999999, + "p99_us": 8538.9375, + "checkpoint_ms": 53.1238125, + "reopen_ms": 27.784520999999998, + "drain_ops_per_second": 1709.6047723617023, + "metadata_allocated_mib": 1046.787109375, + "metadata_logical_mib": 1050.4798803329468, + "view_ms": 4260.059589, + "apply_ms": 588.675039, + "object_ms": 3280.194167, + "commits": 1403.0, + "cas_retries": 0, + "max_rss_mib": 385.0390625, + "per_operation": { + "list": { + "mean_us": 2157.9096077943614, + "p50_us": 1190.125, + "p95_us": 7721.0625, + "p99_us": 11942.125, + "count": 603.0 + }, + "publish": { + "mean_us": 2050.7807909783987, + "p50_us": 1734.146, + "p95_us": 4608.166499999999, + "p99_us": 6426.063, + "count": 787.0 + }, + "read": { + "mean_us": 2387.2896466625843, + "p50_us": 935.0835, + "p95_us": 2819.833, + "p99_us": 4981.75, + "count": 1633.0 + }, + "remove_recreate": { + "mean_us": 3190.49218, + "p50_us": 2978.5415000000003, + "p95_us": 6540.0415, + "p99_us": 9021.375, + "count": 125.0 + }, + "rename_roundtrip": { + "mean_us": 1943.0019453551913, + "p50_us": 1499.2915, + "p95_us": 5123.1665, + "p99_us": 6971.2080000000005, + "count": 183.0 + }, + "snapshot": { + "mean_us": 3252.8852957746476, + "p50_us": 2139.4790000000003, + "p95_us": 8887.4585, + "p99_us": 12462.437, + "count": 71.0 + }, + "stat": { + "mean_us": 791.3836847826087, + "p50_us": 598.396, + "p95_us": 2296.0209999999997, + "p99_us": 3381.3755, + "count": 598.0 + } + } + }, + "sqlite": { + "runs": 4, + "ops_per_second": 1679.9562386577127, + "ops_min": 1475.3684246506668, + "ops_max": 1836.3016988087927, + "p50_us": 390.2495, + "p95_us": 7526.1044999999995, + "p99_us": 23516.146, + "checkpoint_ms": 10.809750000000001, + "reopen_ms": 2.0665835, + "drain_ops_per_second": 1672.2564622188293, + "metadata_allocated_mib": 3.08984375, + "metadata_logical_mib": 2.654296875, + "view_ms": 169.3161965, + "apply_ms": 4703.1220865, + "object_ms": 3179.7045935, + "commits": 1403.0, + "cas_retries": 0, + "max_rss_mib": 11.4140625, + "per_operation": { + "list": { + "mean_us": 141.46143946932006, + "p50_us": 115.479, + "p95_us": 299.5415, + "p99_us": 474.58299999999997, + "count": 603.0 + }, + "publish": { + "mean_us": 5219.821047649301, + "p50_us": 2992.9790000000003, + "p95_us": 12729.3335, + "p99_us": 42389.292, + "count": 787.0 + }, + "read": { + "mean_us": 1341.2829874464176, + "p50_us": 466.271, + "p95_us": 1231.354, + "p99_us": 3486.9790000000003, + "count": 1633.0 + }, + "remove_recreate": { + "mean_us": 7342.937796, + "p50_us": 4949.3125, + "p95_us": 21493.7705, + "p99_us": 54621.1875, + "count": 125.0 + }, + "rename_roundtrip": { + "mean_us": 4384.295076502733, + "p50_us": 620.896, + "p95_us": 16128.9165, + "p99_us": 72864.3755, + "count": 183.0 + }, + "snapshot": { + "mean_us": 282.8271830985915, + "p50_us": 226.7295, + "p95_us": 684.5205, + "p99_us": 1170.0625, + "count": 71.0 + }, + "stat": { + "mean_us": 27.92623244147157, + "p50_us": 16.8745, + "p95_us": 66.9375, + "p99_us": 164.7295, + "count": 598.0 + } + } + }, + "holt_vs_sqlite_throughput_ratio": 1.0443605958709021 + }, + "metadata_async_4_background_normal": { + "holt": { + "runs": 4, + "ops_per_second": 1974.5395382043198, + "ops_min": 1920.5376353056274, + "ops_max": 1993.604061634945, + "p50_us": 1103.708, + "p95_us": 7389.875, + "p99_us": 14533.0415, + "checkpoint_ms": 179.265917, + "reopen_ms": 24.8242495, + "drain_ops_per_second": 1805.913070500968, + "metadata_allocated_mib": 1420.734375, + "metadata_logical_mib": 1468.1466889381409, + "view_ms": 7282.135564, + "apply_ms": 753.904317, + "object_ms": 0.0, + "commits": 1403.0, + "cas_retries": 0, + "max_rss_mib": 453.8125, + "per_operation": { + "list": { + "mean_us": 5557.552982587064, + "p50_us": 4303.5835, + "p95_us": 15246.7915, + "p99_us": 23614.146, + "count": 603.0 + }, + "publish": { + "mean_us": 1405.8393430749682, + "p50_us": 1073.0625, + "p95_us": 3687.708, + "p99_us": 6148.0205, + "count": 787.0 + }, + "read": { + "mean_us": 1045.1426163502756, + "p50_us": 830.75, + "p95_us": 2525.4165000000003, + "p99_us": 4432.6875, + "count": 1633.0 + }, + "remove_recreate": { + "mean_us": 2711.360348, + "p50_us": 2175.75, + "p95_us": 6795.0, + "p99_us": 9347.875, + "count": 125.0 + }, + "rename_roundtrip": { + "mean_us": 2766.1117704918033, + "p50_us": 2376.4795, + "p95_us": 6032.9375, + "p99_us": 9293.0835, + "count": 183.0 + }, + "snapshot": { + "mean_us": 5671.207133802817, + "p50_us": 4357.833500000001, + "p95_us": 13950.0415, + "p99_us": 20670.2085, + "count": 71.0 + }, + "stat": { + "mean_us": 1064.697431438127, + "p50_us": 830.646, + "p95_us": 2734.2915000000003, + "p99_us": 5013.1875, + "count": 598.0 + } + } + }, + "sqlite": { + "runs": 4, + "ops_per_second": 1557.7701188117285, + "ops_min": 1254.0106953531376, + "ops_max": 1789.9151262207001, + "p50_us": 60.3125, + "p95_us": 9264.6455, + "p99_us": 16880.042, + "checkpoint_ms": 8.847854, + "reopen_ms": 5.084604, + "drain_ops_per_second": 1553.2338859113697, + "metadata_allocated_mib": 2.791015625, + "metadata_logical_mib": 2.517578125, + "view_ms": 204.043739, + "apply_ms": 9574.682860500001, + "object_ms": 0.0, + "commits": 1403.0, + "cas_retries": 0, + "max_rss_mib": 11.1484375, + "per_operation": { + "list": { + "mean_us": 184.5990232172471, + "p50_us": 133.77100000000002, + "p95_us": 444.375, + "p99_us": 629.271, + "count": 603.0 + }, + "publish": { + "mean_us": 8202.128750952987, + "p50_us": 3437.292, + "p95_us": 11183.083, + "p99_us": 102764.95850000001, + "count": 787.0 + }, + "read": { + "mean_us": 30.608764237599512, + "p50_us": 14.7295, + "p95_us": 115.479, + "p99_us": 209.1875, + "count": 1633.0 + }, + "remove_recreate": { + "mean_us": 10109.076519999999, + "p50_us": 8184.3955, + "p95_us": 19128.561999999998, + "p99_us": 90948.7495, + "count": 125.0 + }, + "rename_roundtrip": { + "mean_us": 11510.051546448089, + "p50_us": 6318.041499999999, + "p95_us": 18385.396, + "p99_us": 164407.333, + "count": 183.0 + }, + "snapshot": { + "mean_us": 436.72095070422534, + "p50_us": 382.22950000000003, + "p95_us": 952.5835, + "p99_us": 1285.4375, + "count": 71.0 + }, + "stat": { + "mean_us": 31.34108277591973, + "p50_us": 14.062999999999999, + "p95_us": 109.0835, + "p99_us": 222.3955, + "count": 598.0 + } + } + }, + "holt_vs_sqlite_throughput_ratio": 1.2675423121548282 + } + } + } +} diff --git a/benches/agent_workspace/run_ab.py b/benches/agent_workspace/run_ab.py new file mode 100644 index 0000000..a7863b7 --- /dev/null +++ b/benches/agent_workspace/run_ab.py @@ -0,0 +1,156 @@ +#!/usr/bin/env python3 +"""Balanced baseline / patched / SQLite runs of the unchanged workspace driver.""" + +import argparse +import hashlib +import json +from pathlib import Path +import platform +import re +import shutil +import statistics +import subprocess +import sys +import time + +sys.dont_write_bytecode = True +from run_comparison import CASES, capture, domain_fingerprint, environment as base_environment + + +def environment(): + value = base_environment() + value["disk_usage_bytes"] = shutil.disk_usage(Path(__file__).resolve())._asdict() + return value + + +def report(out, rows): + summary = {} + for case in sorted({r["case"] for r in rows}): + summary[case] = {} + for variant in ["baseline", "patched", "sqlite"]: + selected = [r for r in rows if r["case"] == case and r["variant"] == variant] + if not selected: + continue + median = lambda f: statistics.median(f(r) for r in selected) + summary[case][variant] = { + "runs": len(selected), + "ops_per_second": median(lambda r: r["ops_per_second"]), + "ops_range": [min(r["ops_per_second"] for r in selected), max(r["ops_per_second"] for r in selected)], + "drain_ops_per_second": median(lambda r: r["drain_inclusive_ops_per_second"]), + "p50_us": median(lambda r: r["latency"]["p50_us"]), + "p95_us": median(lambda r: r["latency"]["p95_us"]), + "p99_us": median(lambda r: r["latency"]["p99_us"]), + "rss_mib": median(lambda r: r["maximum_resident_set_bytes"] / 2**20), + "metadata_mib": median(lambda r: r["space_after"]["metadata_allocated"] / 2**20), + "checkpoint_ms": median(lambda r: r["checkpoint_ms"]), + "view_ms": median(lambda r: r["stages"]["view_ms"]), + "apply_ms": median(lambda r: r["stages"]["apply_ms"]), + "cas_retries": sum(r["stages"]["cas_retries"] for r in selected), + "orphan_backlog": median(lambda r: (r["holt_stats"] or {}).get("orphan_backlog", 0)), + "live_slots": median(lambda r: (r["holt_stats"] or {}).get("live_slots", 0)), + "per_operation": { + op: {metric: median(lambda r: r["per_operation"][op][metric]) for metric in ["p50_us", "p95_us", "p99_us", "mean_us"]} + for op in selected[0]["per_operation"] + }, + } + (out / "summary.json").write_text(json.dumps(summary, indent=2) + "\n") + lines = [ + "# Agent workspace: baseline / patched Holt / SQLite", + "", + "Unchanged shared workspace handler, workload, dataset, consistency, durability and checkpoint policies. Four independent processes per variant in balanced order; medians of per-process metrics. No concurrent compiler/tests/benchmarks during these runs. Each database is fresh and removed only after full domain fingerprint and clean-reopen validation. Raw samples, commands, resource logs and environment remain alongside this report.", + "", + "Async OFF is diagnostic, not crash-equivalent to Holt. Background `_normal` uses native checkpoint policies (Holt 500 ms/512 blobs, SQLite NORMAL/1000 pages), not matched checkpoint cadence. Sync uses Holt sync=true and SQLite FULL with macOS fullfsync/checkpoint_fullfsync. These are bounded desktop workloads, not long-running maintenance or power-cut qualification.", + "", + "| Case | Old H ops/s | New H ops/s | SQLite ops/s | New/old | New/SQLite | Old/new p95 us |", + "|---|---:|---:|---:|---:|---:|---:|", + ] + for case, variants in summary.items(): + if len(variants) != 3: + continue + a, b, s = (variants[k] for k in ["baseline", "patched", "sqlite"]) + lines.append(f"| {case} | {a['ops_per_second']:,.0f} | {b['ops_per_second']:,.0f} | {s['ops_per_second']:,.0f} | {b['ops_per_second']/a['ops_per_second']:.2f}x | {b['ops_per_second']/s['ops_per_second']:.2f}x | {a['p95_us']:.1f} / {b['p95_us']:.1f} |") + lines += ["", "| Case | Old/new RSS MiB | Old/new metadata MiB | Old/new final checkpoint ms | Old/new post-drain orphan debt |", "|---|---:|---:|---:|---:|"] + for case, variants in summary.items(): + if "baseline" not in variants or "patched" not in variants: + continue + a, b = variants["baseline"], variants["patched"] + lines.append(f"| {case} | {a['rss_mib']:.1f} / {b['rss_mib']:.1f} | {a['metadata_mib']:.1f} / {b['metadata_mib']:.1f} | {a['checkpoint_ms']:.1f} / {b['checkpoint_ms']:.1f} | {a['orphan_backlog']:.0f} / {b['orphan_backlog']:.0f} |") + (out / "RESULTS.md").write_text("\n".join(lines) + "\n") + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--baseline", type=Path, required=True) + parser.add_argument("--patched", type=Path, required=True) + parser.add_argument("--out", type=Path, required=True) + parser.add_argument("--cases", default=",".join(k for k in CASES if "point" not in k and ("background" not in k or k.endswith("normal")))) + parser.add_argument("--rounds", type=int, choices=[3, 6, 12], default=12) + args = parser.parse_args() + out = args.out.resolve() + out.mkdir(parents=True, exist_ok=False) + selected = args.cases.split(",") + if any(k not in CASES for k in selected): + parser.error("unknown case") + binaries = {"baseline": args.baseline.resolve(), "patched": args.patched.resolve(), "sqlite": args.patched.resolve()} + evidence = { + "commit": capture(["git", "rev-parse", "HEAD"]), + "diff": capture(["git", "diff", "--", "src", "tests", "ARCHITECTURE.md"]), + "status": capture(["git", "status", "--short"]), + "rust": capture(["rustc", "-Vv"]), + "binary_sha256": {k: hashlib.sha256(v.read_bytes()).hexdigest() for k, v in binaries.items()}, + "source_sha256": {name: hashlib.sha256((Path(__file__).resolve().parent / name).read_bytes()).hexdigest() for name in ["Cargo.toml", "Cargo.lock", "src/main.rs", "run_ab.py", "run_comparison.py"]}, + "initial_environment": environment(), + "cases": {k: CASES[k] for k in selected}, + } + (out / "environment.json").write_text(json.dumps(evidence, indent=2) + "\n") + # Forward/reverse matched triplets: each pair appears in each order twice. + # Four processes per variant, twelve total. + sequence = ["baseline", "patched", "sqlite", "sqlite", "patched", "baseline", "patched", "baseline", "sqlite", "sqlite", "baseline", "patched"][:args.rounds] + rows = [] + for case in selected: + expected = None + for number, variant in enumerate(sequence, 1): + label = f"{case}-{number:02}-{variant}" + directory = out / "stores" / label + directory.parent.mkdir(exist_ok=True) + command = [str(binaries[variant]), "--engine", "sqlite" if variant == "sqlite" else "holt", "--dir", str(directory), "--warmup", "200", "--workspaces", "8", "--files", "512"] + for key, value in CASES[case].items(): + command += [f"--{key}", str(value).lower()] + before = environment() + started = time.monotonic() + (out / f"{label}.command.json").write_text(json.dumps(command, indent=2) + "\n") + process = subprocess.run(["/usr/bin/time", "-l" if platform.system() == "Darwin" else "-v", *command], text=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) + (out / f"{label}.stdout.json").write_text(process.stdout) + (out / f"{label}.stderr.txt").write_text(process.stderr) + (out / f"{label}.environment.json").write_text(json.dumps({"before": before, "after": environment(), "process_seconds": time.monotonic() - started}, indent=2) + "\n") + if process.returncode: + raise SystemExit(f"{label} failed: {process.returncode}\n{process.stderr}") + row = json.loads(process.stdout) + fingerprint = domain_fingerprint(row) + if expected is None: + expected = fingerprint + if expected != fingerprint or not row["verified_reopen"]: + raise SystemExit(f"fingerprint/reopen mismatch: {label}") + match = re.search(r"(\d+)\s+maximum resident set size", process.stderr) + if match: + rss = int(match.group(1)) + else: + match = re.search(r"Maximum resident set size \(kbytes\):\s*(\d+)", process.stderr) + if not match: + raise SystemExit(f"missing peak RSS: {label}") + rss = int(match.group(1)) * 1024 + row.update(case=case, round=number, variant=variant, maximum_resident_set_bytes=rss) + rows.append(row) + (out / "rows.json").write_text(json.dumps(rows, indent=2) + "\n") + assert directory.parent == out / "stores" and not directory.is_symlink() + shutil.rmtree(directory) + print(f"{label}: {row['ops_per_second']:.0f} ops/s, RSS {rss/2**20:.1f} MiB, cp {row['checkpoint_ms']:.1f} ms, CRC {fingerprint['crc32']}", flush=True) + report(out, rows) + evidence["final_environment"] = environment() + (out / "environment.json").write_text(json.dumps(evidence, indent=2) + "\n") + sums = [f"{hashlib.sha256(p.read_bytes()).hexdigest()} {p.name}" for p in sorted(out.iterdir()) if p.is_file() and p.name != "SHA256SUMS"] + (out / "SHA256SUMS").write_text("\n".join(sums) + "\n") + + +if __name__ == "__main__": + main() diff --git a/benches/agent_workspace/run_comparison.py b/benches/agent_workspace/run_comparison.py new file mode 100644 index 0000000..5936ce4 --- /dev/null +++ b/benches/agent_workspace/run_comparison.py @@ -0,0 +1,230 @@ +#!/usr/bin/env python3 +"""Serial, balanced runs of the shared workspace workflow; retain raw evidence.""" + +import argparse +import hashlib +import json +import os +from pathlib import Path +import platform +import re +import shutil +import statistics +import subprocess +import sys +import time +import tomllib + + +CASES = { + "metadata_async_1": {"ops": 12000, "threads": 1, "payload": 0, "sync": False}, + "metadata_async_4": {"ops": 12000, "threads": 4, "payload": 0, "sync": False}, + "metadata_sync_1": {"ops": 4000, "threads": 1, "payload": 0, "sync": True}, + "content_async_1": {"ops": 6000, "threads": 1, "payload": 4096, "sync": False}, + "content_async_4": {"ops": 6000, "threads": 4, "payload": 4096, "sync": False}, + "content_sync_1": {"ops": 1600, "threads": 1, "payload": 4096, "sync": True}, + "attached_metadata_sync_1": {"ops": 4000, "threads": 1, "payload": 0, "sync": True, "attached": True}, + "attached_content_sync_1": {"ops": 1600, "threads": 1, "payload": 4096, "sync": True, "attached": True}, + "metadata_point_async_1": {"ops": 12000, "threads": 1, "payload": 0, "sync": False, "read-mode": "point"}, + "metadata_async_4_short": {"ops": 4000, "threads": 4, "payload": 0, "sync": False}, + "metadata_async_4_background": {"ops": 4000, "threads": 4, "payload": 0, "sync": False, "background": True}, + "content_async_4_background": {"ops": 4000, "threads": 4, "payload": 4096, "sync": False, "background": True}, + "metadata_async_4_background_normal": {"ops": 4000, "threads": 4, "payload": 0, "sync": False, "background": True, "sqlite-normal": True}, + "content_async_4_background_normal": {"ops": 4000, "threads": 4, "payload": 4096, "sync": False, "background": True, "sqlite-normal": True}, +} + + +def capture(command): + try: + result = subprocess.run(command, text=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) + return {"command": command, "exit_code": result.returncode, "output": result.stdout} + except OSError as error: + return {"command": command, "exit_code": None, "output": str(error)} + + +def environment(): + return { + "utc": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()), + "platform": platform.platform(), + "cpu": capture(["/usr/sbin/sysctl", "-n", "machdep.cpu.brand_string", "hw.memsize", "hw.logicalcpu"]), + "load": os.getloadavg(), + "thermal": capture(["pmset", "-g", "therm"]), + "processes": capture(["ps", "-A", "-o", "pid,pcpu,comm"]), + } + + +def domain_fingerprint(row): + value = json.loads(json.dumps(row["fingerprint"])) + for table in value["tables"].values(): + table.pop("blobs", None) + return value + + +def summarize(rows): + summaries = {} + for name in sorted({row["case"] for row in rows}): + summary = {} + for engine in ["holt", "sqlite"]: + data = [row for row in rows if row["case"] == name and row["engine"] == engine] + median = lambda f: statistics.median(f(row) for row in data) + summary[engine] = { + "runs": len(data), + "ops_per_second": median(lambda r: r["ops_per_second"]), + "ops_min": min(r["ops_per_second"] for r in data), + "ops_max": max(r["ops_per_second"] for r in data), + "p50_us": median(lambda r: r["latency"]["p50_us"]), + "p95_us": median(lambda r: r["latency"]["p95_us"]), + "p99_us": median(lambda r: r["latency"]["p99_us"]), + "checkpoint_ms": median(lambda r: r["checkpoint_ms"]), + "reopen_ms": median(lambda r: r["reopen_ms"]), + "drain_ops_per_second": median(lambda r: r["drain_inclusive_ops_per_second"]), + "metadata_allocated_mib": median(lambda r: r["space_after"]["metadata_allocated"] / 2**20), + "metadata_logical_mib": median(lambda r: r["space_after"]["metadata_logical"] / 2**20), + "view_ms": median(lambda r: r["stages"]["view_ms"]), + "apply_ms": median(lambda r: r["stages"]["apply_ms"]), + "object_ms": median(lambda r: r["stages"]["object_ms"]), + "commits": median(lambda r: r["stages"]["metadata_commits"]), + "cas_retries": sum(r["stages"]["cas_retries"] for r in data), + "max_rss_mib": median(lambda r: r.get("maximum_resident_set_bytes", 0) / 2**20), + "per_operation": {}, + } + for operation in data[0]["per_operation"]: + summary[engine]["per_operation"][operation] = { + key: median(lambda r: r["per_operation"][operation][key]) + for key in ["mean_us", "p50_us", "p95_us", "p99_us", "count"] + } + summary["holt_vs_sqlite_throughput_ratio"] = summary["holt"]["ops_per_second"] / summary["sqlite"]["ops_per_second"] + summaries[name] = summary + return summaries + + +def write_report(out, rows, evidence): + for row in rows: + label = f"{row['case']}-{row['round']:02}-{row['engine']}" + resource = (out / f"{label}.stderr.txt").read_text() + match = re.search(r"(\d+)\s+maximum resident set size", resource) + if match: + row["maximum_resident_set_bytes"] = int(match.group(1)) + else: + match = re.search(r"Maximum resident set size \(kbytes\):\s*(\d+)", resource) + if match: + row["maximum_resident_set_bytes"] = int(match.group(1)) * 1024 + summaries = summarize(rows) + (out / "summary.json").write_text(json.dumps(summaries, indent=2) + "\n") + lines = [ + "# Holt / SQLite agent-workspace workflow comparison", + "", + "Application-level end-to-end boundary: JSON request encode/decode → shared workspace handler → metadata transaction/read view → optional local content file I/O + checksum → JSON response roundtrip. This is not a NoKV SDK/RPC or remote-object-provider benchmark.", + "", + f"Holt source: `{evidence['commit']}`; local library package {evidence.get('holt_version', '0.9.2')}. SQLite version is recorded in every raw row. All runs use the exact same optimized executable.", + "", + "Each run starts from 8 workspaces × 512 existing paths. Eight identically encoded metadata families map to Holt named trees and SQLite `WITHOUT ROWID` tables. Holt has a shared 64 MiB cache target; SQLite has a 64 MiB target per connection. These knobs are not a total-memory equivalence: measured SQLite RSS is below 20 MiB, while Holt can exceed its target through protected COW/dirty frames. Workload: 40% read, 15% stat, 15% list (64 records), 20% publish, 5% rename roundtrip, 3% remove+recreate, 2% consistent 8-family snapshot. Request choices are deterministic, so actual counts are reported per row.", + "", + "Single-client runs use one active workspace. Four clients use distinct workspaces in one shared DB; they still share metadata families. Rename and remove+recreate contain two metadata commits. Publish updates all eight families. Immutable content and history/receipts are retained. Setup and warmup are excluded; final checkpoint is reported separately and also included in a second throughput metric.", + "", + "Default read semantics: Holt `DB::view` and SQLite read transactions. Diagnostic `point` rows weaken only read/stat cross-tree consistency and are excluded from the main apples-to-apples result. Base cases disable background checkpoints and use an explicit final drain. Cases marked `background` enable Holt's native 500 ms/512-dirty-blob policy and SQLite's native 1000-page WAL autocheckpoint. Case `_normal` uses SQLite WAL synchronous=NORMAL; other async cases use OFF. These policies do not imply identical checkpoint cadence or bytes per round. These are bounded workload measurements, not a long-running maintenance qualification.", + "", + "Async diagnostic: Holt `Wal { sync:false }`, SQLite WAL `synchronous=OFF`; acknowledgements and crash consistency differ, so OFF is not a production-equivalent durability claim. Background `_normal` rows use SQLite WAL NORMAL (sync at checkpoint/log reuse) and Holt async WAL with background durable checkpoints; they still have different buffering and checkpoint policies. Sync: Holt `sync:true`, SQLite `synchronous=FULL`, **`fullfsync=ON` and `checkpoint_fullfsync=ON`** on macOS, matching Rust's F_FULLFSYNC path. Content writes in sync mode sync both the file and its parent directory for both engines. Final drains use durable checkpoint settings in every profile. No power-cut qualification was performed.", + "", + "Attached rows add the same canonical mutation bytes to Holt's attached WAL envelope and SQLite's transactional recovery table. A final durable checkpoint retires this local recovery material. The profile measures one ordered recovery funnel and does not claim shared-log or indefinite retention semantics.", + "", + "Every run verifies exact replay without another commit and checks the full ordered-domain fingerprint before and after reopen. Every engine and repetition within a case must produce identical domain fingerprints. Hardware/OS/load and raw command lines are preserved beside this report. Desktop background load limits portable claims about absolute timings.", + "", + "Reported values are medians across independent process runs. The balanced order is H,S,S,H / S,H,H,S; full per-run ranges remain in summary.json. p95/p99 are medians of per-run quantiles, not pooled quantiles.", + "", + "| Case | H ops/s | S ops/s | H/S | H p50 µs | S p50 µs | H p95 µs | S p95 µs |", + "|---|---:|---:|---:|---:|---:|---:|---:|", + ] + for name, summary in summaries.items(): + h, s = summary["holt"], summary["sqlite"] + lines.append(f"| {name} | {h['ops_per_second']:,.0f} | {s['ops_per_second']:,.0f} | {summary['holt_vs_sqlite_throughput_ratio']:.2f}× | {h['p50_us']:.1f} | {s['p50_us']:.1f} | {h['p95_us']:.1f} | {s['p95_us']:.1f} |") + lines += ["", "## Checkpoint and final metadata footprint", "", "Content bytes are excluded from this table. No vacuum is run. Both logical bytes and allocated filesystem blocks are retained in the raw data.", "", "| Case | H checkpoint ms | S checkpoint ms | H metadata allocated MiB | S metadata allocated MiB | H peak RSS MiB | S peak RSS MiB | H drain-inclusive ops/s | S drain-inclusive ops/s |", "|---|---:|---:|---:|---:|---:|---:|---:|---:|"] + for name, summary in summaries.items(): + h, s = summary["holt"], summary["sqlite"] + lines.append(f"| {name} | {h['checkpoint_ms']:.1f} | {s['checkpoint_ms']:.1f} | {h['metadata_allocated_mib']:.2f} | {s['metadata_allocated_mib']:.2f} | {h['max_rss_mib']:.1f} | {s['max_rss_mib']:.1f} | {h['drain_ops_per_second']:,.0f} | {s['drain_ops_per_second']:,.0f} |") + for name, summary in summaries.items(): + lines += ["", f"## {name}: operation latency", "", "| Operation | H p50 µs | S p50 µs | H p95 µs | S p95 µs |", "|---|---:|---:|---:|---:|"] + h, s = summary["holt"], summary["sqlite"] + for operation in h["per_operation"]: + a, b = h["per_operation"][operation], s["per_operation"][operation] + lines.append(f"| {operation} | {a['p50_us']:.1f} | {b['p50_us']:.1f} | {a['p95_us']:.1f} | {b['p95_us']:.1f} |") + lines += ["", f"Stage totals (median, sums across clients): H view+reads {h['view_ms']:.1f} ms, apply {h['apply_ms']:.1f} ms, objects {h['object_ms']:.1f} ms; S view+reads {s['view_ms']:.1f} ms, apply {s['apply_ms']:.1f} ms, objects {s['object_ms']:.1f} ms. These are inclusive stage timers, not pure lock-wait or snapshot-copy timings. Metadata commits: H {h['commits']}, S {s['commits']}; CAS retries H {h['cas_retries']}, S {s['cas_retries']}."] + (out / "RESULTS.md").write_text("\n".join(lines) + "\n") + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--binary", type=Path, required=True) + parser.add_argument("--out", type=Path, required=True) + parser.add_argument("--cases", default=",".join(CASES)) + parser.add_argument("--rounds", type=int, choices=[2, 4, 8], default=8) + parser.add_argument("--keep-stores", action="store_true", help="Retain generated test databases; full suites can use tens of GiB") + args = parser.parse_args() + out = args.out.resolve() + out.mkdir(parents=True, exist_ok=True) + selected = args.cases.split(",") + for name in selected: + if name not in CASES: + parser.error(f"unknown case {name}") + evidence = { + "commit": subprocess.check_output(["git", "rev-parse", "HEAD"], text=True).strip(), + "git_status": subprocess.check_output(["git", "status", "--short"], text=True), + "rust": capture(["rustc", "--version"]), + "binary_sha256": hashlib.sha256(args.binary.read_bytes()).hexdigest(), + "holt_version": tomllib.loads((Path(__file__).resolve().parents[2] / "Cargo.toml").read_text())["package"]["version"], + "source_sha256": { + name: hashlib.sha256((Path(__file__).resolve().parent / name).read_bytes()).hexdigest() + for name in ["Cargo.toml", "Cargo.lock", "src/main.rs", "run_comparison.py"] + }, + "cases": {name: CASES[name] for name in selected}, + "initial_environment": environment(), + } + (out / "environment.json").write_text(json.dumps(evidence, indent=2) + "\n") + sequence = ["holt", "sqlite", "sqlite", "holt", "sqlite", "holt", "holt", "sqlite"][:args.rounds] + rows = [] + for case in selected: + expected = None + for number, engine in enumerate(sequence, 1): + label = f"{case}-{number:02}-{engine}" + directory = out / "stores" / label + directory.parent.mkdir(exist_ok=True) + command = [str(args.binary.resolve()), "--engine", engine, "--dir", str(directory), "--warmup", "200", "--workspaces", "8", "--files", "512"] + for key, value in CASES[case].items(): + command += [f"--{key}", str(value).lower()] + (out / f"{label}.command.json").write_text(json.dumps(command, indent=2) + "\n") + before = environment() + started = time.monotonic() + resource_flag = "-l" if platform.system() == "Darwin" else "-v" + process = subprocess.run(["/usr/bin/time", resource_flag, *command], text=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) + (out / f"{label}.stdout.json").write_text(process.stdout) + (out / f"{label}.stderr.txt").write_text(process.stderr) + (out / f"{label}.environment.json").write_text(json.dumps({"before": before, "after": environment(), "process_seconds": time.monotonic() - started}, indent=2) + "\n") + if process.returncode: + print(process.stderr, file=sys.stderr) + raise SystemExit(f"{label} failed: {process.returncode}") + row = json.loads(process.stdout) + fingerprint = domain_fingerprint(row) + if expected is None: + expected = fingerprint + if expected != fingerprint: + raise SystemExit(f"domain state mismatch: {label}") + row["case"], row["round"] = case, number + rows.append(row) + (out / "rows.json").write_text(json.dumps(rows, indent=2) + "\n") + if not args.keep_stores: + assert directory.parent == out / "stores" and not directory.is_symlink() + shutil.rmtree(directory) + print(f"{label}: {row['ops_per_second']:.0f} ops/s, p50={row['latency']['p50_us']:.1f}us p95={row['latency']['p95_us']:.1f}us, cp={row['checkpoint_ms']:.1f}ms, CRC={fingerprint['crc32']}", flush=True) + write_report(out, rows, evidence) + evidence["final_environment"] = environment() + (out / "environment.json").write_text(json.dumps(evidence, indent=2) + "\n") + sums = [] + for path in sorted(out.iterdir()): + if path.is_file() and path.name != "SHA256SUMS": + sums.append(f"{hashlib.sha256(path.read_bytes()).hexdigest()} {path.name}") + (out / "SHA256SUMS").write_text("\n".join(sums) + "\n") + print(out / "RESULTS.md", flush=True) + + +if __name__ == "__main__": + main() diff --git a/benches/agent_workspace/src/main.rs b/benches/agent_workspace/src/main.rs new file mode 100644 index 0000000..58670da --- /dev/null +++ b/benches/agent_workspace/src/main.rs @@ -0,0 +1,1165 @@ +//! Application-level workspace workflow, with a shared handler and two stores. +//! This measures local content + metadata + JSON, not the NoKV SDK/RPC server. + +use std::collections::BTreeMap; +use std::error::Error; +use std::fs::{self, File, OpenOptions}; +use std::io::Write; +use std::os::unix::fs::MetadataExt; +use std::path::{Path, PathBuf}; +use std::sync::atomic::{AtomicU64, Ordering}; +use std::sync::{mpsc, Arc, Barrier, Mutex}; +use std::time::{Duration, Instant}; + +use holt::{ + DBView, Durability, JournalAnchor, JournalEnvelope, RangeEntry, RecordVersion, TreeConfig, DB, +}; +use rusqlite::{params, Connection, OptionalExtension}; +use serde::{Deserialize, Serialize}; +use serde_json::{json, Value}; + +type Result = std::result::Result>; +type Pair = (Vec, Vec); +const TABLES: [&str; 8] = [ + "workspace", + "path", + "revision", + "object_ref", + "operation", + "dedupe", + "history", + "event", +]; +const WS: usize = 0; +const PATHS: usize = 1; +const REV: usize = 2; +const REFS: usize = 3; +const OPS: usize = 4; +const DEDUPE: usize = 5; +const HISTORY: usize = 6; +const EVENTS: usize = 7; + +#[derive(Clone, Debug)] +struct Config { + engine: String, + directory: PathBuf, + sync: bool, + attached: bool, + threads: usize, + operations: usize, + warmup: usize, + workspaces: usize, + files: usize, + payload: usize, + seed: u64, + read_mode: String, + background: bool, + sqlite_normal: bool, +} + +impl Config { + fn parse() -> Result { + let mut args = std::env::args().skip(1); + let mut values = BTreeMap::new(); + while let Some(key) = args.next() { + let value = args.next().ok_or("every argument needs a value")?; + values.insert(key, value); + } + let text = |key: &str, default: &str| { + values + .get(key) + .cloned() + .unwrap_or_else(|| default.to_owned()) + }; + let cfg = Self { + engine: text("--engine", "holt"), + directory: PathBuf::from(values.get("--dir").ok_or("--dir is required")?), + sync: text("--sync", "false").parse()?, + attached: text("--attached", "false").parse()?, + threads: text("--threads", "1").parse()?, + operations: text("--ops", "4000").parse()?, + warmup: text("--warmup", "200").parse()?, + workspaces: text("--workspaces", "8").parse()?, + files: text("--files", "512").parse()?, + payload: text("--payload", "4096").parse()?, + seed: text("--seed", "456789").parse()?, + read_mode: text("--read-mode", "view").to_owned(), + background: text("--background", "false").parse()?, + sqlite_normal: text("--sqlite-normal", "false").parse()?, + }; + assert!(matches!(cfg.engine.as_str(), "holt" | "sqlite")); + assert!(matches!(cfg.read_mode.as_str(), "view" | "point")); + assert!(cfg.threads > 0 && cfg.threads <= cfg.workspaces && cfg.files >= 64); + assert!(cfg.operations >= cfg.threads); + Ok(cfg) + } +} + +#[derive(Clone, Serialize, Deserialize)] +struct Marker { + generation: u64, +} +#[derive(Clone, Serialize, Deserialize)] +struct PathMeta { + revision: String, + size: usize, + crc: u32, + generation: u64, +} +#[derive(Clone, Serialize, Deserialize)] +struct Revision { + object: String, + size: usize, + crc: u32, +} +#[derive(Clone, Serialize, Deserialize)] +struct Receipt { + request_crc: u32, + response: Response, +} + +#[derive(Clone, Serialize, Deserialize)] +struct Request { + op: String, + workspace: String, + path: String, + id: String, + content: String, +} + +#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)] +struct Response { + generation: u64, + size: usize, + crc: u32, + items: usize, + content: String, +} + +#[derive(Serialize)] +struct Guard { + table: usize, + key: Vec, + version: Option, +} +#[derive(Serialize)] +struct Change { + table: usize, + key: Vec, + value: Option>, +} +#[derive(Default, Serialize)] +struct Mutation { + guards: Vec, + changes: Vec, +} + +impl Mutation { + fn guard(&mut self, table: usize, key: &[u8], row: Option<&Row>) { + self.guards.push(Guard { + table, + key: key.to_vec(), + version: row.map(|r| r.version), + }); + } + fn put(&mut self, table: usize, key: &[u8], value: Vec) { + self.changes.push(Change { + table, + key: key.to_vec(), + value: Some(value), + }); + } + fn delete(&mut self, table: usize, key: &[u8]) { + self.changes.push(Change { + table, + key: key.to_vec(), + value: None, + }); + } +} + +struct Row { + value: Vec, + version: u64, +} + +trait Reader { + fn get(&self, table: usize, key: &[u8]) -> Result>; + fn list(&self, table: usize, prefix: &[u8], limit: usize) -> Result>; +} + +struct HoltReader<'a> { + view: &'a DBView, +} +impl Reader for HoltReader<'_> { + fn get(&self, table: usize, key: &[u8]) -> Result> { + Ok(self + .view + .tree(TABLES[table]) + .unwrap() + .get_record(key)? + .map(|r| Row { + value: r.value, + version: r.version.as_u64(), + })) + } + fn list(&self, table: usize, prefix: &[u8], limit: usize) -> Result> { + self.view + .tree(TABLES[table]) + .unwrap() + .scan(prefix)? + .into_iter() + .take(limit) + .map(|entry| match entry? { + RangeEntry::Key { key, value, .. } => Ok((key, value)), + _ => Err("unexpected delimiter entry".into()), + }) + .collect() + } +} + +struct SqlReader<'a> { + connection: &'a Connection, +} +impl Reader for SqlReader<'_> { + fn get(&self, table: usize, key: &[u8]) -> Result> { + Ok(self + .connection + .prepare_cached(&format!("SELECT v,ver FROM t_{} WHERE k=?1", TABLES[table]))? + .query_row([key], |row| { + Ok(Row { + value: row.get(0)?, + version: row.get::<_, i64>(1)? as u64, + }) + }) + .optional()?) + } + fn list(&self, table: usize, prefix: &[u8], limit: usize) -> Result> { + let mut upper = prefix.to_vec(); + let last = upper.last_mut().ok_or("empty prefix")?; + assert!(*last < 255); + *last += 1; + let mut stmt = self.connection.prepare_cached(&format!( + "SELECT k,v FROM t_{} WHERE k>=?1 AND k, _>>()?) + } +} + +struct Shared { + cfg: Config, + db: Option, + tail: Mutex, + version: AtomicU64, +} + +#[derive(Default)] +struct Counters { + view_ns: u64, + views: u64, + apply_ns: u64, + commits: u64, + object_ns: u64, + object_reads: u64, + object_writes: u64, + retries: u64, +} + +struct Session { + shared: Arc, + sqlite: Option, + trees: Vec, + counters: Counters, +} + +fn sql_open(cfg: &Config) -> Result { + let conn = Connection::open(cfg.directory.join("metadata/sqlite.db"))?; + conn.busy_timeout(Duration::from_secs(30))?; + conn.set_prepared_statement_cache_capacity(128); + conn.execute_batch(&format!("PRAGMA journal_mode=WAL; PRAGMA synchronous={}; PRAGMA fullfsync=ON; PRAGMA checkpoint_fullfsync=ON; PRAGMA cache_size=-65536; PRAGMA temp_store=MEMORY; PRAGMA wal_autocheckpoint={}; PRAGMA mmap_size=0;", if cfg.sync { "FULL" } else if cfg.sqlite_normal {"NORMAL"} else { "OFF" }, if cfg.background {1000} else {0}))?; + Ok(conn) +} + +impl Session { + fn open(shared: Arc) -> Result { + let (sqlite, trees) = match &shared.db { + Some(db) => ( + None, + TABLES + .iter() + .map(|name| db.open_tree(name)) + .collect::>>()?, + ), + None => (Some(sql_open(&shared.cfg)?), Vec::new()), + }; + Ok(Self { + shared, + sqlite, + trees, + counters: Counters::default(), + }) + } + fn current(&self, table: usize, key: &[u8]) -> Result> { + if let Some(connection) = &self.sqlite { + SqlReader { connection }.get(table, key) + } else { + Ok(self.trees[table].get_record(key)?.map(|r| Row { + value: r.value, + version: r.version.as_u64(), + })) + } + } + fn view( + &mut self, + tables: &[usize], + workspace: &str, + f: impl FnOnce(&dyn Reader) -> Result, + ) -> Result { + let started = Instant::now(); + let result = if let Some(conn) = &self.sqlite { + let tx = conn.unchecked_transaction()?; + let result = f(&SqlReader { connection: &tx })?; + tx.commit()?; + Ok(result) + } else { + let prefix = format!("{workspace}/"); + let scopes = tables + .iter() + .map(|&table| (TABLES[table], prefix.as_bytes())) + .collect::>(); + // The callback result is carried through Holt's native error type. + self.shared + .db + .as_ref() + .unwrap() + .view(&scopes, |view| Ok(f(&HoltReader { view })))? + }; + self.counters.views += 1; + self.counters.view_ns += started.elapsed().as_nanos() as u64; + result + } + fn apply(&mut self, mutation: &Mutation) -> Result { + let started = Instant::now(); + let canonical = if self.shared.cfg.attached { + Some(serde_json::to_vec(mutation)?) + } else { + None + }; + let committed = if let Some(connection) = &self.sqlite { + connection.execute_batch("BEGIN IMMEDIATE")?; + let result = (|| -> Result { + let reader = SqlReader { connection }; + for guard in &mutation.guards { + if reader.get(guard.table, &guard.key)?.map(|r| r.version) != guard.version { + return Ok(false); + } + } + let version = (self.shared.version.fetch_add(1, Ordering::Relaxed) + 1) as i64; + for change in &mutation.changes { + match &change.value { + Some(value) => { + connection.prepare_cached(&format!("INSERT INTO t_{}(k,v,ver) VALUES(?1,?2,?3) ON CONFLICT(k) DO UPDATE SET v=excluded.v,ver=excluded.ver", TABLES[change.table]))?.execute(params![change.key, value, version])?; + } + None => { + connection + .prepare_cached(&format!( + "DELETE FROM t_{} WHERE k=?1", + TABLES[change.table] + ))? + .execute([&change.key])?; + } + } + } + if let Some(canonical) = &canonical { + connection + .prepare_cached("INSERT INTO recovery_log(seq,payload) VALUES(?1,?2)")? + .execute(params![version, canonical])?; + } + Ok(true) + })(); + match result { + Ok(true) => { + connection.execute_batch("COMMIT")?; + true + } + Ok(false) => { + connection.execute_batch("ROLLBACK")?; + false + } + Err(error) => { + let _ = connection.execute_batch("ROLLBACK"); + return Err(error); + } + } + } else { + let db = self.shared.db.as_ref().unwrap(); + let build = |batch: &mut holt::DBAtomicBatch| { + for guard in &mutation.guards { + match guard.version { + Some(version) => batch.assert_version( + TABLES[guard.table], + &guard.key, + RecordVersion::from_raw(version), + ), + None => batch.assert_absent(TABLES[guard.table], &guard.key), + } + } + for change in &mutation.changes { + match &change.value { + Some(value) => batch.put(TABLES[change.table], &change.key, value), + None => batch.delete(TABLES[change.table], &change.key), + } + } + }; + if let Some(canonical) = canonical { + // One application recovery stream, with the same single-funnel + // admission as the NoKV profile. Ordinary mode has no funnel. + let mut tail = self.shared.tail.lock().unwrap(); + let mut digest = [0; 32]; + digest[..8].copy_from_slice(&(tail.sequence() + 1).to_le_bytes()); + digest[8..12].copy_from_slice(&crc32fast::hash(&canonical).to_le_bytes()); + let next = JournalAnchor::new(tail.sequence() + 1, digest); + let committed = db.atomic_with_journal_envelope( + JournalEnvelope::new(*tail, next, canonical)?, + build, + )?; + if committed { + *tail = next; + } + committed + } else { + db.atomic(build)? + } + }; + self.counters.apply_ns += started.elapsed().as_nanos() as u64; + if committed { + self.counters.commits += 1; + } else { + self.counters.retries += 1; + } + Ok(committed) + } +} + +fn workspace_key(ws: &str) -> Vec { + format!("{ws}/marker").into_bytes() +} +fn file_path(ws: &str, file: usize) -> String { + format!("{ws}/src/dir-{:03}/file-{file:06}.txt", file / 64) +} +fn encode(value: &T) -> Result> { + Ok(serde_json::to_vec(value)?) +} +fn decode Deserialize<'a>>(bytes: &[u8]) -> Result { + Ok(serde_json::from_slice(bytes)?) +} + +fn write_object(session: &mut Session, name: &str, content: &str) -> Result<()> { + if session.shared.cfg.payload == 0 { + return Ok(()); + } + let started = Instant::now(); + let directory = session.shared.cfg.directory.join("objects"); + let path = directory.join(name); + match OpenOptions::new().write(true).create_new(true).open(path) { + Ok(mut file) => { + file.write_all(content.as_bytes())?; + if session.shared.cfg.sync { + file.sync_all()?; + File::open(&directory)?.sync_all()?; + } + } + Err(e) if e.kind() == std::io::ErrorKind::AlreadyExists => { + assert_eq!(fs::read(directory.join(name))?, content.as_bytes()); + } + Err(e) => return Err(e.into()), + } + session.counters.object_writes += 1; + session.counters.object_ns += started.elapsed().as_nanos() as u64; + Ok(()) +} + +fn mutate( + session: &mut Session, + request: &Request, + kind: &str, + from: &str, + to: &str, +) -> Result { + let request_crc = crc32fast::hash(&encode(&(request, kind, from, to))?); + let receipt_key = format!("{}/requests/{}-{kind}", request.workspace, request.id).into_bytes(); + if let Some(row) = session.current(DEDUPE, &receipt_key)? { + let receipt: Receipt = decode(&row.value)?; + assert_eq!(request_crc, receipt.request_crc, "request replay mismatch"); + return Ok(receipt.response); + } + if kind == "publish" { + write_object(session, &request.id, &request.content)?; + } + for _ in 0..32 { + let (marker_row, source, destination) = + session.view(&[WS, PATHS], &request.workspace, |reader| { + Ok(( + reader + .get(WS, &workspace_key(&request.workspace))? + .ok_or("workspace missing")?, + reader.get(PATHS, from.as_bytes())?, + if from != to { + reader.get(PATHS, to.as_bytes())? + } else { + None + }, + )) + })?; + let marker: Marker = decode(&marker_row.value)?; + let generation = marker.generation + 1; + let mut mutation = Mutation::default(); + mutation.guard(WS, &workspace_key(&request.workspace), Some(&marker_row)); + mutation.guard(PATHS, from.as_bytes(), source.as_ref()); + mutation.guard(DEDUPE, &receipt_key, None); + mutation.put( + WS, + &workspace_key(&request.workspace), + encode(&Marker { generation })?, + ); + let old = source.as_ref().map(|row| row.value.clone()); + let (size, crc) = match kind { + "publish" => { + let revision_key = format!("{}/revision/{}", request.workspace, request.id); + let meta = PathMeta { + revision: revision_key.clone(), + size: request.content.len(), + crc: crc32fast::hash(request.content.as_bytes()), + generation, + }; + mutation.guard(REV, revision_key.as_bytes(), None); + mutation.put(PATHS, to.as_bytes(), encode(&meta)?); + mutation.put( + REV, + revision_key.as_bytes(), + encode(&Revision { + object: request.id.clone(), + size: meta.size, + crc: meta.crc, + })?, + ); + mutation.put( + REFS, + revision_key.as_bytes(), + encode(&json!({"references": 1}))?, + ); + (meta.size, meta.crc) + } + "rename_out" | "rename_back" => { + assert!(destination.is_none()); + mutation.guard(PATHS, to.as_bytes(), None); + let mut meta: PathMeta = + decode(&source.as_ref().ok_or("rename source missing")?.value)?; + meta.generation = generation; + mutation.delete(PATHS, from.as_bytes()); + mutation.put(PATHS, to.as_bytes(), encode(&meta)?); + (meta.size, meta.crc) + } + "remove" => { + let meta: PathMeta = + decode(&source.as_ref().ok_or("remove source missing")?.value)?; + mutation.delete(PATHS, from.as_bytes()); + (meta.size, meta.crc) + } + _ => return Err("unknown mutation".into()), + }; + let response = Response { + generation, + size, + crc, + items: 1, + content: String::new(), + }; + let receipt = encode(&Receipt { + request_crc, + response: response.clone(), + })?; + mutation.put(OPS, &receipt_key, receipt.clone()); + mutation.put(DEDUPE, &receipt_key, receipt); + mutation.put( + HISTORY, + &receipt_key, + old.unwrap_or_else(|| b"null".to_vec()), + ); + mutation.put( + EVENTS, + &format!( + "{}/events/{generation:020}/{}-{kind}", + request.workspace, request.id + ) + .into_bytes(), + encode(&json!({"kind": kind, "path": to, "generation": generation}))?, + ); + if session.apply(&mutation)? { + return Ok(response); + } + } + Err("workspace CAS retry exhausted".into()) +} + +fn read_metadata(session: &mut Session, request: &Request) -> Result<(Marker, PathMeta, Revision)> { + if session.shared.cfg.read_mode == "point" { + // Diagnostic only: no cross-tree snapshot guarantee. The report keeps + // this separate from the default consistent-view workspace results. + let marker = session + .current(WS, &workspace_key(&request.workspace))? + .ok_or("marker missing")?; + let path = session + .current(PATHS, request.path.as_bytes())? + .ok_or("path missing")?; + let path: PathMeta = decode(&path.value)?; + let revision = session + .current(REV, path.revision.as_bytes())? + .ok_or("revision missing")?; + return Ok((decode(&marker.value)?, path, decode(&revision.value)?)); + } + session.view(&[WS, PATHS, REV], &request.workspace, |reader| { + let marker = reader + .get(WS, &workspace_key(&request.workspace))? + .ok_or("marker missing")?; + let path = reader + .get(PATHS, request.path.as_bytes())? + .ok_or("path missing")?; + let path: PathMeta = decode(&path.value)?; + let revision = reader + .get(REV, path.revision.as_bytes())? + .ok_or("revision missing")?; + Ok((decode(&marker.value)?, path, decode(&revision.value)?)) + }) +} + +fn handle(session: &mut Session, request: &Request) -> Result { + match request.op.as_str() { + "read" | "stat" => { + let (marker, path, revision) = read_metadata(session, request)?; + assert!(path.generation <= marker.generation); + assert_eq!((path.size, path.crc), (revision.size, revision.crc)); + let mut content = String::new(); + if request.op == "read" && session.shared.cfg.payload > 0 { + let started = Instant::now(); + let bytes = fs::read( + session + .shared + .cfg + .directory + .join("objects") + .join(revision.object), + )?; + assert_eq!( + (bytes.len(), crc32fast::hash(&bytes)), + (path.size, path.crc) + ); + content = String::from_utf8(bytes)?; + session.counters.object_reads += 1; + session.counters.object_ns += started.elapsed().as_nanos() as u64; + } + Ok(Response { + generation: marker.generation, + size: path.size, + crc: path.crc, + items: 1, + content, + }) + } + "list" => { + let prefix = format!("{}/", request.path.rsplit_once('/').unwrap().0); + session.view(&[WS, PATHS], &request.workspace, |reader| { + let marker: Marker = decode( + &reader + .get(WS, &workspace_key(&request.workspace))? + .unwrap() + .value, + )?; + let rows = reader.list(PATHS, prefix.as_bytes(), 64)?; + let mut crc = crc32fast::Hasher::new(); + for (key, value) in &rows { + crc.update(key); + crc.update(value); + } + let entries = rows + .iter() + .map(|(key, value)| { + Ok((String::from_utf8(key.clone())?, decode::(value)?)) + }) + .collect::>>()?; + Ok(Response { + generation: marker.generation, + size: 0, + crc: crc.finalize(), + items: rows.len(), + content: serde_json::to_string(&entries)?, + }) + }) + } + "publish" => mutate(session, request, "publish", &request.path, &request.path), + "rename_roundtrip" => { + let temporary = format!("{}.moving", request.path); + mutate(session, request, "rename_out", &request.path, &temporary)?; + mutate(session, request, "rename_back", &temporary, &request.path) + } + "remove_recreate" => { + mutate(session, request, "remove", &request.path, &request.path)?; + mutate(session, request, "publish", &request.path, &request.path) + } + "snapshot" => session.view(&(0..8).collect::>(), &request.workspace, |reader| { + let mut crc = crc32fast::Hasher::new(); + let mut items = 0; + for table in 0..8 { + for (key, value) in + reader.list(table, format!("{}/", request.workspace).as_bytes(), 32)? + { + crc.update(&key); + crc.update(&value); + items += 1; + } + } + let marker: Marker = decode( + &reader + .get(WS, &workspace_key(&request.workspace))? + .unwrap() + .value, + )?; + Ok(Response { + generation: marker.generation, + size: 0, + crc: crc.finalize(), + items, + content: String::new(), + }) + }), + _ => Err("unknown operation".into()), + } +} + +fn call(session: &mut Session, request: &Request) -> Result { + let encoded = encode(request)?; + let decoded: Request = decode(&encoded)?; + let response = handle(session, &decoded)?; + let result = decode::(&encode(&response)?)?; + assert_eq!(response, result); + Ok(result) +} + +fn contents(bytes: usize, token: usize) -> String { + let mut out = vec![b'a' + (token % 26) as u8; bytes]; + for index in (0..bytes).step_by(80) { + out[index] = b'\n'; + } + String::from_utf8(out).unwrap() +} + +fn preload(cfg: &Config) -> Result> { + fs::create_dir(&cfg.directory)?; + fs::create_dir(cfg.directory.join("metadata"))?; + fs::create_dir(cfg.directory.join("objects"))?; + let db = if cfg.engine == "holt" { + let mut config = TreeConfig::new(cfg.directory.join("metadata")); + config.durability = Durability::Wal { sync: false }; + config.buffer_pool_size = 128; + config.checkpoint.enabled = false; + let db = DB::open(config)?; + for table in TABLES { + db.create_tree(table)?; + } + Some(db) + } else { + None + }; + let connection = if db.is_none() { + let conn = sql_open(cfg)?; + for name in TABLES { + conn.execute_batch(&format!("CREATE TABLE t_{name}(k BLOB PRIMARY KEY,v BLOB NOT NULL,ver INTEGER NOT NULL) WITHOUT ROWID"))?; + } + conn.execute_batch( + "CREATE TABLE recovery_log(seq INTEGER PRIMARY KEY,payload BLOB NOT NULL)", + )?; + Some(conn) + } else { + None + }; + let mut rows = Vec::<(usize, Vec, Vec)>::new(); + for ws in 0..cfg.workspaces { + let name = format!("agent-{ws:03}"); + rows.push((WS, workspace_key(&name), encode(&Marker { generation: 0 })?)); + for file in 0..cfg.files { + let id = format!("initial-{ws:03}-{file:06}"); + let revision = format!("{name}/revision/{id}"); + let content = contents(cfg.payload, file); + let crc = crc32fast::hash(content.as_bytes()); + if cfg.payload > 0 { + fs::write(cfg.directory.join("objects").join(&id), &content)?; + } + rows.push(( + PATHS, + file_path(&name, file).into_bytes(), + encode(&PathMeta { + revision: revision.clone(), + size: cfg.payload, + crc, + generation: 0, + })?, + )); + rows.push(( + REV, + revision.as_bytes().to_vec(), + encode(&Revision { + object: id, + size: cfg.payload, + crc, + })?, + )); + rows.push(( + REFS, + revision.as_bytes().to_vec(), + encode(&json!({"references": 1}))?, + )); + } + } + if let Some(db) = &db { + for chunk in rows.chunks(256) { + assert!(db.atomic(|batch| { + for (table, key, value) in chunk { + batch.put(TABLES[*table], key, value); + } + })?); + } + db.checkpoint()?; + } else { + let conn = connection.as_ref().unwrap(); + conn.execute_batch("BEGIN IMMEDIATE")?; + for (table, key, value) in &rows { + conn.prepare_cached(&format!("INSERT INTO t_{} VALUES(?1,?2,0)", TABLES[*table]))? + .execute(params![key, value])?; + } + conn.execute_batch("COMMIT; PRAGMA wal_checkpoint(TRUNCATE)")?; + } + drop(connection); + // Reopen in the actual acknowledgement mode; setup is never timed. + drop(db); + let db = if cfg.engine == "holt" { + let mut config = TreeConfig::new(cfg.directory.join("metadata")); + config.durability = Durability::Wal { sync: cfg.sync }; + config.buffer_pool_size = 128; + config.checkpoint.enabled = cfg.background; + let db = DB::open(config)?; + if cfg.attached { + db.initialize_journal_stream(JournalAnchor::new(0, [0; 32]))?; + } + Some(db) + } else { + None + }; + Ok(Arc::new(Shared { + cfg: cfg.clone(), + db, + tail: Mutex::new(JournalAnchor::new(0, [0; 32])), + version: AtomicU64::new(0), + })) +} + +fn next_random(state: &mut u64) -> u64 { + *state ^= *state << 13; + *state ^= *state >> 7; + *state ^= *state << 17; + *state +} + +fn request(cfg: &Config, worker: usize, index: usize, phase: &str, random: &mut u64) -> Request { + let n = next_random(random); + let op = match n % 100 { + 0..=39 => "read", + 40..=54 => "stat", + 55..=69 => "list", + 70..=89 => "publish", + 90..=94 => "rename_roundtrip", + 95..=97 => "remove_recreate", + _ => "snapshot", + }; + let workspace = format!("agent-{worker:03}"); + Request { + op: op.to_owned(), + path: file_path(&workspace, ((n >> 8) as usize) % cfg.files), + workspace, + id: format!("{phase}-{worker:03}-{index:08}"), + content: if matches!(op, "publish" | "remove_recreate") { + contents(cfg.payload, index) + } else { + String::new() + }, + } +} + +fn latency(values: &mut [u64]) -> Value { + values.sort_unstable(); + let quantile = |p: f64| values[((values.len() - 1) as f64 * p).ceil() as usize] as f64 / 1000.0; + json!({"count": values.len(), "p50_us": quantile(0.5), "p95_us": quantile(0.95), "p99_us": quantile(0.99), "max_us": quantile(1.0), "mean_us": values.iter().sum::() as f64 / values.len() as f64 / 1000.0}) +} + +fn space(directory: &Path) -> Result { + let mut logical = 0; + let mut allocated = 0; + let mut files = serde_json::Map::new(); + fn walk(path: &Path, logical: &mut u64, allocated: &mut u64) -> Result<()> { + for entry in fs::read_dir(path)? { + let entry = entry?; + let meta = entry.metadata()?; + if meta.is_dir() { + walk(&entry.path(), logical, allocated)?; + } else { + *logical += meta.len(); + *allocated += meta.blocks() * 512; + } + } + Ok(()) + } + walk(directory, &mut logical, &mut allocated)?; + for entry in fs::read_dir(directory.join("metadata"))? { + let entry = entry?; + let meta = entry.metadata()?; + files.insert( + entry.file_name().to_string_lossy().into_owned(), + json!({"logical": meta.len(), "allocated": meta.blocks() * 512}), + ); + } + let metadata_logical = files + .values() + .map(|v| v["logical"].as_u64().unwrap()) + .sum::(); + let metadata_allocated = files + .values() + .map(|v| v["allocated"].as_u64().unwrap()) + .sum::(); + Ok(json!({"logical": logical, "allocated": allocated, + "metadata_logical": metadata_logical, "metadata_allocated": metadata_allocated, + "objects_logical": logical - metadata_logical, + "objects_allocated": allocated - metadata_allocated, "metadata_files": files})) +} + +fn fingerprint(session: &mut Session) -> Result { + let mut fingerprint = crc32fast::Hasher::new(); + let mut tables = serde_json::Map::new(); + for (table, name) in TABLES.iter().enumerate() { + let mut count = 0; + let mut bytes = 0; + if let Some(conn) = &session.sqlite { + let mut stmt = conn.prepare(&format!("SELECT k,v FROM t_{name} ORDER BY k"))?; + let pairs = stmt.query_map([], |row| { + Ok((row.get::<_, Vec>(0)?, row.get::<_, Vec>(1)?)) + })?; + for pair in pairs { + let (key, value) = pair?; + fingerprint.update(&key); + fingerprint.update(&value); + count += 1; + bytes += key.len() + value.len(); + } + tables.insert(name.to_string(), json!({"rows": count, "bytes": bytes})); + } else { + for entry in session.trees[table].range() { + if let RangeEntry::Key { key, value, .. } = entry? { + fingerprint.update(&key); + fingerprint.update(&value); + count += 1; + bytes += key.len() + value.len(); + } + } + let stats = session.trees[table].stats()?; + tables.insert( + name.to_string(), + json!({"rows": count, "bytes": bytes, "blobs": stats.blob_count}), + ); + } + } + Ok(json!({"crc32": fingerprint.finalize(), "tables": tables})) +} + +fn checkpoint(shared: &Shared, keeper: &Session) -> Result<()> { + if let Some(db) = &shared.db { + db.checkpoint()?; + } else { + let conn = keeper.sqlite.as_ref().unwrap(); + conn.execute_batch("PRAGMA synchronous=FULL; PRAGMA wal_checkpoint(TRUNCATE)")?; + if shared.cfg.attached { + // The same local, checkpoint-bounded recovery retention as Holt. + conn.execute_batch("DELETE FROM recovery_log; PRAGMA wal_checkpoint(TRUNCATE)")?; + } + if !shared.cfg.sync { + conn.execute_batch(if shared.cfg.sqlite_normal { + "PRAGMA synchronous=NORMAL" + } else { + "PRAGMA synchronous=OFF" + })?; + } + } + Ok(()) +} + +fn main() -> Result<()> { + let cfg = Config::parse()?; + let setup = Instant::now(); + let shared = preload(&cfg)?; + let mut keeper = Session::open(Arc::clone(&shared))?; + // Sanity: exact replay has one commit and returns the exact response. + let sanity = Request { + op: "publish".into(), + workspace: "agent-000".into(), + path: file_path("agent-000", 0), + id: "sanity".into(), + content: contents(cfg.payload, 7), + }; + let first = call(&mut keeper, &sanity)?; + let commits = keeper.counters.commits; + assert_eq!(first, call(&mut keeper, &sanity)?); + assert_eq!(commits, keeper.counters.commits); + checkpoint(&shared, &keeper)?; + let setup_seconds = setup.elapsed().as_secs_f64(); + let barrier = Arc::new(Barrier::new(cfg.threads + 1)); + let (ready_tx, ready_rx) = mpsc::channel(); + let mut workers = Vec::new(); + for worker in 0..cfg.threads { + let shared = Arc::clone(&shared); + let barrier = Arc::clone(&barrier); + let ready_tx = ready_tx.clone(); + workers.push(std::thread::spawn( + move || -> Result<(BTreeMap>, Counters)> { + let mut session = Session::open(Arc::clone(&shared))?; + let mut random = shared.cfg.seed ^ ((worker as u64 + 1) * 0x09e3_779b); + for index in 0..shared.cfg.warmup / shared.cfg.threads { + let req = request(&shared.cfg, worker, index, "warmup", &mut random); + call(&mut session, &req)?; + } + session.counters = Counters::default(); + ready_tx.send(())?; + barrier.wait(); + let mut samples: BTreeMap> = BTreeMap::new(); + for index in 0..shared.cfg.operations / shared.cfg.threads { + // JSON and all domain/storage/content work are inside timing; + // caller-side request generation is outside each latency sample. + let req = request(&shared.cfg, worker, index, "timed", &mut random); + let started = Instant::now(); + std::hint::black_box(call(&mut session, &req)?); + samples + .entry(req.op) + .or_default() + .push(started.elapsed().as_nanos() as u64); + } + Ok((samples, session.counters)) + }, + )); + } + for _ in 0..cfg.threads { + ready_rx.recv_timeout(Duration::from_secs(120))?; + } + checkpoint(&shared, &keeper)?; + let before = space(&cfg.directory)?; + let before_stats = shared.db.as_ref().map(DB::stats); + let started = Instant::now(); + barrier.wait(); + let mut samples: BTreeMap> = BTreeMap::new(); + let mut counters = Counters::default(); + for worker in workers { + let (worker_samples, worker_counters) = worker.join().map_err(|_| "worker panicked")??; + for (op, values) in worker_samples { + samples.entry(op).or_default().extend(values); + } + counters.view_ns += worker_counters.view_ns; + counters.views += worker_counters.views; + counters.apply_ns += worker_counters.apply_ns; + counters.commits += worker_counters.commits; + counters.object_ns += worker_counters.object_ns; + counters.object_reads += worker_counters.object_reads; + counters.object_writes += worker_counters.object_writes; + counters.retries += worker_counters.retries; + } + let elapsed = started.elapsed(); + let before_checkpoint = space(&cfg.directory)?; + let checkpoint_start = Instant::now(); + checkpoint(&shared, &keeper)?; + let checkpoint_elapsed = checkpoint_start.elapsed(); + let after = space(&cfg.directory)?; + let final_fingerprint = fingerprint(&mut keeper)?; + let stats = if let Some(db) = &shared.db { + let stats = db.stats(); + let old = before_stats.as_ref().unwrap(); + json!({ + "walker_ops": stats.bm_walker_ops - old.bm_walker_ops, + "walker_blob_hops": stats.bm_walker_blob_hops - old.bm_walker_blob_hops, + "spillovers": stats.bm_spillovers - old.bm_spillovers, + "live_slots": stats.store.live_slots, "next_slot": stats.store.next_slot, + "orphan_backlog": stats.bm_gc_orphan_backlog_count, + "wal_appends": stats.journal.as_ref().unwrap().appends - old.journal.as_ref().unwrap().appends, + "wal_syncs": stats.journal.as_ref().unwrap().syncs - old.journal.as_ref().unwrap().syncs, + }) + } else { + json!({}) + }; + let mut all = samples.values().flatten().copied().collect::>(); + let operations = all.len(); + let aggregate_latency = latency(&mut all); + let per_op = samples + .iter_mut() + .map(|(name, samples)| (name.clone(), latency(samples))) + .collect::>(); + // Clean reopen verifies persisted domain state for both engines. + drop(keeper); + let shared = Arc::try_unwrap(shared).ok().ok_or("shared handle leaked")?; + drop(shared); + let reopen_start = Instant::now(); + let reopened = if cfg.engine == "holt" { + let mut config = TreeConfig::new(cfg.directory.join("metadata")); + config.checkpoint.enabled = false; + config.buffer_pool_size = 128; + Some(DB::open(config)?) + } else { + None + }; + let reopened = Arc::new(Shared { + cfg: cfg.clone(), + db: reopened, + tail: Mutex::new(JournalAnchor::new(0, [0; 32])), + version: AtomicU64::new(0), + }); + let mut session = Session::open(reopened)?; + let reopen_ms = reopen_start.elapsed().as_secs_f64() * 1000.0; + assert_eq!( + final_fingerprint, + fingerprint(&mut session)?, + "reopen state differs" + ); + println!( + "{}", + json!({ + "schema": "holt-sqlite.agent-workspace.v1", "engine": cfg.engine, + "sqlite_version": rusqlite::version(), "sync": cfg.sync, "attached": cfg.attached, + "threads": cfg.threads, "operations": operations, "warmup": cfg.warmup, + "workspaces": cfg.workspaces, "files_per_workspace": cfg.files, "payload_bytes": cfg.payload, + "read_mode": cfg.read_mode, "seed": cfg.seed, + "background_checkpoints": cfg.background, + "sqlite_synchronous": if cfg.sync {"FULL"} else if cfg.sqlite_normal {"NORMAL"} else {"OFF"}, + "holt_cache_target_bytes": 64 * 1024 * 1024, + "sqlite_cache_target_bytes_per_connection": 64 * 1024 * 1024, + "elapsed_seconds": elapsed.as_secs_f64(), "ops_per_second": operations as f64 / elapsed.as_secs_f64(), + "drain_inclusive_ops_per_second": operations as f64 / (elapsed + checkpoint_elapsed).as_secs_f64(), + "checkpoint_ms": checkpoint_elapsed.as_secs_f64() * 1000.0, "reopen_ms": reopen_ms, + "setup_seconds": setup_seconds, "latency": aggregate_latency, "per_operation": per_op, + "latencies_ns": samples, + "stages": { "view_calls": counters.views, "view_ms": counters.view_ns as f64 / 1e6, + "metadata_commits": counters.commits, "apply_ms": counters.apply_ns as f64 / 1e6, + "object_reads": counters.object_reads, "object_writes": counters.object_writes, + "object_ms": counters.object_ns as f64 / 1e6, "cas_retries": counters.retries }, + "space_before": before, "space_before_checkpoint": before_checkpoint, "space_after": after, + "fingerprint": final_fingerprint, "holt_stats": stats, "verified_reopen": true, + "qualification": "Shared application workflow; local metadata/content/JSON. No NoKV RPC, network, remote object store, or power-loss test. Checkpoint mode is recorded explicitly." + }) + ); + Ok(()) +}