Report the unresolvable chunk when the gc mark phase fails#1561
Merged
Conversation
Version-Control Performance CeilingsRuns: median of 3 executions per benchmark, excluding fixture setup. The
|
Sysbench-Style Benchmark: Doltlite vs SQLiteIn-MemoryReads
Writes
File-BackedReads
Writes
File-Backed (autocommit)Each statement runs as its own transaction — exposes per-commit ReadsReads have no commit cost; these are the same SQL files as the
Writes
100000 rows, median of 5 invocations per test; autocommit writes use 9, workload-only timing via host monotonic clock when available. Performance Ceiling Check (2.5x individual, 2x average; autocommit writes: 6x / 5x)All tests within ceilings. |
Sysbench-Style Benchmark (composite PK): Doltlite vs SQLiteCompanion to the classic Sysbench-Style Benchmark. Every workload here In-MemoryReads
Writes
File-BackedReads
Writes
File-Backed (autocommit)Each statement runs as its own transaction — exposes per-commit ReadsReads have no commit cost; these are the same SQL files as the
Writes
100000 rows, median of 5 invocations per test; autocommit writes use 9, workload-only timing via host monotonic clock when available. Performance Ceiling Check (2.5x individual, 2x average; autocommit writes: 6x / 5x)All tests within ceilings. |
Sysbench-Style Benchmark (BLOB PK): Doltlite vs SQLiteCompanion to the classic Sysbench-Style Benchmark. Every workload here In-MemoryReads
Writes
File-BackedReads
Writes
File-Backed (autocommit)Each statement runs as its own transaction — exposes per-commit ReadsReads have no commit cost; these are the same SQL files as the
Writes
100000 rows, median of 5 invocations per test; autocommit writes use 9, workload-only timing via host monotonic clock when available. Performance Ceiling Check (2.5x individual, 2x average; autocommit writes: 6x / 5x)All tests within ceilings. |
Sysbench-Style Benchmark (TEXT PK): Doltlite vs SQLiteCompanion to the classic Sysbench-Style Benchmark. Every workload here In-MemoryReads
Writes
File-BackedReads
Writes
File-Backed (autocommit)Each statement runs as its own transaction — exposes per-commit ReadsReads have no commit cost; these are the same SQL files as the
Writes
100000 rows, median of 5 invocations per test; autocommit writes use 9, workload-only timing via host monotonic clock when available. Performance Ceiling Check (2.5x individual, 2x average; autocommit writes: 6x / 5x)All tests within ceilings. |
A bare "gc mark phase failed" gives a field report nothing to act on. Track each queued hash's provenance through the mark walk and name the missing chunk, its parent hash, the edge source (refs table, branch tip/working set, tag, pending, session, or a child edge inside a stored chunk), the parent's chunk type — with level/item count for prolly nodes — the child ordinal, and the result code. This is the diagnostic layer from the closed #1556 without its test-build counters and stress gate; its field capture is what pinpointed the #1547 misclassification. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
timsehn
force-pushed
the
bug/gc-mark-failure-diagnostics
branch
from
35d4d2f to
9ab2f39
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #1547 / #1559, reviving the diagnostic output from the closed #1556 as a standalone change (per request) — without #1556's test-build counters, debug SQL registration, or CI stress gate.
What it does
dolt_gc()mark failures now report exactly what could not be resolved instead of a baregc mark phase failed:sourcenames the edge that produced the hash:refs-table,branch-commit,branch-working-set,tag-commit,pending,session, orchild(an edge inside a stored chunk).childedges: the parent chunk hash, its classified type (including the conflicts / constraint-violations types added in Fix GC classification of commit chunks #1559), the child ordinal, andparent_level/parent_itemswhen the parent is a prolly node.rcdistinguishes not-found (12) from corrupt reads (11).This exact output is what let the #1547 reporter capture an actionable diagnostic from their production store; keeping it means the next field report starts with the offending bytes instead of a guess.
Testing
gc_mark_failure_names_missing_chunkcase intest/doltlite_gc.sh: flips the first WAL record's tag byte in a healthy store and asserts the failure names a 40-hex chunk hash withsource=andrc=. Rebased on master post-Fix GC classification of commit chunks #1559: gc suite 55/55, classify suite 9/9,correctness_test.sh44/44.parent_type=working-set child_index=0 rc=12), and reportssource=pending rc=11on real byte corruption.🤖 Generated with Claude Code