Add single-token fingerprint complementarity analysis - #9
Draft
ktwu01 wants to merge 3 commits into
Draft
Conversation
Verified bruckner2026onetoken against arXiv:2607.10252 and Zenodo DOI 10.5281/zenodo.21278557: AUC 0.971, EER 7.3%, 165 models, released JSD matrix all match the cited values. - Demote the main-text paragraph to one sentence and drop 'orthogonal' in favor of 'largely complementary' / 'nearly independent', matching the observed weak correlations (rho=0.282 Jaccard, 0.074 HSS). - Record the source divergence-matrix.csv sha256 in the alignment output so the exact input can be pinned on reproduction. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
bojieli
reviewed
Jul 29, 2026
bojieli
left a comment
bojieli
left a comment
Contributor
There was a problem hiding this comment.
The core analysis reproduces correctly, and the combined suite passes on current main. A few changes before merge:
- Rebase onto current main, retain both scripts 19 and 20 in scripts/README.md, and regenerate paper/main.pdf from the resolved sources.
- same_vendor currently means the served-ID/OpenRouter namespace. Please rename/document that grouping or use an explicit vendor mapping.
- Pin the stem-to-served-model-ID alignment in the output provenance; those data/results inputs currently affect the cohort without being hashed.
- Small robustness fix: handle a non-finite Spearman statistic so constant inputs do not generate JSON NaN.
Please also refresh the stale test count and PDF page count after rebasing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Findings
The exact non-thinking endpoint join shows weak overlap between the two signals: Spearman rho is 0.282 for IKP Jaccard and 0.074 for HSS overall. Within-vendor correlations are 0.374 and 0.261. This supports complementarity without claiming that a fused verifier has already improved error rates.
Validation
python3 -m unittest discover -s tests -v(4/4 pass)10.5281/zenodo.21278557latexmk -cd -pdf -interaction=nonstopmode -halt-on-error paper/main.texCloses #8