fix(#788): tell a late transcription from a missing one - #790
Merged
Conversation
The check asks whether the MERGE COMMIT wrote the entry. Right at merge time, unanswerable afterwards, so an entry transcribed later reports as missing forever and a reader cannot tell the two apart. `--verify-transcribed` reads each flagged merge's PR body and asks the question a reader actually has, is the entry there, sorting into five outcomes instead of one: transcribed late, genuinely missing, no entry section at all, declared none by design, and unverifiable. Opt-in, because CI runs the bare report and this needs `gh` and one API call per flagged merge. Every failure mode of that call degrades to "unverified" rather than to "clean": no `gh`, no auth, a deleted PR, a rate limit, a subject with no PR number. **Running it against this branch found more than the flag count did.** 19 genuinely missing, not the 17 I first reported from a truncated `tail`, and 11 that correctly declare none. Nine of the 19 are entries I had not found by reading, so they are still unwritten. **Two blind spots came out of the removal matrix, not out of review.** A section saying "No entry. Process change only." was classified MISSING. The template asks for that answer rather than an empty section, so it is correct, and eleven merges use it. Now its own bucket. And an entry written as prose, with no `###` heading and no bullet, could not be matched at all. Falls back to the section's longest line, which survives reflow. The matrix also found one of my own fixtures decorative: disabling comment stripping in `declares_none()` changed nothing, because my boilerplate fixture began with `<!--` and so never matched the pattern either way. Replaced with boilerplate whose line starts with "None", which is the shape the real template uses. Disabling stripping now costs three cases. 22/22, nine mechanisms each individually disabled and confirmed to drop the count, each row's failing case recorded so a row that stops isolating is visible.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #788. Blocking the release steps behind it, per the plan in #786.
The problem
check-changelog-transcription.pyasks whether the merge commit changeddocs/CHANGELOG.md.That is the right question at merge time and the reason the check works. It is also unanswerable
afterwards, so an entry transcribed later reports as missing forever, and a reader cannot tell a
real omission from one already fixed.
The fix
--verify-transcribedreads each flagged merge's PR body and asks whether the entry is actually inthe file. Five outcomes instead of one:
Opt-in: CI runs the bare report, and this needs
ghplus one API call per flagged merge. Everyfailure mode of that call degrades to unverified rather than to clean.
What running it found
19 genuinely missing, 11 correctly declaring none, 1 with no section, 0 unverified.
Ten of the 19 are what PR #789 transcribes. Nine are entries I had not found by reading, so they
are still unwritten and now have names. That is the tool paying for itself inside its own PR.
Two blind spots the removal matrix found, not review
A declared "None" read as MISSING. The template asks for "None, reason" rather than an empty
section, and eleven merges use it. Classifying those as defects would have buried the real ones.
A prose entry could not be matched. No
###, no bullet, so nothing to look for. Falls back tothe section's longest line, which survives reflow.
And the matrix found one of my own fixtures decorative
Disabling comment stripping in
declares_none()changed nothing: my boilerplate fixture began with<!--so it never matched the pattern either way. The fixture proved nothing while looking likecoverage, which is
prove-the-test-fails.md's own subject. Replaced with boilerplate whose linestarts with "None", the shape the real template uses. Disabling stripping now costs three cases.
Verification
22/22 self-test. Nine mechanisms each individually disabled and confirmed to drop the count, with
each row's failing cases recorded, so a row that stops isolating is visible rather than silently
green.
CHANGELOG entry
None, tooling change with no effect on the library's public surface or behaviour.
SemVer impact
NONE.