Commit 169b341
test(tokenizer): add a differential harness for fastForwardTo
`fastForwardTo` can be written as a `charCodeAt` loop or as a single
`String.indexOf`, and swapping one for the other is only safe if they agree
on (return value, resulting `index`, exception) for every reachable
argument. The four invariants that matter are the ones real markup never
exercises: the scan starts at `index + 1` because the caller already looked
at `index`; success reports an absolute index; failure lands on
`buffer.length + offset - 1`, one short, to compensate for the `parse`
loop's increment; nothing else is touched.
Transcribe the current loop as an oracle and compare the shipped method
against it over ~13k exhaustive short-string cases, the empty buffer,
non-zero offsets, start positions on both sides of the buffer, astral-plane
content and 3000 seeded random buffers across all seven reachable sentinels.
Add stream-level equivalence on top: the same tokenizer run with the oracle
swapped onto the prototype must emit an identical event stream, across long
style bodies, `<` inside script and style literals, inputs that end before
their sentinel arrives, CDATA, processing instructions, every attribute
quoting style, and write-chunk splits of 1, 2, 7, 64 and 1000 - the one
dimension where the index/offset arithmetic crosses a buffer boundary.
While the loop is what ships, both comparisons are the oracle against
itself and prove nothing, so the file carries its own self-tests: mutants
for a dropped failure-path `- 1`, an unguarded needle table above code 127,
and a scan that re-examines the current character, each of which must be
caught, plus a correct implementation that must not be.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>1 parent 9d40676 commit 169b341
1 file changed
Lines changed: 605 additions & 0 deletions
0 commit comments