Add a streaming JSON parser - #5658
Merged
Merged
Conversation
SeanTAllen
force-pushed
the
streaming-json-token-parser
branch
from
July 3, 2026 22:09
5eac443 to
57d32e5
Compare
SeanTAllen
force-pushed
the
streaming-json-token-parser
branch
from
July 4, 2026 00:17
57d32e5 to
71bc4ba
Compare
SeanTAllen
force-pushed
the
streaming-json-token-parser
branch
from
July 4, 2026 02:14
71bc4ba to
19ade51
Compare
The json package could only parse a whole document held in memory. This makes JsonTokenParser an incremental streaming parser: feed it bytes as they arrive with feed(), it emits tokens as they complete through the notifier, and it builds no tree, so the caller controls how much memory the parse uses. A raw-notifier consumer that drops the tokens it doesn't need stays flat however large the document. Two API changes come with it, both breaking for JsonTokenParser users (JsonParser is unchanged): tokens now carry their own value — JsonTokenKey/String/Number are val classes with a .value field, replacing last_string/last_number on the parser — and the parser is driven with feed()/finish() instead of parse(). A key or string value that fits within a single fed chunk with no escapes is a zero-copy view into that chunk rather than a fresh copy. JsonReassembler folds a token stream back into a JsonValue when the caller wants the tree, and JsonParseLimits bounds depth and string and number length for untrusted input. Design: #5655 Performance ----------- Rough numbers from a simple single-actor benchmark on one machine (16-core WSL2), not a rigorous benchmark suite — read them as direction, not precise figures. Best of 40 timed runs after warmup, one scheduler thread (--ponymaxthreads=1), peak memory from getrusage. The technique carries built-in inaccuracy: best-of-N reports the least-interrupted run, so it flatters; garbage-collection pauses add variance, worst on the tree-building (batch) path; and peak memory is the process high-water from ru_maxrss, which is coarse. Treat them as indicative, not authoritative. Zero-copy views versus the same parser decoding a copy of each string. Throughput (MB/s parsed) document views copy views faster by 100-char strings token ~185 ~111 1.7x batch ~85-115 ~48-74 ~1.5x (GC-noisy) 10 KB strings token ~925 ~208 4.4x batch ~865 ~209 4.1x Peak memory (RSS) document views copy views less by 8 MB single string 10 MB 26 MB 62% 40k small objects (200-char) 58 MB 79 MB 27% Views win on both speed and memory, and the win grows with string size: copying a big string per value is what the view avoids. For short strings ~1.7x faster and 27% less memory; for large strings ~4x faster and 62% less. (A byte-at-a-time buffered.Reader ran ~34 MB/s on the 10 KB case, about 27x slower, which the _ChunkReader design avoids.)
SeanTAllen
force-pushed
the
streaming-json-token-parser
branch
from
August 12, 2026 14:45
19ade51 to
bc6dff2
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
jsonpackage could only parse a whole document held in memory. This makesJsonTokenParseran incremental streaming parser: feed it bytes as they arrive withfeed(), it emits tokens as they complete through the notifier, and it builds no tree, so the caller controls how much memory the parse uses. A raw-notifier consumer that drops the tokens it doesn't need stays flat however large the document.Two API changes come with it, both breaking for
JsonTokenParserusers (JsonParseris unchanged):JsonTokenKey/JsonTokenString/JsonTokenNumberarevalclasses with a.valuefield, replacinglast_string/last_numberon the parser.feed()/finish()instead ofparse().A key or string value that fits within a single fed chunk with no escapes is a zero-copy view into that chunk rather than a fresh copy.
JsonReassemblerfolds a token stream back into aJsonValuewhen the caller wants the tree, andJsonParseLimitsbounds depth and string and number length for untrusted input.Design: #5655
Performance
These are rough numbers from a simple single-actor benchmark on one machine (16-core WSL2), not a rigorous benchmark suite — read them as direction, not precise figures. Best of 40 timed runs after warmup, one scheduler thread (
--ponymaxthreads=1), peak memory fromgetrusage. The technique carries built-in inaccuracy: best-of-N reports the least-interrupted run, so it flatters; garbage-collection pauses add variance, worst on the tree-building (batch) path; and peak memory is the process high-water fromru_maxrss, which is coarse. Treat them as indicative, not authoritative.Zero-copy string views versus the same parser decoding a copy of each string.
Throughput (MB/s parsed)
Peak memory (RSS)
Views win on both speed and memory, and the win grows with string size — copying a big string per value is what the view avoids. For short strings it's ~1.7× faster and 27% less memory; for large strings ~4× faster and 62% less. (For contrast, a byte-at-a-time
buffered.Readerran ~34 MB/s on the 10 KB case — about 27× slower than the shipped code — which the_ChunkReaderdesign avoids.)