Skip to content

Add a streaming JSON parser - #5658

Merged
SeanTAllen merged 1 commit into
mainfrom
streaming-json-token-parser
Aug 12, 2026
Merged

Add a streaming JSON parser#5658
SeanTAllen merged 1 commit into
mainfrom
streaming-json-token-parser

Conversation

@SeanTAllen

@SeanTAllen SeanTAllen commented Jul 3, 2026

Copy link
Copy Markdown
Member

The json package could only parse a whole document held in memory. This makes JsonTokenParser an incremental streaming parser: feed it bytes as they arrive with feed(), it emits tokens as they complete through the notifier, and it builds no tree, so the caller controls how much memory the parse uses. A raw-notifier consumer that drops the tokens it doesn't need stays flat however large the document.

Two API changes come with it, both breaking for JsonTokenParser users (JsonParser is unchanged):

  • Tokens now carry their own value — JsonTokenKey/JsonTokenString/JsonTokenNumber are val classes with a .value field, replacing last_string/last_number on the parser.
  • The parser is driven with feed() / finish() instead of parse().

A key or string value that fits within a single fed chunk with no escapes is a zero-copy view into that chunk rather than a fresh copy. JsonReassembler folds a token stream back into a JsonValue when the caller wants the tree, and JsonParseLimits bounds depth and string and number length for untrusted input.

Design: #5655

Performance

These are rough numbers from a simple single-actor benchmark on one machine (16-core WSL2), not a rigorous benchmark suite — read them as direction, not precise figures. Best of 40 timed runs after warmup, one scheduler thread (--ponymaxthreads=1), peak memory from getrusage. The technique carries built-in inaccuracy: best-of-N reports the least-interrupted run, so it flatters; garbage-collection pauses add variance, worst on the tree-building (batch) path; and peak memory is the process high-water from ru_maxrss, which is coarse. Treat them as indicative, not authoritative.

Zero-copy string views versus the same parser decoding a copy of each string.

Throughput (MB/s parsed)

document views copy views faster by
100-char strings token ~185 ~111 1.7×
batch ~85–115 ~48–74 ~1.5× (GC-noisy)
10 KB strings token ~925 ~208 4.4×
batch ~865 ~209 4.1×

Peak memory (RSS)

document views copy views less by
8 MB single string 10 MB 26 MB 62%
40k small objects (200-char strings) 58 MB 79 MB 27%

Views win on both speed and memory, and the win grows with string size — copying a big string per value is what the view avoids. For short strings it's ~1.7× faster and 27% less memory; for large strings ~4× faster and 62% less. (For contrast, a byte-at-a-time buffered.Reader ran ~34 MB/s on the 10 KB case — about 27× slower than the shipped code — which the _ChunkReader design avoids.)

@ponylang-main ponylang-main added the discuss during sync Should be discussed during an upcoming sync label Jul 3, 2026
@SeanTAllen
SeanTAllen force-pushed the streaming-json-token-parser branch from 5eac443 to 57d32e5 Compare July 3, 2026 22:09
@SeanTAllen SeanTAllen changed the title Make JsonTokenParser an incremental streaming parser Add a streaming JSON parser Jul 3, 2026
@SeanTAllen
SeanTAllen force-pushed the streaming-json-token-parser branch from 57d32e5 to 71bc4ba Compare July 4, 2026 00:17
@SeanTAllen SeanTAllen added the do not merge This PR should not be merged at this time label Jul 4, 2026
@SeanTAllen
SeanTAllen force-pushed the streaming-json-token-parser branch from 71bc4ba to 19ade51 Compare July 4, 2026 02:14
The json package could only parse a whole document held in memory. This makes
JsonTokenParser an incremental streaming parser: feed it bytes as they arrive
with feed(), it emits tokens as they complete through the notifier, and it
builds no tree, so the caller controls how much memory the parse uses. A
raw-notifier consumer that drops the tokens it doesn't need stays flat however
large the document.

Two API changes come with it, both breaking for JsonTokenParser users
(JsonParser is unchanged): tokens now carry their own value —
JsonTokenKey/String/Number are val classes with a .value field, replacing
last_string/last_number on the parser — and the parser is driven with
feed()/finish() instead of parse().

A key or string value that fits within a single fed chunk with no escapes is a
zero-copy view into that chunk rather than a fresh copy. JsonReassembler folds a
token stream back into a JsonValue when the caller wants the tree, and
JsonParseLimits bounds depth and string and number length for untrusted input.

Design: #5655

Performance
-----------

Rough numbers from a simple single-actor benchmark on one machine (16-core
WSL2), not a rigorous benchmark suite — read them as direction, not precise
figures. Best of 40 timed runs after warmup, one scheduler thread
(--ponymaxthreads=1), peak memory from getrusage. The technique carries built-in
inaccuracy: best-of-N reports the least-interrupted run, so it flatters;
garbage-collection pauses add variance, worst on the tree-building (batch) path;
and peak memory is the process high-water from ru_maxrss, which is coarse. Treat
them as indicative, not authoritative.

Zero-copy views versus the same parser decoding a copy of each string.

Throughput (MB/s parsed)

  document                     views     copy      views faster by
  100-char strings    token    ~185      ~111      1.7x
                      batch    ~85-115   ~48-74    ~1.5x (GC-noisy)
  10 KB strings       token    ~925      ~208      4.4x
                      batch    ~865      ~209      4.1x

Peak memory (RSS)

  document                        views    copy     views less by
  8 MB single string              10 MB    26 MB    62%
  40k small objects (200-char)    58 MB    79 MB    27%

Views win on both speed and memory, and the win grows with string size: copying
a big string per value is what the view avoids. For short strings ~1.7x faster
and 27% less memory; for large strings ~4x faster and 62% less. (A byte-at-a-time
buffered.Reader ran ~34 MB/s on the 10 KB case, about 27x slower, which the
_ChunkReader design avoids.)
@SeanTAllen
SeanTAllen force-pushed the streaming-json-token-parser branch from 19ade51 to bc6dff2 Compare August 12, 2026 14:45
@SeanTAllen SeanTAllen removed the do not merge This PR should not be merged at this time label Aug 12, 2026
@SeanTAllen
SeanTAllen merged commit 019e114 into main Aug 12, 2026
16 checks passed
@SeanTAllen
SeanTAllen deleted the streaming-json-token-parser branch August 12, 2026 17:02
@ponylang-main ponylang-main removed the discuss during sync Should be discussed during an upcoming sync label Aug 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants