Repository navigation
Conversation
XmlSaxParser feeds the document to libxml2 as raw-byte chunks, split at arbitrary positions, and reports its content through SAX2 callbacks while it is being parsed, without building a document tree - peak memory stays independent of the document size. The encoding is detected from the document itself; callbacks always receive UTF-8. The XmlSaxHandler callbacks cover document boundaries, namespace-aware element events with declarations and attributes, character data and CDATA (delivered as zero-copy views into the WebAssembly memory), comments and processing instructions. All callbacks are optional: events without a callback never cross the WebAssembly boundary. Parsing can be ended early with stop(), including from within a callback; push() and finish() throw XmlParseError on fatal diagnostics, while warning-level ones are collected in `warnings`. Handler exceptions abort the parse and are rethrown from push/finish rather than unwinding through libxml2's C frames, and re-entering or disposing the parser from a callback is rejected. Like the other wrappers owning libxml2 memory, the parser is released with dispose() or `using`, and the entity-store document libxml2 keeps for the internal subset is freed with it, whether parsing completed, failed or was stopped. Adds the _xmlCreatePushParserCtxt, _xmlParseChunk, _xmlStopParser, _xmlCtxtSetOptions and _xmlCtxtGetDocument exports to the binding.
The guard checked _malloc for the 0 it returns once the WebAssembly heap cannot grow anymore, instead of writing the chunk to address 0. But withCString is one of three marshalling sites that write to the returned address unchecked - allocUTF8Buffer and allocCStringArray make the same assumption - so guarding this one leaves the primitive intact everywhere else, and none of it is SAX work. Fixing all three belongs in its own change, together with the try/finally that xmlCreatePushParserCtxt then needs: it frees its xmlSAXHandler only after withStringUTF8 returns, so an allocUTF8Buffer that can throw turns that into a leak.
Three branches in sax.mts cannot be reached from a test, leaving the file short of the 100% the rest of src/ reports. Annotate them the way the other defensive branches in the codebase already are: - the null context from xmlCreatePushParserCtxt, which only happens when the WebAssembly heap cannot grow anymore; - the empty details fallback, which needs an error storage index that a live context can never be missing - errorIndices is populated at creation and cleared only by the free that zeroes _ptr; - the path out of parseChunk's catch into its finally, which V8 emits as a block and reports uncovered even though the catch always rethrows and can never take it. sax.mts now reports 100% of statements, branches, functions and lines.
|
Coverage after merging sax-push-parser into master will be
Coverage Report
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this adds
A new
XmlSaxParserfor parsing large XML files without loading them entirely into memory. Instead of building a complete document tree likeXmlDocumentdoes, the parser processes chunks of raw bytes and fires callbacks for each piece it encounters — opening and closing tags, text content, comments, and so on.This is useful when you're working with very large XML files and either don't need the whole document in memory at once, or only care about specific parts of it.
How it works
You create a parser with a set of optional callbacks, then feed it byte chunks:
The parser handles the messy parts for you: chunks might split in the middle of a tag or even a UTF-8 character, but the parser reassembles them transparently. (One text node may still arrive as several
characterscalls, hence the streamingTextDecoderin the example.) Encoding is detected from the BOM or XML declaration, and callbacks always receive UTF-8 bytes.Why this matters
Memory efficiency. A gigabyte XML file doesn't require a gigabyte of memory — the parser only keeps what it's actively working on in the buffer.
Pay for what you use. Callbacks are optional. If you don't care about processing instructions, you don't register that callback, and the events are skipped without even crossing the WebAssembly boundary. Large text nodes can be inspected as direct references to the parser's memory — valid only for the duration of the callback — rather than copied into JavaScript strings. And when you've seen the part of the document you care about,
stop()ends the parse early without an error.Clear error handling. If the XML becomes unparsable,
pushthrows immediately. Whenfinishreturns without throwing, you know the whole document was well-formed — including namespace rules, which plain parseability alone doesn't cover. Non-fatal diagnostics are collected in awarningsgetter instead of interrupting the parse.Safe boundaries. Exceptions thrown in callbacks abort the parse cleanly rather than corrupting libxml2's internal state. Re-entering the parser from a callback is rejected with an error instead of silently breaking things.
What changed internally
Compatibility notes
The parser rejects options that don't work in streaming mode (like DTD validation, which requires a complete tree) rather than silently misbehaving. Most standard parsing options are supported. The API is marked
@alphawhile the interface stabilizes based on feedback.