Summary
annotation.RawFromNode renders two scalar tags in a form that does not carry what the source wrote, in the channel whose documented promise is verbatim preservation (ir.UnmodeledEntry.Value). This affects every raw-preservation site: x-* extensions, oneOf/anyOf sibling preservation, the §4.7 validation-only carve-out, and tuple items-after-prefix.
Timestamps are normalized to RFC 3339, so the source spelling is gone:
is preserved as "2021-01-01T00:00:00Z". The value is still a date, but 2021-1-1 is not recoverable, and the construct silently acquires a time and a zone the source never wrote.
Binary scalars are base64-decoded and the decoded bytes are then stored as a JSON string, which is lossy twice over:
x-blob: !!binary aGVsbG8= # preserved as "hello" — the base64 spelling is gone
x-raw: !!binary /w== # preserved as "�" — the byte 0xFF is destroyed
The second case is outright data loss: 0xFF is not valid UTF-8, so encoding/json replaces it with U+FFFD. Nothing downstream can tell that from a source that really did write U+FFFD.
Root cause
compilers/openapi/internal/annotation/rawjson.go, the !!timestamp and !!binary arms of scalar. Both reproduce what a whole-tree node.Decode(&any) used to do, which is how the compiler has always behaved — they were left alone when the numeric arms were changed to read source text (#32), because they are a different mechanism and deserve their own reasoning rather than being swept along.
Expected
Decide, and write down, what "verbatim" means for a scalar JSON has no type for. The plausible answer for both is the source spelling — 2021-1-1 stays 2021-1-1, and a !!binary keeps its base64 text — which is what the sibling value.FromNode channel already does for timestamps.
Whichever way it goes, ir.UnmodeledEntry.Value's doc comment should state it, since that comment is where a consumer learns what the field guarantees.
Notes
value.FromNode (the ir.Value channel) already keeps a timestamp's verbatim source spelling, so the two channels disagree today for the same source scalar.- Changing either arm will move
testdata/conformance/openapi/yaml-timestamp-scalars.* goldens; the fixture already exists.
Summary
annotation.RawFromNoderenders two scalar tags in a form that does not carry what the source wrote, in the channel whose documented promise is verbatim preservation (ir.UnmodeledEntry.Value). This affects every raw-preservation site:x-*extensions, oneOf/anyOf sibling preservation, the §4.7 validation-only carve-out, and tuple items-after-prefix.Timestamps are normalized to RFC 3339, so the source spelling is gone:
is preserved as
"2021-01-01T00:00:00Z". The value is still a date, but2021-1-1is not recoverable, and the construct silently acquires a time and a zone the source never wrote.Binary scalars are base64-decoded and the decoded bytes are then stored as a JSON string, which is lossy twice over:
The second case is outright data loss:
0xFFis not valid UTF-8, soencoding/jsonreplaces it with U+FFFD. Nothing downstream can tell that from a source that really did write U+FFFD.Root cause
compilers/openapi/internal/annotation/rawjson.go, the!!timestampand!!binaryarms ofscalar. Both reproduce what a whole-treenode.Decode(&any)used to do, which is how the compiler has always behaved — they were left alone when the numeric arms were changed to read source text (#32), because they are a different mechanism and deserve their own reasoning rather than being swept along.Expected
Decide, and write down, what "verbatim" means for a scalar JSON has no type for. The plausible answer for both is the source spelling —
2021-1-1stays2021-1-1, and a!!binarykeeps its base64 text — which is what the siblingvalue.FromNodechannel already does for timestamps.Whichever way it goes,
ir.UnmodeledEntry.Value's doc comment should state it, since that comment is where a consumer learns what the field guarantees.Notes
value.FromNode(their.Valuechannel) already keeps a timestamp's verbatim source spelling, so the two channels disagree today for the same source scalar.testdata/conformance/openapi/yaml-timestamp-scalars.*goldens; the fixture already exists.