Skip to content

Commit 8cd44f5

Browse files
authored
fix: preserve leading newline of multiline string built with string() (#551)
`tomlkit.string(value, multiline=True)` rendered the value as `"""<value>"""`. When `value` starts with a newline the output looks like `"""\n..."""`, and per the TOML spec a newline immediately following the opening delimiter is trimmed on parse. So a value that begins with a newline was silently lost on the very next round-trip: >>> s = tomlkit.string("\nfoo", multiline=True) >>> str(s) '\nfoo' >>> s.as_string() '"""\nfoo"""' >>> str(tomlkit.parse(f"k = {s.as_string()}")["k"]) 'foo' # leading newline dropped The value the String object reports is correct; only its serialized form was wrong, so the corruption only surfaced after re-parsing. Both tomllib and tomlkit's own parser agree the old output decodes to the wrong value, so this is a genuine serialization bug (not a TOML 1.0 vs 1.1 difference). Affects basic and literal multiline strings, including values that start with `\r\n`. Fix: in `String.from_raw`, when the string is multiline and its rendered body starts with a newline, emit an extra leading newline (the one the parser trims) so the value survives round-tripping. This is exactly how the spec suggests representing such a value. Single-line strings and multiline strings without a leading newline are unchanged. An existing `test_create_string` case asserted the old lossy output (`"""\nMy\nString\n"""`); it only checked `as_string()`, never a round-trip, so it had locked in the bug. Updated it to the correct, round-trippable rendering and added a dedicated round-trip regression test covering basic/literal multiline and `\r\n`-leading values.
1 parent 67d3e86 commit 8cd44f5

3 files changed

Lines changed: 33 additions & 1 deletion

File tree

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@
2121

2222
### Fixed
2323

24+
- Fix `string()` dropping a leading newline of a multiline string on round-trip: a value beginning with a newline is now rendered with an extra leading newline (the one the parser trims after the opening delimiter) so it survives re-parsing.
2425
- Fix invalid serialization with a duplicated comma when removing a non-edge element from a parsed inline table. ([#486](https://github.com/python-poetry/tomlkit/pull/486))
2526
- Fix invalid serialization with a duplicated comma when appending or inserting into a comma-first formatted array. ([#499](https://github.com/python-poetry/tomlkit/pull/499))
2627
- Fix `ParseError` when a sub-table extends the last element of an array of tables after an unrelated table. ([#261](https://github.com/python-poetry/tomlkit/issues/261))

‎tests/test_api.py‎

Lines changed: 25 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -506,7 +506,7 @@ def test_create_super_table_with_aot() -> None:
506506
({}, "My String\x12", '"My String\\u0012"'),
507507
({}, "My String\x7f", '"My String\\u007f"'),
508508
({"escape": False}, "My String\u0001", '"My String\u0001"'),
509-
({"multiline": True}, "\nMy\nString\n", '"""\nMy\nString\n"""'),
509+
({"multiline": True}, "\nMy\nString\n", '"""\n\nMy\nString\n"""'),
510510
({"multiline": True}, 'My"String', '"""My"String"""'),
511511
({"multiline": True}, 'My""String', '"""My""String"""'),
512512
({"multiline": True}, 'My"""String', '"""My""\\"String"""'),
@@ -537,6 +537,30 @@ def test_create_string(kwargs: dict[str, Any], example: str, expected: str) -> N
537537
assert value.as_string() == expected
538538

539539

540+
@pytest.mark.parametrize(
541+
"kwargs, example",
542+
[
543+
({"multiline": True}, "\n"),
544+
({"multiline": True}, "\nMy\nString\n"),
545+
({"multiline": True}, "\r\nMy\nString"),
546+
({"multiline": True, "literal": True}, "\nMy\nString"),
547+
({"multiline": True, "literal": True}, "\r\nMy\nString"),
548+
],
549+
)
550+
def test_create_multiline_string_with_leading_newline_round_trips(
551+
kwargs: dict[str, Any], example: str
552+
) -> None:
553+
"""A multiline string whose value starts with a newline must survive a
554+
round-trip. The parser trims a newline immediately following the opening
555+
delimiter, so the rendered form has to account for it (otherwise the
556+
leading newline is silently dropped when the output is parsed again).
557+
"""
558+
value = tomlkit.string(example, **kwargs)
559+
assert str(value) == example
560+
reparsed = parse(f"k = {value.as_string()}\n")["k"]
561+
assert str(reparsed) == example
562+
563+
540564
@pytest.mark.parametrize(
541565
"kwargs, example",
542566
[

‎tomlkit/items.py‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2251,6 +2251,13 @@ def from_raw(
22512251
escaped = type_.escaped_sequences
22522252
string_value = escape_string(value, escaped) if escape and escaped else value
22532253

2254+
if type_.is_multiline() and string_value[:1] in ("\n", "\r"):
2255+
# A newline immediately following the opening delimiter of a
2256+
# multiline string is trimmed by the parser, so a value that
2257+
# starts with a newline would otherwise lose it on round-trip.
2258+
# Emit an extra leading newline (the trimmed one) to preserve it.
2259+
string_value = "\n" + string_value
2260+
22542261
return cls(type_, decode(value), string_value, Trivia())
22552262

22562263

0 commit comments

Comments
 (0)