parse_json for a string where string contains "" inside the key while reading message from kafka

[yjagdale]

I am reading message from kafka and trying to parse the json, For some message I am getting "" inside the value of the key.

i tried replace but its not helping. Can someone help here? Thanks in advance

sample message:
vrl code:
VRL CODE

sample message:

{
    "message": "{\"test\": \"this\", \"key_with_quotes\": \"\"quotes\"\"}"
}

code:

.msg, .replaceerr = replace(.message, s'""', s'\"\"')
.out, .err = parse_json(.msg)

[pront]

Hi @yjagdale, the key_with_quotes value is an invalid JSON string, it needs to be escaped like so: \"\\\"quotes\\\"\"

Ideally your source would emit valid JSON strings but you can insert these manually as well.

[yjagdale]

I tried doing that. if you see the code to replace. but its not working for me :(

[pront]

Try this in the VRL Playground

The inner message is invalid JSON: the quotation marks that belong to the string value need to be escaped. The producer should emit this JSON text:

{"test":"this","key_with_quotes":"\"quotes\""}

Then .out = parse_json!(.message) parses it directly. The playground includes the complete outer event and correct escaping. A global replacement of "" cannot reliably repair arbitrary JSON; it also matches valid empty strings.