Add support for output `MapType[StringType, ArrayType[StringType]]` in `from_json` SQL function [databricks] by ttnghia · Pull Request #15134 · NVIDIA/cudf-spark

ttnghia · 2026-06-24T04:56:18Z

This adds support for output type MapType[StringType, ArrayType[StringType]] in from_json SQL function. Previously, it can only output MapType[StringType, StringType].

Depends on:

Support output MAP<STRING, ARRAY<STRING>> when parsing JSON using extractRawMapFromJsonString cudf-spark-jni#4741

Benchmark

from_json → MAP<STRING,ARRAY<STRING>>, GPU vs CPU (2M rows, short regime of 1–4 element arrays, isolated method, 16 partitions, concurrentGpuTasks=4, UTC session — required for the GPU from_json path):

column	keys / map	avg JSON len	GPU avg (s)	CPU avg (s)	speedup
`j_narrow`	1–3	59 B	0.1852	0.4259	2.30× 🏆
`j_wide`	6–12	264 B	0.4428	0.6421	1.45× 🏆

The kernel-level microbenchmarks (array length, null density, mismatch density, and scalar-path regression) live in the dependency PR NVIDIA/cudf-spark-jni#4741; this table is the plugin-level end-to-end comparison, so a second plugin benchmark is not required in this wrapper PR.

Checklists

Documentation

Updated for new or modified user-facing features or behaviors
No user-facing change

Testing

Added or modified tests to cover new code paths
Covered by existing tests
(Please provide the names of the existing tests in the PR description.)
Not required

Performance

Tests ran and results are added in the PR description
Issue filed with a link in the PR description
Not required

--------- Signed-off-by: Nghia Truong <nghiat@nvidia.com>

Copilot

Pull request overview

Adds GPU support for from_json when the target schema is MAP<STRING,ARRAY<STRING>>, extending the existing MAP<STRING,STRING> path. The implementation uses a “raw extraction” approach (no JSON unescaping/normalization) and documents known CPU/GPU divergences.

Changes:

Extend GpuJsonToStructs to extract MAP<STRING,ARRAY<STRING>> via JSONUtils.extractRawMapFromJsonString(..., MapValueType.ARRAY_OF_STRING).
Update JsonToStructs GPU override type support/messaging to include MAP<STRING,ARRAY<STRING>>.
Add integration tests (including corner cases + non-strict xfail probes) and update docs to describe the new supported map schema and its divergences.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
sql-plugin/src/main/scala/org/apache/spark/sql/rapids/GpuJsonToStructs.scala	Adds map-value-type dispatch for `MAP<STRING,ARRAY<STRING>>` vs `MAP<STRING,STRING>` raw extraction.
sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuOverrides.scala	Expands `from_json` type signature + tagging logic and updates fallback message for the new map schema.
integration_tests/src/main/python/json_test.py	Adds coverage for `MAP<STRING,ARRAY<STRING>>` including fallback + corner/xfail probes.
docs/supported_ops.md	Updates the supported-ops note for `from_json` MAP output to include `ARRAY<STRING>` values.
docs/compatibility.md	Documents support for `MAP<STRING,ARRAY<STRING>>` and clarifies the “raw extraction” divergences.

greptile-apps · 2026-06-24T05:02:03Z

Greptile Summary

This PR extends the GPU from_json (JsonToStructs) implementation to support MAP<STRING, ARRAY<STRING>> as an output schema, in addition to the already-supported MAP<STRING, STRING>. The dispatch in doColumnar now uses an explicit MapValueType enum argument to the JNI layer (replacing the deprecated 2-arg extractRawMapFromJsonString) and adds a defensive throw for any unrecognized map value type that may be added in the future without a matching dispatch arm.

GpuOverrides: The TypeSig is widened to MAP.nested(STRING + ARRAY.nested(STRING)) with a PS note; tagExprForGpu adds an explicit case for MapType(StringType, ArrayType(StringType, _), _) so only the two valid map shapes reach GPU execution.
GpuJsonToStructs: Refactors the _: MapType catch-all into two explicit cases (ARRAY_OF_STRING and STRING) plus a defensive throw for other map value types, addressing a previously flagged implicit structural contract.
Tests: Adds seven new tests covering random-data GPU/CPU equality, fixed-literal corner cases, fallback for non-string array element types, and xfail probes for three documented divergences (escape handling, nested element re-serialization, numeric token rendering); compatibility.md is also updated with detailed notes on the divergences.

Confidence Score: 5/5

Safe to merge — the change is a well-bounded feature addition that follows all established patterns in the codebase and is guarded at both the TypeSig and tagExprForGpu layers.

The new MAP<STRING, ARRAY> dispatch arm mirrors the existing MAP<STRING, STRING> path exactly, using the same JNI call site with a new MapValueType argument. Known CPU/GPU divergences (escape handling, nested element re-serialization, numeric token rendering) are fully documented in docs/compatibility.md and covered by dedicated non-strict xfail probes. The defensive throw for unrecognised map value types closes the previously noted silent fall-through. Tests span random data, fixed-literal corner cases, and fallback verification for unsupported element types.

No files require special attention. All changed files are clean.

Important Files Changed

Filename	Overview
sql-plugin/src/main/scala/org/apache/spark/sql/rapids/GpuJsonToStructs.scala	Replaces the `_: MapType` catch-all with explicit dispatch arms for STRING and ARRAY_OF_STRING, using the new 3-arg JNI overload; adds a defensive throw for future map value types; resource lifecycle follows the same pattern as the pre-existing STRING path.
sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuOverrides.scala	TypeSig widened to allow ARRAY as a map value; tagExprForGpu adds the explicit MapType(StringType, ArrayType(StringType, _), _) case; fallback message updated — all correct and consistent with surrounding patterns.
integration_tests/src/main/python/json_test.py	Adds seven new tests: random-data GPU/CPU equality (with map_entries workaround), fixed-literal corner cases, fallback for ARRAY/ARRAY, and non-strict xfail probes for three documented divergences; all use assert_gpu_and_cpu_are_equal_collect or assert_gpu_fallback_collect per the integration-test-gpu-verification rule.
docs/compatibility.md	Documents the three divergence classes for MAP<STRING,ARRAY> (escape handling, nested re-serialization, numeric rendering), explains why Spark 4.0 enableExactStringParsing is a no-op for from_json on a string column, and notes matching behaviour (bad-record null, duplicate-key order).
docs/supported_ops.md	PS note updated to reflect the expanded map value support; wording is accurate and consistent with GpuOverrides.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["from_json(col, schema)"] --> B{TypeSig check in GpuOverrides}
    B -->|"MAP nested type not in STRING + ARRAY<STRING>"| C[CPU fallback]
    B -->|"Passes TypeSig"| D{tagExprForGpu schema match}
    D -->|"MapType(StringType, StringType, _)"| E[GPU: GpuJsonToStructs]
    D -->|"MapType(StringType, ArrayType(StringType, _), _)"| E
    D -->|"StructType (+ datetime/dup-key checks)"| E
    D -->|"anything else"| F["willNotWorkOnGpu → CPU fallback"]
    E --> G{doColumnar schema match}
    G -->|"MapType(StringType, ArrayType(StringType, _), _)"| H["extractRawMapFromJsonString(MapValueType.ARRAY_OF_STRING)"]
    G -->|"MapType(StringType, StringType, _)"| I["extractRawMapFromJsonString(MapValueType.STRING)"]
    G -->|"MapType(_, other, _)"| J["throw IllegalArgumentException (defensive guard)"]
    G -->|"StructType"| K["fromJSONToStructs + optional datetime convert"]
    H --> L["cudf.ColumnVector MAP<STRING, LIST<STRING>>"]
    I --> M["cudf.ColumnVector MAP<STRING, STRING>"]

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A["from_json(col, schema)"] --> B{TypeSig check in GpuOverrides}
    B -->|"MAP nested type not in STRING + ARRAY<STRING>"| C[CPU fallback]
    B -->|"Passes TypeSig"| D{tagExprForGpu schema match}
    D -->|"MapType(StringType, StringType, _)"| E[GPU: GpuJsonToStructs]
    D -->|"MapType(StringType, ArrayType(StringType, _), _)"| E
    D -->|"StructType (+ datetime/dup-key checks)"| E
    D -->|"anything else"| F["willNotWorkOnGpu → CPU fallback"]
    E --> G{doColumnar schema match}
    G -->|"MapType(StringType, ArrayType(StringType, _), _)"| H["extractRawMapFromJsonString(MapValueType.ARRAY_OF_STRING)"]
    G -->|"MapType(StringType, StringType, _)"| I["extractRawMapFromJsonString(MapValueType.STRING)"]
    G -->|"MapType(_, other, _)"| J["throw IllegalArgumentException (defensive guard)"]
    G -->|"StructType"| K["fromJSONToStructs + optional datetime convert"]
    H --> L["cudf.ColumnVector MAP<STRING, LIST<STRING>>"]
    I --> M["cudf.ColumnVector MAP<STRING, STRING>"]

_{Reviews (10): Last reviewed commit: "Merge branch 'main' into from-json-array" | Re-trigger Greptile}

Make the GpuJsonToStructs map dispatch explicit: match MAP<STRING,STRING> directly and throw on any other (currently unreachable) map value type instead of silently extracting it as STRING, so widening the GpuOverrides map gate later fails loudly rather than producing wrong results. Compare the array-valued-map equality test result via map_entries for deterministic ordering, parametrize the array fallback test over ARRAY<INT> and ARRAY<DOUBLE>, and rewrap the GpuJsonToStructs and GpuOverrides comments to the 100-column limit. --------- Signed-off-by: Nghia Truong <nghiat@nvidia.com>

# Conflicts: # sql-plugin/src/main/scala/org/apache/spark/sql/rapids/GpuJsonToStructs.scala

zpuller · 2026-07-07T18:44:18Z

regarding the checklist at the top, what is the performance story? testing not required I assume?

amahussein

Thanks @ttnghia

The schema gating and runtime dispatch look sound, and the null/empty matrix is strong: SQL-null input, JSON-null root, empty map/list/key, null map value, null element, and empty-string element are covered.
My blocking concern is the untested numeric normalization category described inline; the current “only two divergences” claim is not supported by the raw JNI behavior. Please add those cases/update the compatibility contract and run the final build with [databricks].

Please append [databricks] to the title before the final build. These common Scala/Python paths do not auto-trigger Databricks CI, while correctness here depends on matching the runtime CPU from_json behavior and the PR already identifies version-sensitive duplicate-key behavior.
Performance checklist: Please link those results here and explicitly mark performance testing as covered by NVIDIA/cudf-spark-jni#4741 /not required in this wrapper PR. I do not think a second plugin benchmark is necessary: This PR is dispatch wiring and JNI contains the kernel benchmarks, including array length, null density, mismatch density, and scalar-path regression results.
NVIDIA/cudf-spark-jni#4794 is not compile-required, but because it fixes the shared concat_json path, I recommend consuming that fix before the final #15134 validation. CC: @nartal1

ttnghia · 2026-07-07T21:35:19Z

I've implemented a benchmark for this:

from_json → MAP<STRING,ARRAY<STRING>> — GPU vs CPU

2M rows · short regime (1–4 elem arrays) · isolated method · 16 partitions · cgt=4

┌──────────┬────────────┬──────────────┬─────────────┬─────────────┬──────────┐
│  column  │ keys / map │ avg JSON len │ GPU avg (s) │ CPU avg (s) │ speedup  │
├──────────┼────────────┼──────────────┼─────────────┼─────────────┼──────────┤
│ j_narrow │    1–3     │     59 B     │   0.1852    │   0.4259    │ 2.30× 🏆 │
├──────────┼────────────┼──────────────┼─────────────┼─────────────┼──────────┤
│ j_wide   │    6–12    │    264 B     │   0.4428    │   0.6421    │ 1.45× 🏆 │
└──────────┴────────────┴──────────────┴─────────────┴─────────────┴──────────┘

Scripts:
bench_from_json.py
gen_from_json_data.py

nartal1 · 2026-07-07T22:41:39Z

NVIDIA/cudf-spark-jni#4794 is not compile-required, but because it fixes the shared concat_json path, I recommend consuming that fix before the final #15134 validation. CC: @nartal1

Merged NVIDIA/cudf-spark-jni#4794.

amahussein · 2026-07-07T23:10:27Z

I've implemented a benchmark for this:

from_json → MAP<STRING,ARRAY<STRING>> — GPU vs CPU

2M rows · short regime (1–4 elem arrays) · isolated method · 16 partitions · cgt=4

┌──────────┬────────────┬──────────────┬─────────────┬─────────────┬──────────┐
│  column  │ keys / map │ avg JSON len │ GPU avg (s) │ CPU avg (s) │ speedup  │
├──────────┼────────────┼──────────────┼─────────────┼─────────────┼──────────┤
│ j_narrow │    1–3     │     59 B     │   0.1852    │   0.4259    │ 2.30× 🏆 │
├──────────┼────────────┼──────────────┼─────────────┼─────────────┼──────────┤
│ j_wide   │    6–12    │    264 B     │   0.4428    │   0.6421    │ 1.45× 🏆 │
└──────────┴────────────┴──────────────┴─────────────┴─────────────┴──────────┘

Scripts: bench_from_json.py gen_from_json_data.py

can we make that part of the PR description to get the visibility? Usually, no one scroll to read comments after the PR is merged.

The GPU `from_json` `MAP<STRING,ARRAY<STRING>>` path raw-extracts array element bytes (surrounding quotes stripped, not unescaped or re-serialized), so number tokens can diverge from Spark, which re-renders numbers: non-canonical numeric spellings are kept verbatim (`007`, `1.00000`, `1e2`) where Spark emits `7`, `1.0`, `100.0`, and `NaN`/`Infinity` stay bare under `allowNonNumericNumbers` while Spark quotes them. Document this third corner case in `docs/compatibility.md` and the `GpuJsonToStructs` class doc, and note that canonical tokens (`1`, `1.5`, `true`) already match. Add xfail probes `test_from_json_map_with_arrays_numeric_xfail` and `test_from_json_map_with_arrays_nonnumeric_xfail` pinning the divergence, an options-propagation test `test_from_json_map_with_arrays_options` proving `allowSingleQuotes` and `allowUnquotedControlChars` reach the `ARRAY<STRING>` dispatch, and the canonical decimal `1.5` to the corner-case matches. --------- Signed-off-by: Nghia Truong <nghiat@nvidia.com>

…-json-array

ttnghia · 2026-07-08T05:10:24Z

build

ttnghia · 2026-07-08T18:58:21Z

run the final build with [databricks].

@amahussein what is the reason this needs to be tested on Databrick? There is no specific rules/optimization that is changed on Databrick.

amahussein · 2026-07-08T20:25:50Z

run the final build with [databricks].

@amahussein what is the reason this needs to be tested on Databrick? There is no specific rules/optimization that is changed on Databrick.

You're right that this PR does not change a Databricks-specific shim or optimization.
My reason for requesting the run is runtime semantic parity: these integration tests compare GPU output against the runtime's CPU from_json, and DBR's Spark fork has had documented from_json differences (for example #11719 and the DBR-specific xfails in #12483/#12488). This PR also exercises option-dependent normalization and version-sensitive duplicate keys across DBR 13.3/14.3/17.3; Spark 4's exact-string parsing makes the 17.3 result especially relevant even though the GPU code is common.

amahussein

The original numeric coverage/documentation, option-plumbing, performance, null/empty, and JNI dependency concerns are now addressed. JNI #4794 also merged with a raw-map-array embedded-NUL test. I found no remaining Scala implementation bug. Before I approve, I would like the new raw-token documentation/xfails qualified for Spark 4 exact-string parsing, plus either a successful Databricks rerun or an explicit waiver for the unrelated spark-protobuf setup failure. The request to link xfails to a tracking issue is a reasonable non-blocking cleanup.

Spark 4.0 introduced `spark.sql.json.enableExactStringParsing` (default `true`), which makes `JacksonParser` return the raw source bytes of non-string JSON tokens for a `StringType` target instead of re-serializing them via `copyCurrentStructure`. The GPU `from_json` raw-map path already emits those raw tokens, so on Spark 4.0+ the CPU matches the GPU for the numeric, non-numeric, and nested-element cases that diverge on Spark 3.5. The previous unconditional non-strict `xfail` probes therefore silently XPASS on Spark 4.0 and give no real coverage there. Gate `test_from_json_map_with_arrays_nested_elements_xfail`, `_numeric_xfail`, and `_nonnumeric_xfail` on `condition=is_before_spark_400()` so Spark 4.0+ runs a hard-equality assertion while Spark 3.5 keeps the tolerant probe. The escape-sequence probe stays unconditional because escaped string elements are `VALUE_STRING` tokens, unescaped via `getText` on every Spark version, so `enableExactStringParsing` does not affect them. Duplicate-key handling is left unchanged as it is orthogonal (`mapKeyDedupPolicy`/version-sensitive). Qualify the `from_json` divergence contract in `docs/compatibility.md` and the `GpuJsonToStructs` Scaladoc to scope the nested and numeric cases to Spark before 4.0, and link the raw-extraction tracking issue NVIDIA#15240 in the affected `xfail` reasons. --------- Signed-off-by: Nghia Truong <nghiat@nvidia.com>

…-json-array

ttnghia · 2026-07-09T03:32:03Z

build

res-life

LGTM

ttnghia · 2026-07-09T06:02:02Z

build

ttnghia · 2026-07-09T17:21:57Z

Remove databricks from CICD as it is currently broken.

ttnghia · 2026-07-09T17:22:01Z

build

amahussein · 2026-07-09T21:47:02Z

Remove databricks from CICD as it is currently broken.

What is the failure from Databricks and is it caused by the changes here?

ttnghia · 2026-07-09T21:52:50Z

What is the failure from Databricks and is it caused by the changes here?

I checked blossom and see that there is not a single databricks build could success recently. Will try databricks again in a bit after the json fix from jni built into snapshot.

amahussein · 2026-07-09T22:08:13Z

What is the failure from Databricks and is it caused by the changes here?

I checked blossom and see that there is not a single databricks build could success recently. Will try databricks again in a bit after the json fix from jni built into snapshot.

This does not mean that the databricks is the broken one.
The pipeline is designed to fail fast. If the common premerge-CI fails, it will sig-kill the Databricks pipeline. In that case, the failure comes from the parent pipeline (not databricks).

ttnghia · 2026-07-09T22:26:55Z

build

The Spark 4.0.0 CI newly exercised the from_json MAP<STRING,ARRAY<STRING>> raw-map xfails and three failed: the `nested_elements`, `numeric`, and `nonnumeric` probes were gated `condition=is_before_spark_400()` on the belief that the Spark 4.0.0 `enableExactStringParsing` option (default `true`) returns the raw source bytes for non-string tokens, matching the GPU. That belief is wrong: `from_json` on a string column parses via a Reader (the Spark `CreateJacksonParser.utf8String` helper), and the exact-parsing raw-bytes fast path in `JacksonParser` fires only for `Array[Byte]` / `PositionedReadable` (file/binary) sources, never a Reader, so the CPU always re-serializes non-string tokens via `copyCurrentStructure` on every Spark version, identical to 3.5. The nested-object, numeric, and non-numeric element divergences from the raw GPU substrings therefore persist on Spark 4.0.0 too. Drop the `is_before_spark_400()` condition from the three xfails so they xfail on all versions, and repurpose `exact_string_parsing_off_xfail` from a claim that the option reintroduces the divergence to a regression guard that the option is a no-op for from_json on a string column. Correct the same wrong 4.0.0 claim in the `GpuJsonToStructs` class doc and `docs/compatibility.md`. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Signed-off-by: Nghia Truong <nghiat@nvidia.com>

ttnghia · 2026-07-10T05:17:36Z

build

The `enableExactStringParsing` doc lines added in `cb99e73` exceeded the 100-column scalastyle limit (`file.line.length`) at lines 51-52. Reflow the paragraph to fit; no wording change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Signed-off-by: Nghia Truong <nghiat@nvidia.com>

ttnghia · 2026-07-10T05:38:34Z

build

ttnghia · 2026-07-10T06:40:35Z

Now added databricks back to the tests after tried without databrick. Now confirm that the previous failure was not due to databricks specific.

…-json-array

ttnghia · 2026-07-10T10:23:42Z

build

ttnghia · 2026-07-10T14:53:52Z

CICD failed due to unrelated issue:

 premerge-init-jenkins-rapids-databricks-premerge-github-3-14tvn was marked offline: Connection was broken: java.nio.channels.ClosedChannelException

amahussein · 2026-07-10T16:20:52Z

CICD failed due to unrelated issue:

 premerge-init-jenkins-rapids-databricks-premerge-github-3-14tvn was marked offline: Connection was broken: java.nio.channels.ClosedChannelException

I don't think this is the root issue. My guess is that this one part of the job that got killed due to a previous failure.

ttnghia · 2026-07-10T20:50:25Z

build

ttnghia · 2026-07-11T23:39:44Z

build

amahussein

Thanks @ttnghia -- The implementation and test concerns are addressed, and I verified that your Spark 4 enableExactStringParsing explanation is correct.

LGTM

Add GPU from_json support for MapType[StringType, ArrayType[StringType]]

fbe2de7

--------- Signed-off-by: Nghia Truong <nghiat@nvidia.com>

ttnghia self-assigned this Jun 24, 2026

Copilot AI review requested due to automatic review settings June 24, 2026 04:56

ttnghia added feature request New feature or request SQL part of the SQL/Dataframe plugin task Work required that improves the product but is not user facing labels Jun 24, 2026

Copilot started reviewing on behalf of ttnghia June 24, 2026 04:56 View session

Copilot AI reviewed Jun 24, 2026

View reviewed changes

Comment thread sql-plugin/src/main/scala/org/apache/spark/sql/rapids/GpuJsonToStructs.scala

Comment thread integration_tests/src/main/python/json_test.py

greptile-apps Bot reviewed Jun 24, 2026

View reviewed changes

Comment thread sql-plugin/src/main/scala/org/apache/spark/sql/rapids/GpuJsonToStructs.scala Outdated

ttnghia added 2 commits June 23, 2026 22:49

Merge branch 'main' into from-json-array

ba41b5b

# Conflicts: # sql-plugin/src/main/scala/org/apache/spark/sql/rapids/GpuJsonToStructs.scala

sameerz requested a review from a team July 7, 2026 19:22

amahussein requested changes Jul 7, 2026

View reviewed changes

Comment thread docs/compatibility.md Outdated

Comment thread integration_tests/src/main/python/json_test.py Outdated

amahussein mentioned this pull request Jul 7, 2026

Prevent NUL from being selected as the JSON row delimiter NVIDIA/cudf-spark-jni#4794

Merged

amahussein reviewed Jul 7, 2026

View reviewed changes

Comment thread integration_tests/src/main/python/json_test.py

ttnghia changed the title ~~Add support for output MapType[StringType, ArrayType[StringType]] in from_json SQL function~~ Add support for output MapType[StringType, ArrayType[StringType]] in from_json SQL function [databricks] Jul 8, 2026

ttnghia added 2 commits July 7, 2026 21:54

Merge branch 'main' of https://github.com/NVIDIA/cudf-spark into from…

72f32ea

…-json-array

ttnghia requested a review from amahussein July 8, 2026 05:09

res-life reviewed Jul 8, 2026

View reviewed changes

Comment thread integration_tests/src/main/python/json_test.py Outdated

amahussein reviewed Jul 8, 2026

View reviewed changes

Comment thread docs/compatibility.md

Comment thread integration_tests/src/main/python/json_test.py Outdated

ttnghia mentioned this pull request Jul 8, 2026

[BUG] from_json MAP<STRING,STRING> / MAP<STRING,ARRAY<STRING>> return array/value tokens as raw JSON text, diverging from Spark #15240

Open

ttnghia added 2 commits July 8, 2026 18:08

Merge branch 'main' of https://github.com/NVIDIA/cudf-spark into from…

ff11e16

…-json-array

ttnghia requested a review from amahussein July 9, 2026 03:34

res-life previously approved these changes Jul 9, 2026

View reviewed changes

ttnghia changed the title ~~Add support for output MapType[StringType, ArrayType[StringType]] in from_json SQL function [databricks]~~ Add support for output MapType[StringType, ArrayType[StringType]] in from_json SQL function Jul 9, 2026

ttnghia changed the title ~~Add support for output MapType[StringType, ArrayType[StringType]] in from_json SQL function~~ Add support for output MapType[StringType, ArrayType[StringType]] in from_json SQL function [databricks] Jul 9, 2026

ttnghia dismissed res-life’s stale review via cb99e73 July 10, 2026 05:16

Merge branch 'main' of https://github.com/NVIDIA/cudf-spark into from…

b547c9e

…-json-array

Merge branch 'main' into from-json-array

081c819

amahussein approved these changes Jul 13, 2026

View reviewed changes

amahussein merged commit f58a048 into NVIDIA:main Jul 13, 2026
50 checks passed

ttnghia deleted the from-json-array branch July 13, 2026 14:32

Uh oh!

Conversation

ttnghia commented Jun 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmark

Checklists

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

greptile-apps Bot commented Jun 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Flowchart

Uh oh!

Uh oh!

zpuller commented Jul 7, 2026

Uh oh!

amahussein left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

ttnghia commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nartal1 commented Jul 7, 2026

Uh oh!

amahussein commented Jul 7, 2026

Uh oh!

Uh oh!

ttnghia commented Jul 8, 2026

Uh oh!

Uh oh!

ttnghia commented Jul 8, 2026

Uh oh!

amahussein commented Jul 8, 2026

Uh oh!

amahussein left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

ttnghia commented Jul 9, 2026

Uh oh!

res-life left a comment

Choose a reason for hiding this comment

Uh oh!

ttnghia commented Jul 9, 2026

Uh oh!

ttnghia commented Jul 9, 2026

Uh oh!

ttnghia commented Jul 9, 2026

Uh oh!

amahussein commented Jul 9, 2026

Uh oh!

ttnghia commented Jul 9, 2026

Uh oh!

amahussein commented Jul 9, 2026

Uh oh!

ttnghia commented Jul 9, 2026

Uh oh!

ttnghia commented Jul 10, 2026

Uh oh!

ttnghia commented Jul 10, 2026

Uh oh!

ttnghia commented Jul 10, 2026

Uh oh!

ttnghia commented Jul 10, 2026

Uh oh!

ttnghia commented Jul 10, 2026

Uh oh!

amahussein commented Jul 10, 2026

Uh oh!

ttnghia commented Jun 24, 2026 •

edited

Loading

greptile-apps Bot commented Jun 24, 2026 •

edited

Loading

ttnghia commented Jul 7, 2026 •

edited

Loading