Skip to content

Remove inaccessible and unused AccessibleArrowColumnVector - #15075

Merged
sdrp713 merged 2 commits into
NVIDIA:mainfrom
WilliamK112:remove-unused-accessiblearrowcolumnvector
Jul 29, 2026
Merged

Remove inaccessible and unused AccessibleArrowColumnVector#15075
sdrp713 merged 2 commits into
NVIDIA:mainfrom
WilliamK112:remove-unused-accessiblearrowcolumnvector

Conversation

@WilliamK112

Copy link
Copy Markdown
Contributor

Summary

Fixes #11863

AccessibleArrowColumnVector was originally added (in #1622) for a customer who planned to write their own DataSource V2. That customer is not using it, and as noted in the issue, the class is not actually accessible at runtime due to the shimming / parallel-world packaging. The branches that reference it are therefore dead code.

Changes

  • Delete sql-plugin/src/main/java/org/apache/spark/sql/vectorized/rapids/AccessibleArrowColumnVector.java.
  • Remove its references in HostColumnarToGpu.scala:
    • the import
    • the case av: AccessibleArrowColumnVector branch in arrowColumnarCopy (the case _ already throws for any other vector type)
    • the isInstanceOf[AccessibleArrowColumnVector] check when selecting the Arrow batch builder

Testing

  • No remaining references to AccessibleArrowColumnVector in the repo (verified via grep).
  • CI to validate compile/build (local environment does not have the full Scala/Maven build set up).

Made with Cursor

@greptile-apps

greptile-apps Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR removes the AccessibleArrowColumnVector Java class and all its call sites in HostColumnarToGpu.scala. The class was originally added for a customer-specific DataSource V2 use case but was never reachable at runtime due to spark-rapids' parallel-world shim packaging.

  • Deletes AccessibleArrowColumnVector.java (507 lines), a dead subclass of Spark's ColumnVector that provided public access to the underlying Arrow ValueVector.
  • Removes the case av: AccessibleArrowColumnVector branch from arrowColumnarCopy and the isInstanceOf[AccessibleArrowColumnVector] guard in initNewBatch, leaving the standard ArrowColumnVector path and the existing case _ error throw intact.

Confidence Score: 5/5

Safe to merge — removes dead code that was provably unreachable at runtime; no functional paths are altered.

Both changed files make purely subtractive changes to dead-code paths. The case _ error branch in arrowColumnarCopy already covered any non-ArrowColumnVector input, and a grep of the full repo confirms zero remaining references to AccessibleArrowColumnVector. The remaining Arrow and non-Arrow copy paths in HostColumnarToGpu are unchanged.

Files Needing Attention: No files require special attention.

Important Files Changed

Filename Overview
sql-plugin/src/main/java/org/apache/spark/sql/vectorized/rapids/AccessibleArrowColumnVector.java Entire file deleted — 507-line dead-code class that was inaccessible at runtime due to shim packaging; no callers remain in the codebase.
sql-plugin/src/main/scala/com/nvidia/spark/rapids/HostColumnarToGpu.scala Removed import, dead match branch, and isInstanceOf check for AccessibleArrowColumnVector; remaining ArrowColumnVector path and error fallback are unaffected.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[HostToGpuCoalesceIterator.initNewBatch] -->|useArrowCopyOpt && numCols > 0 && arrowTypesSupported| B{column 0 type?}
    B -->|ArrowColumnVector| C[GpuArrowColumnarBatchBuilder]
    B -->|anything else| D[GpuColumnarBatchBuilder]

    E[arrowColumnarCopy] -->|cv match| F{ColumnVector type?}
    F -->|ArrowColumnVector| G[getArrowValueVector via reflection]
    F -->|anything else| H[throw IllegalStateException]

    style B fill:#d4edda,stroke:#28a745
    style F fill:#d4edda,stroke:#28a745
Loading

Reviews (3): Last reviewed commit: "Merge branch 'main' into remove-unused-a..." | Re-trigger Greptile

@WilliamK112
WilliamK112 force-pushed the remove-unused-accessiblearrowcolumnvector branch from b1c9258 to c96159c Compare June 12, 2026 18:15
@nvauto

nvauto commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

NOTE: release/26.08 has been created from main. Please retarget your PR to release/26.08 if it should be included in the release.

@sdrp713 sdrp713 assigned sdrp713 and unassigned sdrp713 Jul 28, 2026
@sdrp713
sdrp713 self-requested a review July 28, 2026 16:07
AccessibleArrowColumnVector was originally added for a customer that planned to
write their own DataSource V2, but it is no longer used. Due to the shimming and
parallel-world packaging, the class is not actually accessible at runtime, so the
code branches referencing it in HostColumnarToGpu are dead. Remove the class and
its references.

Fixes NVIDIA#11863

Signed-off-by: Ching Wei Kang <ckang53@wisc.edu>
Co-authored-by: Cursor <cursoragent@cursor.com>
@sdrp713
sdrp713 force-pushed the remove-unused-accessiblearrowcolumnvector branch from c96159c to ad9b5a8 Compare July 28, 2026 16:08
@sdrp713

sdrp713 commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

LGTM

@sdrp713

sdrp713 commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

build

@sdrp713
sdrp713 merged commit fc1e8f1 into NVIDIA:main Jul 29, 2026
55 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Remove inaccessible and unused class AccessibleArrowColumnVector

4 participants