Skip to content

Can we use the arrow PyCapsule/C api when ingesting from pyarrow.Table? #2729

New issue
New issue
Closed
Closed
Can we use the arrow PyCapsule/C api when ingesting from pyarrow.Table?#2729
Labels
enhancementFeature requests or improvements

Description

@timkpaine
timkpaine
opened on Aug 30, 2024

we have a relatively long-winded ingestion from a pyarrow.Table:

https://github.com/finos/perspective/blob/768ca4068c79a2c2243aa24fd3a073f09c6dc97d/rust/perspective-python/src/client/python.rs#L166

Even though it would start as a different format, can we save time by going through the pycapsule/c interface: https://arrow.apache.org/docs/format/CDataInterface/PyCapsuleInterface.html#consume-a-pycapsule

xref: #1157

Activity

  1. timkpaine
    added
    enhancementFeature requests or improvements
    on Aug 30, 2024

  2. texodus commented on Feb 2, 2025

    @texodus
    texodus
    on Feb 2, 2025
    Member

    Converting to a discussion - definitely interested in higher performance Arrow ingestion!

  3. perspective-dev
    locked and limited conversation to collaborators on Feb 2, 2025
  4. texodus
    converted this issue into a discussion #2911 on Feb 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementFeature requests or improvements

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions