Legate rejects cuPyNumeric array real component

[elliottslaughter]

The following program fails:

from legate.core.task import task, InputArray
import cupynumeric as cpn

@task
def f(x: InputArray):
    pass

y = cpn.zeros(3, dtype=cpn.complex128)
z = y.real
f(z)

Error:

$ python test_complex.py 
Traceback (most recent call last):
  File "test_complex.py", line 10, in <module>
    f(z)
  File "py_task.pyx", line 228, in legate.core._ext.task.py_task.PyTask.__call__
  File "py_task.pyx", line 198, in legate.core._ext.task.py_task.PyTask.prepare_call
  File "invoker.pyx", line 580, in legate.core._ext.task.invoker.VariantInvoker.prepare_call
  File "invoker.pyx", line 483, in legate.core._ext.task.invoker.VariantInvoker._prepare_params
  File "invoker.pyx", line 420, in legate.core._ext.task.invoker.VariantInvoker._handle_param
TypeError: Argument: 'x' expected one of (<class 'legate.core._lib.data.logical_store.LogicalStore'>, <class 'legate.core._lib.data.logical_array.LogicalArray'>), got <class 'cupynumeric._array.array.ndarray'>

Oddly enough you can work around it by passing z to cpn.array:

from legate.core.task import task, InputArray
import cupynumeric as cpn

@task
def f(x: InputArray):
    pass

y = cpn.zeros(3, dtype=cpn.complex128)
z = y.real
w = cpn.array(z)
f(w)

$ python test_complex2.py

I can't see a reason why Legate should be rejecting the .real component of a complex array when it is already a valid cuPyNumeric array.

$ legate-issue
Python      :  3.12.8 | packaged by conda-forge | (main, Dec  5 2024, 14:24:40) [GCC 13.3.0]
Platform    :  Linux-4.18.0-553.40.1.el8_10.x86_64-x86_64-with-glibc2.28
Legion      :  25.3.0 (commit: 2bef0fe6d5f7fc9f08898b57769930f5de964a6c)
Legate      :  25.3.0.dev37+g79203c03
cuPynumeric :  25.03.00.dev+8.g5fff8e5f
Numpy       :  1.26.4
Scipy       :  1.15.1
Numba       :  (failed to detect)
CTK package :  cuda-version-12.8-h5d125a7_3 (conda-forge)
GPU driver  :  570.86.15
GPU devices :  
  GPU 0: NVIDIA A100-SXM4-40GB
  GPU 1: NVIDIA A100-SXM4-40GB
  GPU 2: NVIDIA A100-SXM4-40GB
  GPU 3: NVIDIA A100-SXM4-40GB

[Jacobfaib]

@elliottslaughter can you print the contents of

>>> print(y.real.__legate_data_interface__)

[elliottslaughter]

Ok, I understand what's going on now. The array in question falls under CUPYNUMERIC_MIN_GPU_CHUNK and therefore is executed with cuPyNumeric's eager code path. That causes the array to be allocated with NumPy. Then, when I execute the task, the array is forced to be moved over to Legate (because the task is always deferred; there is no eager code path for it). The error happens at the point where that conversion occurs.

I can get the original reproducer to work by setting CUPYNUMERIC_MIN_GPU_CHUNK=2 CUPYNUMERIC_MIN_CPU_CHUNK=2 CUPYNUMERIC_MIN_OMP_CHUNK=2 when I run. That reduces the chunk size small enough to force the array through the deferred code path.

Unfortunately, that is not a general-purpose solution for me, because I do in fact have a lot of smaller arrays that are very inefficient to handle with Legate. I need the eager code path to work, which means correctly handling the conversions back and forth between the two.

Answering your direct question, the error I get when I request this is below. I get this because y is an eager array.

Traceback (most recent call last):
  File "test_complex.py", line 9, in <module>
    print(y.real.__legate_data_interface__)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "miniforge3/envs/legate-2025-03-19/lib/python3.1/site-packages/cupynumeric/_array/array.py", line 156, in __legate_data_interface__
    deferred_thunk = runtime.to_deferred_array(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "miniforge3/envs/legate-2025-03-19/lib/python3.1/site-packages/cupynumeric/runtime.py", line 591, in to_deferred_array
    return array.to_deferred_array(read_only)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "miniforge3/envs/legate-2025-03-19/lib/python3.1/site-packages/cupynumeric/_thunk/eager.py", line 320, in to_deferred_array
    self._convert_tree()
  File "miniforge3/envs/legate-2025-03-19/lib/python3.1/site-packages/cupynumeric/_thunk/eager.py", line 295, in _convert_tree
    self._convert_subtree()
  File "miniforge3/envs/legate-2025-03-19/lib/python3.1/site-packages/cupynumeric/_thunk/eager.py", line 272, in _convert_subtree
    deferred = runtime.find_or_create_array_thunk(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "miniforge3/envs/legate-2025-03-19/lib/python3.1/site-packages/cupynumeric/runtime.py", line 457, in find_or_create_array_thunk
    store = legate_runtime.create_store_from_buffer(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "runtime.pyx", line 810, in legate.core._lib.runtime.runtime.Runtime.create_store_from_buffer
  File "runtime.pyx", line 858, in legate.core._lib.runtime.runtime.Runtime.create_store_from_buffer
  File "external_allocation.pyx", line 73, in legate.core._lib.data.external_allocation.create_from_buffer
ValueError: ndarray is not C-contiguous

In contrast, when I run with CUPYNUMERIC_MIN_GPU_CHUNK=2 CUPYNUMERIC_MIN_CPU_CHUNK=2 CUPYNUMERIC_MIN_OMP_CHUNK=2 I instead get:

{'version': 1, 'data': {Field('cuPyNumeric Array'): <legate.core._lib.data.logical_array.LogicalArray object at 0x147758683990>}}