feat[next-dace]: lowering to SchedTree

[edopao]

Summary

Adds a second GTIR→SDFG lowering path for the gt4py.next DaCe backend that goes through
DaCe's schedule tree (ScheduleTreeRoot) instead of building the SDFG directly, and
wires six new run_dace_stree_* backends into the test matrix:

• run_dace_stree_fview_{cpu,gpu}[_noopt] — fieldview pipeline (as today)
• run_dace_stree_iview_{cpu,gpu}[_noopt] — with the ITIR common-transforms pipeline
  (apply_common_transforms), which is now supported only on the stree lowering

This follows the direction set by the cartesian schedule-tree decision
(ADR: DaCe backends – schedule tree):
express the program's execution schedule at tree level and let ScheduleTreeRoot.as_sdfg()
produce the state machine, so that macro-level optimizations (loop merging, re-ordering)
can later operate on the tree instead of pattern-matching SDFGs.

Design

Direct GTIR → schedule-tree lowering (no intermediate TreeIR), in the new
runners/dace/lowering_stree/ module (+5144 lines), alongside the existing lowering/:

• gtir_to_stree.py — GTIRToScheduleTree visitor: statements, storage, let-lambdas,
  dispatch. All program storage is pre-allocated on root.containers/root.symbols;
  let-lambdas are inlined into the same root (dict-union shadowing, call stack = implicit
  push/pop) — no nested SDFGs produced by the lowering itself. (as_sdfg() may still
  introduce them downstream; see scan_carry_scalarization.py.)
• gtir_to_stree_codegen.py — StreePythonCodegen: each stencil body becomes one Python
  code string inside a TaskletNode. No library nodes — reductions are lowered
  directly to tasklet code.
• Scan is a ForScope inside a MapScope. For scalar leaves the carry lives in a
  scalar transient that is pre-initialized with init before the loop and read/written
  each iteration. List-typed leaves still read their carry from the result array at the
  previous index and therefore keep a carry if k > start else init ternary to guard the
  first iteration. Backward scans are handled as well.

The legacy lowering is untouched by this PR and remains the default.

Test coverage

• Six new backend IDs in tests/next_tests/definitions.py, each mapped to a dedicated
  DACE_STREE_SKIP_TEST_LIST in BACKEND_SKIP_TEST_MATRIX, so the full next test suite
  runs against the stree backends in the test_next (dace, *) nox sessions.
  Currently unsupported there (skipped): USES_LIFT, USES_SCAN_IN_STENCIL,
  USES_SPARSE_FIELDS, USES_TUPLE_ITERATOR.
• The six new backend entries carry the existing pytest.mark.uses_dace marker (GPU ones
  also requires_gpu), so the established nox matrix selection applies to them unchanged.
• Remaining test adjustments are expect/xfail bookkeeping.

Dependencies

This branch bundles the following transformation-layer fixes, which are still open as
standalone PRs: #2811, #2812, #2813, #2815, #2816, #2817, #2818. Each is a latent hazard
that the stree-lowered SDFGs — structured around regions and boundary states — trip and
that the legacy lowering never produced. They appear in this diff
(transformations/{simplify,state_fusion,redundant_array_removers,utils}.py and
test_distributed_buffer_relocator.py); once they merge individually, rebasing will drop
them from here.

The visitor-dispatch refactor this builds on, #2824, has already merged. Note that it
renamed gtir_dataflow.py → gtir_to_sdfg_lambda.py; gtir_to_sdfg.py kept its name.

[Copilot]

Pull request overview

This PR refactors the DaCe lowering pipeline for gt4py.next by renaming key lowering entry points and restructuring how builtin-function visitors are dispatched in both the fieldview and iterator lowering paths.

Changes:

• Renames/reshapes lowering entry points (e.g., build_sdfg_from_gtir → lower_program_to_sdfg) and adjusts imports/types accordingly.
• Refactors builtin dispatch in GTIRToSDFG (fieldview) and LambdaToDataflow (iterator) via centralized handler maps.
• Updates scan/primitives/type helpers to use the split fieldview/iterator lowering modules.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
src/gt4py/next/program_processors/runners/dace/lowering/gtir_to_sdfg_types.py	Updates local-view helpers to return iterator-based IteratorExpr/MemletExpr types.
src/gt4py/next/program_processors/runners/dace/lowering/gtir_to_sdfg_scan.py	Switches scan lowering to the new fieldview/iterator module split and updated dataflow types.
src/gt4py/next/program_processors/runners/dace/lowering/gtir_to_sdfg_primitives.py	Retypes primitives to the split builders and iterator dataflow edges; small typing-related adjustments.
src/gt4py/next/program_processors/runners/dace/lowering/gtir_to_sdfg_iterator.py	Refactors builtin dispatch in lambda-to-dataflow and retargets builder typing to fieldview interfaces.
src/gt4py/next/program_processors/runners/dace/lowering/gtir_to_sdfg_fieldview.py	Refactors builtin dispatch in program lowering; introduces _visit_* helpers; renames program lowering entry point.
src/gt4py/next/program_processors/runners/dace/lowering/gtir_to_sdfg_concat_where.py	Updates concat_where lowering to use the fieldview lowering module split.
src/gt4py/next/program_processors/runners/dace/lowering/init.py	Updates lowering package exports to point at the refactored lowering entry points.

Comments suppressed due to low confidence (3)

src/gt4py/next/program_processors/runners/dace/lowering/gtir_to_sdfg_iterator.py:58

• gtir_to_sdfg_iterator.py imports gtir_to_sdfg_fieldview using the src.gt4py... package path, which will fail at runtime outside the repository checkout layout. Also, since this module uses from __future__ import annotations, this import is only needed for type checking and can be placed under TYPE_CHECKING to avoid creating an import cycle (fieldview -> primitives -> iterator -> fieldview).
  src/gt4py/next/program_processors/runners/dace/lowering/gtir_to_sdfg_iterator.py:18
• The new TYPE_CHECKING-guarded import for gtir_to_sdfg_fieldview (recommended below) requires TYPE_CHECKING to be imported from typing in this module.
  src/gt4py/next/program_processors/runners/dace/lowering/init.py:23
• lowering/__init__.py re-exports build_sdfg_from_gtir from gtir_to_sdfg_fieldview, but that function was renamed to lower_program_to_sdfg (and build_sdfg_from_gtir no longer exists). This currently breaks imports like gtx_dace_lowering.build_sdfg_from_gtir(...) used elsewhere in the codebase.

from gt4py.next.program_processors.runners.dace.lowering.gtir_to_sdfg_fieldview import (
    build_sdfg_from_gtir,
)
from gt4py.next.program_processors.runners.dace.lowering.gtir_to_sdfg_utils import (
    flatten_tuple_fields,
    get_map_variable,
)


__all__ = [
    "build_sdfg_from_gtir",
    "flatten_tuple_fields",
    "get_map_variable",
]

[Copilot]

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

src/gt4py/next/program_processors/runners/dace/lowering/gtir_to_sdfg_fieldview.py:1231

• This comment line has a stray "to." at the end (likely meant to be "symbols."), which makes the sentence ungrammatical.

[Copilot]

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

src/gt4py/next/program_processors/runners/dace/lowering/gtir_to_sdfg_iterator.py:52

• gtir_to_sdfg_fieldview is imported here only to satisfy type annotations, but this creates an avoidable import edge (and contributes to circular imports: fieldview -> primitives -> iterator -> fieldview). Since this module uses from __future__ import annotations, you can move the import behind typing.TYPE_CHECKING (or otherwise defer it) to keep runtime imports acyclic.

[edopao]

cscs-ci run default

[edopao]

cscs-ci run default

[edopao]

cscs-ci run default

[edopao]

cscs-ci run default

[edopao]

cscs-ci run default

[edopao]

cscs-ci run default

[edopao]

cscs-ci run default

[edopao]

cscs-ci run default

[edopao]

cscs-ci run default

[edopao]

cscs-ci run default

[edopao]

cscs-ci run default

[edopao]

cscs-ci run default