Support for Python 3.12 generics

[jl-wynen]

Python 3.12 introduced generic functions and classes. In principle, they can replace our custom Scope. Doing so would fix #233. But this does not currently work well with our specialisation mechanism. For example:

import sciline as sl

type A = int
type B = int
class Raw[Run](float): ...
class Processed[Run](float): ...

def foo[Run](x: Raw[Run]) -> Processed[Run]:
    return Processed[Run](x * 2)

pl = sl.Pipeline([foo], params={Raw[A]: Raw[A](1.2)})

The above fails with

ValueError: Type variable Run has no constraints. Either constrain the type variable in its definition or via the 'constraints' argument of Pipeline.

Adding constraints via

pl = sl.Pipeline(
    [foo],
    params={Raw[A]: Raw[A](1.2)},
    constraints={Raw: (A, B)},
)

changes the error to

AttributeError: type object 'Raw' has no attribute '__constraints__'

Unfortunately, it is not as simple as fixing how constraints are extracted. The code above has 3 distinct TypeVars called Run:

for param in [
    Raw.__type_params__[0],
    Processed.__type_params__[0],
    foo.__annotations__["x"].__parameters__[0]
]:
    print(param, type(param), id(param), param == Raw.__type_params__[0]))

Run <class 'typing.TypeVar'> 140284653425744 True
Run <class 'typing.TypeVar'> 140284633285488 False
Run <class 'typing.TypeVar'> 140284632928704 False

As the example shows, the 3 TypeVars do not compare equal with each other. So our mechanism of matching TypeVars across providers does not work.

Alternative to class def

There is an alternative way to define generic type aliases:

type Raw[Run] = float

But it has the same problem; Run is a unique TypeVar, local to Raw.

Current workaround

It is already possible to make this work by explicitly listing constraints in every generic:

import sciline as sl

type A = int
type B = int

class Raw[Run: (A, B)](float): ...
class Processed[Run: (A, B)](float): ...

def foo[Run: (A, B)](x: Raw[Run]) -> Processed[Run]:
    return Processed[Run](x * 2)

pl = sl.Pipeline(
    [foo],
    params={Raw[A]: Raw[A](1.2)},
)

But this does not scale well. And it does not work with constraints defined in technique packages.

Possible solution

Since we cannot rely on __eq__ or object identity between type vars, we would have to use names to apply constraints and match variables across providers.

We originally did not want to do this because there can be clashes. However, with our current experience, we don't have this problem. We are keeping names unique and import types from lower-level packages into higher-level ones. So maybe relying on names is ok?

If we go with this, we should also change the mechanism for TypeVar to use the names. It would be very confusing, otherwise. (But what about non-generic types?)

---

For comparison, here is a version that currently works using a global TypeVar:

from typing import TypeVar

import sciline as sl

type A = int
type B = int
Run = TypeVar('Run', A, B)

class Raw(sl.Scope[Run, float], float): ...
class Processed(sl.Scope[Run, float], float): ...

def foo(x: Raw[Run]) -> Processed[Run]:
    return Processed[Run](x * 2)

for param in [
    Raw.__parameters__[0],
    Processed.__parameters__[0],
    foo.__annotations__["x"].__parameters__[0]
]:
    print(param, type(param), id(param), param == Raw.__parameters__[0])

~Run <class 'typing.TypeVar'> 139761038726224 True
~Run <class 'typing.TypeVar'> 139761038726224 True
~Run <class 'typing.TypeVar'> 139761038726224 True

[SimonHeybrock]

One possible solution, prototyped in #236 for discussion: drop the constraints requirement and instantiate generic providers on demand.

The observation behind it: sciline only needs TypeVar identity across definitions for the constraints lookup. Within a single provider, PEP 695's scoped type variables are self-consistent (all annotations of foo above share one Run object), which is all the per-provider binding mechanism needs. So if generic providers are kept as templates and instantiated by unifying their type patterns with the concrete keys the pipeline sees (params, requested targets, mapped keys), the constraints machinery is not needed at all, and the example from this issue works verbatim — for PEP 695 classes, type X[T] = ... aliases, and old-style unconstrained TypeVars alike.

This is a bigger semantic change than matching constraints by name (errors move from insertion to graph build, and ambiguity/shadowing rules need agreeing on), so the PR lists the trade-offs and open questions rather than proposing a decision.