FIX add_weighted_adapter SVD combination types for Conv1d and Conv3d LoRA layers

[Nicholas022400701]

Resolves #3769

LoraModel._svd_generalized_task_arithmetic_weighted_adapter in src/peft/tuners/lora/model.py flattens the delta weight to 2D only when the target is a Conv2d. For Conv1d and Conv3d LoRA layers the 3D or 5D delta weight goes into torch.linalg.svd as a batch of matrices and the code after it fails (RuntimeError: size mismatch, got input (80), mat (80x10), vec (3) for Conv1d, diag(): Supports 1D or 2D tensors. Got 4D for Conv3d). This affects combination_type="svd" and the other SVD based types (ties_svd, dare_linear_svd, dare_ties_svd, magnitude_prune_svd); cat and linear already work for these layers.

The check is now isinstance(target, _ConvNd) and the delta weight is always flattened with delta_weight.flatten(start_dim=1), which turns (out, in, *kernel) into (out, in * prod(kernel)) for every conv type. For a 1x1 Conv2d this gives the same matrix as the previous squeeze(), so nothing changes for the layers that worked before. The reshape of U and Vh back to the lora_B and lora_A shapes at the end already works for any conv type.

Tests: test_add_weighted_adapter_svd_conv_layers in tests/test_custom_models.py, parametrized over Conv1d, Conv2d 3x3, Conv2d 1x1 and Conv3d with 10 input channels, two rank 4 adapters and svd_rank=8, so that the SVD is exact and the merged delta weight has to equal the weighted sum of the two adapters. On main the Conv1d and Conv3d cases fail with the two errors above and the Conv2d cases pass; with this change all four pass. pytest tests/test_custom_models.py -k weighted_adapter: 71 passed (the 67 existing add_weighted_adapter tests plus the 4 new ones). make style clean with ruff 0.16.4.

AI disclosure: I used an AI coding agent to help write this patch, the tests and this description. I have read the change and the tests myself and I will answer review comments personally.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.