uv versus micromamba in CI

[kmuehlbauer]

As @aladinor brought up in #373 (comment) and #373 (comment), it might be time to check consider a change of CI tooling. @openradar/xradar it would be great to get more views on that topic.

Here is @aladinor's Pro/Con writeup:

@kmuehlbauer Fair question — here's a concrete pro/con write-up:

Pros

1. Speed. uv resolves + installs in seconds where micromamba takes minutes. For a typical xradar CI job we'd save ~2 min × 5 jobs × every PR run.

2. No CDN propagation lag (live example from this week). PR #377 needed open-radar-data 0.8.0. PyPI had it instantly; conda-forge took ~50 min to mirror, during which every CI job in PRs #377, #347, and others failed at env resolution. uv hits PyPI directly — zero propagation lag.

3. Deterministic lockfile. uv.lock is platform-independent, deterministic across machines, and committed to the repo. Conda-lock exists but is fragile (channel changes invalidate it) and isn't currently in xradar.

4. PyPI is the source of truth in 2026 for our deps. Every xradar dependency has manylinux wheels on PyPI today: h5py, netCDF4, pyproj, dask, numpy, scipy. The original "conda for binary deps" reason is gone. We already pip-install xarray>=2026.4.0 inside the conda env — i.e., the env is already half-pip.

5. One tool. uv replaces pip, pip-tools, virtualenv, pyenv, conda, conda-lock, and twine. New contributors get a single curl … | sh install and run uv sync. No micromamba create -n xradar-unit-tests -f ci/unittests.yml ... ceremony.

6. Reproducible local dev parity. Today, contributors run pip install -e .[dev] locally but CI uses conda envs from ci/*.yml — two install paths, two failure modes. With uv, local and CI use the same uv.lock.

Cons / things to watch

1. Loss of conda ecosystem familiarity. Scientific-Python users default to conda. We'd need to update the contributor docs. Mitigation: keep environment.yml as a fallback for users who prefer mamba; CI just doesn't use it.

2. Wheels for niche platforms. uv currently has best support on Linux/macOS/Windows x86_64 + arm64. If we ever need a platform that PyPI doesn't ship wheels for, conda-forge would be a workaround. Not a current concern.

3. Migration effort. ~1 PR: add pyproject.toml [tool.uv] section + generate uv.lock, replace mamba-org/setup-micromamba in .github/workflows/ci.yml with astral-sh/setup-uv, rewrite ci/*.yml as uv sync --extra dev calls. Probably half a day; another half day to update notebooks/docs.

Suggested phased approach

1. PR 1: add uv config alongside the existing conda envs (no CI changes). Verify uv sync produces a working local dev env equivalent to the conda one.
2. PR 2: add a uv-based CI job alongside the existing micromamba jobs, behind a workflow_dispatch trigger. Run both in parallel for a week.
3. PR 3: if uv is faster and stable, swap micromamba → uv in .github/workflows/ci.yml. Keep environment.yml as a fallback for users who prefer it.
4. PR 4 (optional, later): retire ci/*.yml if no one uses them.

The recent #377 50-min wait would have been a 0-min wait under uv. I think that alone justifies trying the phased migration.

Originally posted by @aladinor in #373

[kmuehlbauer]

Thanks @aladinor, for the first overview.

My take is, uv optimizes for speed and simplicity, micromamba (conda-forge ecosystem) optimizes for binary compatibility and ecosystem stability. For scientific Python (around compiled dependencies), conda-forge is still the more conservative and battle-tested choice. Conda-forge gives us coordinated rebuilds, ABI consistency, and fewer surprises from missing/broken wheel or source-build fallbacks.

I acknowledge that xradar’s dependencies are now well covered by wheels on PyPI. Still, wheel clashes and binary regressions (especially around packages like netCDF4 and h5py) happen occasionally. xradar only ships a pure-Python wheel, so the challenge is not distributing xradar, but ensuring that the compiled dependency stack for downstream users remains compatible across platforms and Python versions. That's why Py-ART and wradlib adopted conda-forge in the first place and still suggesting for that reason.

That said, I'm not advocating to stick with conda-forge based environments forever. I just think we should be explicit about the tradeoffs and make sure the wheel-first setup is actually solid for both CI and downstream users before fully committing. A week of parallel CI runs should be enough to check the basics (correctness, install reliability, CI speed, caching), even if it won’t catch longer-term ecosystem drift. I’d also keep a conda-forge CI job around with the environment file as a fallback and sanity check against the wider scientific Python stack.

[mgrover1]

I wonder what folks' thoughts are on using pixi?
https://pixi.prefix.dev/latest/

It seems like a good compromise between the two, with solid speed and reproducibility

[syedhamidali]

I think we should use all! 😁

[kmuehlbauer]

Thanks @mgrover1 for adding pixi to the list. It's used by xarray already in CI for several months.

I do not have hands-on experience, but it combines conda-forge and PyPI. It also uses UV under the hood for PyPI based packages.

[mgrover1]

I like the idea of staying in-line with xarray maintenance practices :)