fix: support pandas 3 and anndata 0.13#16
Conversation
CI green on all three legsRun
The 3.12 leg still resolves pandas Worth stating explicitly, because it is the whole question with a CI fix: passed went 1981 → 2021, up by exactly the 40 that were failing, while skipped stayed at 34. Nothing was skipped, xfailed, or loosened, and no matrix entry was dropped — a suppression would have moved the skipped count or removed a leg. The 40 were fixed. Also verified backward-compatible: the affected modules give an identical |
The Python 3.12 CI leg resolves pandas 3.0 and anndata 0.13 (their wheels are 3.12-only), which broke 40 tests. Both releases changed contracts that kompot relied on: - pandas 3 stores plain string columns as StringDtype rather than object. np.issubdtype() cannot interpret pandas extension dtypes and raises TypeError, so every dtype branch in group_utils went down. Detect dtypes with pandas.api.types instead. - pandas 3 dropped the positional fallback in Series.__getitem__, so group_value[0] raised KeyError on a Series with a non-integer index. Index positionally with .iloc. - volcano_de classified a background colour column as categorical by testing dtype == "object", which no longer holds for string columns; it then fed strings to a continuous Normalize. Treat any non-numeric dtype as categorical. - anndata 0.13 exposes X as layers[None], so iterating adata.layers now also yields None. Add kompot.anndata.utils.layer_names() and use it wherever layer names are enumerated. Verified against both pandas 3.0.3/anndata 0.13.1 and pandas 2.3.3/anndata 0.12.19: 241 passed, 3 skipped in the affected modules on each.
settylab-dotto-bot
Bot
force-pushed
the
dominik/pandas3-anndata013-compat
branch
from
4de3cef to
761af00
Compare
|
Independent skeptic validation:
It found one real defect, now corrected: the CHANGELOG claimed "Behaviour on pandas 2 / anndata 0.12 is unchanged." That is not strictly true. Treating any non-numeric dtype as categorical also changes the old stack: a The change is intentional (it aligns nullable booleans with numpy booleans), but it is user-visible, so the CHANGELOG now names it as a deliberate exception rather than denying it. Amended onto the branch as |
1 participant
Why
The
test (3.12)CI leg is red onmainand on every open PR (40 failed, 1981 passed). It is not caused by any recent change: pandas3.0.3and anndata0.13.1ship 3.12-only wheels, so the 3.12 matrix entry resolves them while 3.10/3.11 stay on pandas2.3.3/ anndata0.12.19.pyproject.tomldeclarespandas>=1.3.0with no upper bound, so the new majors were picked up the moment they were released.Verified by checking out
main(a6702b2) into a clean tree and running the same tests against pandas 3.0.3 / anndata 0.13.1: identical 20/20 failures. This branch is the fix, not a workaround.What actually broke
Four independent contract changes, all real incompatibilities in kompot:
np.issubdtypecannot interpret pandas extension dtypes. pandas 3 stores plain string columns asStringDtypeinstead ofobject, sonp.issubdtype(col.dtype, np.number)raisesTypeError: Cannot interpret '<StringDtype(...)>' as a data type. This took out every dtype branch ingroup_utils— i.e.groups="cell_type", the single most common way to group. Now dispatched throughpandas.api.types.Series.__getitem__lost its positional fallback.group_value[0]raisedKeyError: 0for apd.Seriesgroup with a non-integer index. Now indexed with.iloc.volcano_demisclassified categorical backgrounds. It testeddtype == "object"to decide categorical-vs-continuous; a pandas-3 string column isStringDtype, so it fell through to the continuous path and handed'category_1'tompl.colors.Normalize(ValueError: could not convert string to float). Any non-numeric dtype is now treated as categorical.anndata 0.13 exposes
Xaslayers[None]. Iteratingadata.layerstherefore also yieldsNone, which broke_detect_condition_label('NoneType' object has no attribute 'startswith') and madelen(adata.layers)countX. Addedkompot.anndata.utils.layer_names()and used it wherever layer names are enumerated.No suppression
No test was skipped, xfailed, or loosened, and no matrix entry was dropped. The two test edits swap bare
adata.layersiteration forlayer_names(adata)— that is the same set of keys on anndata < 0.13, and the correct set on 0.13, whereXwould otherwise be counted as a layer.Verification
Affected modules, run locally against both dependency sets on Python 3.12:
Backward compatible; no new pin or lower bound is required.
Unblocks
PR
#15(tutorials) is red for exactly this reason and is not at fault. Once this lands onmain, re-running#15's checks turns it green without touching that branch — GitHub tests the PR merged intomain.