[BUG] 21cmMC fails to import with py21cmfast v4 (ModuleNotFoundError: py21cmfast._utils)

[ayansark147]

I am opening this issue to report a hard compatibility break between the current version of 21cmMC and the recent major release of py21cmfast (v4).
With the recent update of 21cmFAST to v4, the _utils.py has moved to the wrapper folder, thereby making most of the functions in 21cmMC erroneous as they use py21cmfast._utils

Steps to reproduce the behavior:

1. Install the latest version of py21cmfast (v4.x.x) and 21cmMC.

2. Attempt to import the core MCMC modules:4. Python

from py21cmmc import analyse
from py21cmmc import mcmc

Expected behavior:
21cmMC should successfully import and interface seamlessly with the v4 backend without triggering internal module import errors.

Actual Behavior:

The pipeline fails immediately upon import. Here is the traceback:

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
Cell In[3], line 14
     10 
     11 # Now the imports will work seamlessly!
     12 get_ipython().run_line_magic('matplotlib', 'inline')
     13 import numpy as np
---> 14 from py21cmmc import analyse
     15 from py21cmmc import mcmc
     16 import py21cmmc as p21mc

File ~/.venv/lib/python3.13/site-packages/py21cmmc/__init__.py:4
      1 """21CMMC: a package for running MCMC analyses using 21cmFAST."""
      3 __version__ = "1.0.0dev3"
----> 4 from .analyse import get_samples, load_primitive_chain
      5 from .core import (
      6     CoreCoevalModule,
      7     CoreLightConeModule,
   (...)
     10     NotSetupError,
     11 )
     12 from .cosmoHammer import HDFStorageUtil

File ~/.venv/lib/python3.13/site-packages/py21cmmc/analyse.py:11
      8 from pathlib import Path
      9 from py21cmfast import yaml
...
---> 15 from py21cmfast._utils import ParameterError
     17 from py21cmmc.ensemble import EnsembleSampler
     19 logger = getLogger()

ModuleNotFoundError: No module named 'py21cmfast._utils' 

Details:

• OS: Windows WSL2/ Ubuntu 22.04
• Python Version: 3.13.0
• py21cmfast version: 4.2
• 21cmMC: 1.0.0dev3

Additional context
Given the massive performance and structural upgrades in py21cmfast v4 (and the shift toward 21cmEMU), are there any ongoing branches or roadmaps for updating 21cmMC to support the v4 ecosystem? Let me know if there is anything I can test to help out!

[steven-murray]

Hi @ayansark147 you're absolutely right -- development on 21cmMC has been.... let's say slow lately. It needs a huge revamp to get it to work with 21cmfast v4. It is definitely something we're thinking of doing, it is just not currently high enough on the priority list to pull my (or others) dev time. The ultimate goal would be to have 21cmEMU hook into 21cmMC as well.

If you're interested in helping out on this, let us know!

[ayansark147]

Hi @ayansark147 you're absolutely right -- development on 21cmMC has been.... let's say slow lately. It needs a huge revamp to get it to work with 21cmfast v4. It is definitely something we're thinking of doing, it is just not currently high enough on the priority list to pull my (or others) dev time. The ultimate goal would be to have 21cmEMU hook into 21cmMC as well.

If you're interested in helping out on this, let us know!

I would be absolutely stoked to help out. Please do let me know what I can do help however, since I have no idea where to begin.

[ayansark147]

Hi @ayansark147 you're absolutely right -- development on 21cmMC has been.... let's say slow lately. It needs a huge revamp to get it to work with 21cmfast v4. It is definitely something we're thinking of doing, it is just not currently high enough on the priority list to pull my (or others) dev time. The ultimate goal would be to have 21cmEMU hook into 21cmMC as well.
If you're interested in helping out on this, let us know!

I would be absolutely stoked to help out. Please do let me know what I can do help however, since I have no idea where to begin.

@steven-murray Please do let me know when convenient about the next steps

[steven-murray]

@DanielaBreitman and @ayansark147 perhaps we should make a time to call over zoom to discuss this?

[DanielaBreitman]

Hi everyone, this is kind of my fault. I have the new 21cmEMUv3 as well as 21cmFASTv4 (though maybe not the most recent version) already working with 21cmMC. I've just been really slow to make a PR. I will put it higher in my list to make a PR with the 21cmEMUv3 and pspec likelihood. 21cmFASTv4's API has been changing a lot recently. We need to choose a version e.g. 4.2 and make another PR for that. I am happy to meet to chat about this.

[ayansark147]

Yeah, I am down to meet as well, please let me know when.