Darwin/LLVM support by theurich · Pull Request #558 · esmf-org/esmf

theurich · 2026-06-16T17:47:28Z

Improve Darwin/LLVM support by recognizing Apple Clang vs LLVM Clang, handle OpenMP, OpenACC, and linking differences.

mathomp4 · 2026-06-16T18:56:02Z

@theurich I'm testing now. First thought, it's...verbose:

"The detected C++ compiler is Apple Clang."
cd /Users/mathomp4/Baselibs/ESMA-Baselibs-9.12.0-ESMFFlang/src/esmf ;\
	/Applications/Xcode.app/Contents/Developer/usr/bin/make ACTION=tree_include tree
"The detected C++ compiler is Apple Clang."
tree_include in: /Users/mathomp4/Baselibs/ESMA-Baselibs-9.12.0-ESMFFlang/src/esmf/src
"The detected C++ compiler is Apple Clang."
tree_include in: /Users/mathomp4/Baselibs/ESMA-Baselibs-9.12.0-ESMFFlang/src/esmf/src/prologue
"The detected C++ compiler is Apple Clang."
tree_include in: /Users/mathomp4/Baselibs/ESMA-Baselibs-9.12.0-ESMFFlang/src/esmf/src/prologue/tests
"The detected C++ compiler is Apple Clang."
tree_include in: /Users/mathomp4/Baselibs/ESMA-Baselibs-9.12.0-ESMFFlang/src/esmf/src/prologue/yaml-cpp
"The detected C++ compiler is Apple Clang."
tree_include in: /Users/mathomp4/Baselibs/ESMA-Baselibs-9.12.0-ESMFFlang/src/esmf/src/prologue/yaml-cpp/src
"The detected C++ compiler is Apple Clang."

Obviously for testing it's fine. The build is slowly running now.

😄

mathomp4 · 2026-06-16T19:24:21Z

Update. It got close:

"The detected C++ compiler is Apple Clang."
mpicc    -pthread -L/Users/mathomp4/installed/MPI/appleclang-flang-22/openmpi-5.0.10/Baselibs/9.21.0-ESMFFlang/Darwin/lib -L/Users/mathomp4/installed/MPI/appleclang-flang-22/openmpi-5.0.10/Baselibs/9.21.0-ESMFFlang/Darwin/lib -L/Users/mathomp4/installed/Compiler/appleclang-flang-22/openmpi/5.0.10/lib -L/Users/mathomp4/installed/Compiler/appleclang-flang-22/openmpi/5.0.10/lib -L/Users/mathomp4/.homebrew/brew/opt/libomp/lib -L/Users/mathomp4/.homebrew/brew/opt/llvm/lib/clang/22/lib/darwin -L/Users/mathomp4/.homebrew/brew/Cellar/flang/22.1.7/lib -L/Users/mathomp4/installed/MPI/appleclang-flang-22/openmpi-5.0.10/Baselibs/9.21.0-ESMFFlang/Darwin/lib -L/Users/mathomp4/installed/MPI/appleclang-flang-22/openmpi-5.0.10/Baselibs/9.21.0-ESMFFlang/Darwin/lib -L/Users/mathomp4/installed/Compiler/appleclang-flang-22/openmpi/5.0.10/lib -L/Users/mathomp4/installed/Compiler/appleclang-flang-22/openmpi/5.0.10/lib -L/Users/mathomp4/.homebrew/brew/opt/libomp/lib -L/Users/mathomp4/.homebrew/brew/opt/llvm/lib/clang/22/lib/darwin -L/Users/mathomp4/.homebrew/brew/Cellar/flang/22.1.7/lib -Wl,-rpath,/Users/mathomp4/installed/MPI/appleclang-flang-22/openmpi-5.0.10/Baselibs/9.21.0-ESMFFlang/Darwin/lib -Wl,-rpath,/Users/mathomp4/installed/MPI/appleclang-flang-22/openmpi-5.0.10/Baselibs/9.21.0-ESMFFlang/Darwin/lib -Wl,-rpath,/Users/mathomp4/installed/Compiler/appleclang-flang-22/openmpi/5.0.10/lib -Wl,-rpath,/Users/mathomp4/installed/Compiler/appleclang-flang-22/openmpi/5.0.10/lib -Wl,-rpath,/Users/mathomp4/.homebrew/brew/opt/libomp/lib -Wl,-rpath,/Users/mathomp4/.homebrew/brew/opt/llvm/lib/clang/22/lib/darwin -Wl,-rpath,/Users/mathomp4/.homebrew/brew/Cellar/flang/22.1.7/lib -Wl,-rpath,/Users/mathomp4/installed/MPI/appleclang-flang-22/openmpi-5.0.10/Baselibs/9.21.0-ESMFFlang/Darwin/lib -Wl,-rpath,/Users/mathomp4/installed/MPI/appleclang-flang-22/openmpi-5.0.10/Baselibs/9.21.0-ESMFFlang/Darwin/lib -Wl,-rpath,/Users/mathomp4/installed/Compiler/appleclang-flang-22/openmpi/5.0.10/lib -Wl,-rpath,/Users/mathomp4/installed/Compiler/appleclang-flang-22/openmpi/5.0.10/lib -Wl,-rpath,/Users/mathomp4/.homebrew/brew/opt/libomp/lib -Wl,-rpath,/Users/mathomp4/.homebrew/brew/opt/llvm/lib/clang/22/lib/darwin -Wl,-rpath,/Users/mathomp4/.homebrew/brew/Cellar/flang/22.1.7/lib -o /Users/mathomp4/installed/MPI/appleclang-flang-22/openmpi-5.0.10/Baselibs/9.21.0-ESMFFlang/Darwin/bin/ESMF_PrintInfoC /Users/mathomp4/Baselibs/ESMA-Baselibs-9.12.0-ESMFFlang/src/esmf/obj/objg/Darwin.llvm.64.openmpi.default/src/apps/ESMF_PrintInfoC/ESMF_PrintInfoC.o -lesmf -lpioc -L/Users/mathomp4/installed/MPI/appleclang-flang-22/openmpi-5.0.10/Baselibs/9.21.0-ESMFFlang/Darwin/lib -lnetcdff -lnetcdf -L/Users/mathomp4/installed/MPI/appleclang-flang-22/openmpi-5.0.10/Baselibs/9.21.0-ESMFFlang/Darwin/lib -lnetcdff -lnetcdf -lm -L/Users/mathomp4/installed/MPI/appleclang-flang-22/openmpi-5.0.10/Baselibs/9.21.0-ESMFFlang/Darwin/lib -L/Users/mathomp4/installed/MPI/appleclang-flang-22/openmpi-5.0.10/Baselibs/9.21.0-ESMFFlang/Darwin/lib -lnetcdf -ljpeg -lmfhdf -ldf -ljpeg -lhdf5_hl -lhdf5 -lm -lbz2 -lxml2 -L/Users/mathomp4/installed/MPI/appleclang-flang-22/openmpi-5.0.10/Baselibs/9.21.0-ESMFFlang/Darwin/lib -lsz -laec -ljpeg -L/Users/mathomp4/.homebrew/brew/opt/openssl@3/lib -L/Users/mathomp4/installed/MPI/appleclang-flang-22/openmpi-5.0.10/Baselibs/9.21.0-ESMFFlang/Darwin/lib -lcurl -lssl -lcrypto -lssl -lcrypto -lz -lm -Wl,-framework,CoreFoundation -Wl,-framework,SystemConfiguration -ldl -lm -L/Users/mathomp4/.homebrew/brew/opt/openssl@3/lib -L/Users/mathomp4/installed/MPI/appleclang-flang-22/openmpi-5.0.10/Baselibs/9.21.0-ESMFFlang/Darwin/lib -lcurl -lssl -lcrypto -lssl -lcrypto -lz -lm -Wl,-framework,CoreFoundation -Wl,-framework,SystemConfiguration -lto_library /Users/mathomp4/.homebrew/brew/Cellar/flang/22.1.7/lib/libLTO.dylib -lto_library /Users/mathomp4/.homebrew/brew/opt/llvm/lib/libLTO.dylib -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi -lflang_rt.runtime -lSystem -lpioc -L/Users/mathomp4/installed/MPI/appleclang-flang-22/openmpi-5.0.10/Baselibs/9.21.0-ESMFFlang/Darwin/lib -lnetcdff -lnetcdf -L/Users/mathomp4/installed/MPI/appleclang-flang-22/openmpi-5.0.10/Baselibs/9.21.0-ESMFFlang/Darwin/lib -lnetcdff -lnetcdf -lm -L/Users/mathomp4/installed/MPI/appleclang-flang-22/openmpi-5.0.10/Baselibs/9.21.0-ESMFFlang/Darwin/lib -L/Users/mathomp4/installed/MPI/appleclang-flang-22/openmpi-5.0.10/Baselibs/9.21.0-ESMFFlang/Darwin/lib -lnetcdf -ljpeg -lmfhdf -ldf -ljpeg -lhdf5_hl -lhdf5 -lm -lbz2 -lxml2 -L/Users/mathomp4/installed/MPI/appleclang-flang-22/openmpi-5.0.10/Baselibs/9.21.0-ESMFFlang/Darwin/lib -lsz -laec -ljpeg -L/Users/mathomp4/.homebrew/brew/opt/openssl@3/lib -L/Users/mathomp4/installed/MPI/appleclang-flang-22/openmpi-5.0.10/Baselibs/9.21.0-ESMFFlang/Darwin/lib -lcurl -lssl -lcrypto -lssl -lcrypto -lz -lm -Wl,-framework,CoreFoundation -Wl,-framework,SystemConfiguration -ldl -lm -L/Users/mathomp4/.homebrew/brew/opt/openssl@3/lib -L/Users/mathomp4/installed/MPI/appleclang-flang-22/openmpi-5.0.10/Baselibs/9.21.0-ESMFFlang/Darwin/lib -lcurl -lssl -lcrypto -lssl -lcrypto -lz -lm -Wl,-framework,CoreFoundation -Wl,-framework,SystemConfiguration -lc++
clang: error: no such file or directory: '/Users/mathomp4/.homebrew/brew/Cellar/flang/22.1.7/lib/libLTO.dylib'
make[7]: *** [/Users/mathomp4/installed/MPI/appleclang-flang-22/openmpi-5.0.10/Baselibs/9.21.0-ESMFFlang/Darwin/bin/ESMF_PrintInfoC] Error 1
make[6]: *** [tree] Error 1
make[5]: *** [build_apps] Error 2
make[4]: *** [install_apps] Error 2
make[3]: *** [install] Error 2
make[2]: *** [install] Error 2
make[1]: *** [esmf.install] Error 2

Here is what is in that directory:

❯ ls /Users/mathomp4/.homebrew/brew/Cellar/flang/22.1.7/lib
 clang/                libFIRAnalysis.dylib         libFIRDialect.dylib           libFIROpenACCTransforms.dylib   libflang_rt.runtime.dylib@       libflangPasses.dylib       libFortranParser.dylib      libHLFIRDialect.dylib
 cmake/                libFIRBuilder.dylib          libFIRDialectSupport.dylib    libFIROpenMPSupport.dylib       libflangFrontend.dylib           libFortranDecimal.dylib    libFortranSemantics.dylib   libHLFIRTransforms.dylib
 libCUFAttrs.dylib     libFIRCodeGen.dylib          libFIROpenACCAnalysis.dylib   libFIRSupport.dylib             libflangFrontendTool.dylib       libFortranEvaluate.dylib   libFortranSupport.dylib     libMIFDialect.dylib
 libCUFDialect.dylib   libFIRCodeGenDialect.dylib   libFIROpenACCSupport.dylib    libFIRTransforms.dylib          libFlangOpenMPTransforms.dylib   libFortranLower.dylib      libFortranUtils.dylib

NOTE: This still could be on me. I didn't do a fresh clone of ESMF, just did a rebuild after switching to this branch. I can try a fresh clean build if you think that might help

theurich · 2026-06-16T20:25:23Z

@mathomp4 at your convenience please provide the output you get with make info on your setup. Thanks.

mathomp4 · 2026-06-17T11:24:41Z

@theurich I think this is it:

makeinfo.log

Note: I will note in my Baselibs I have:

# Dynamically discover Homebrew's flang prefix, regardless of install location
# HDF4 and HDF5 configure script trips over flang's LTO flags on macOS
ifeq ($(ARCH)-$(findstring flang,$(notdir $(FC))),Darwin-flang)
   # Force the Fortran library paths to bypass the broken auto-detection
   FLANG_BREW_PREFIX := $(shell command -v brew >/dev/null 2>&1 && brew --prefix flang)

   ifneq ($(FLANG_BREW_PREFIX),)
      FLANG_LTO_LIBS = FCLIBS="-L$(FLANG_BREW_PREFIX)/lib -lflang_rt.runtime" FLIBS="-L$(FLANG_BREW_PREFIX)/lib -lflang_rt.runtime"
   endif
   export FLANG_LTO_LIBS
endif

so I must have hit something similar with the HDF Group. Of course, as it says, this is sort of "brew flang" specific.

And then in my first attempt at a Darwin.llvm I have:

+# Dynamically find Flang's lib dir to bypass broken libs.llvm script
+FLANG_PREFIX := $(shell brew --prefix flang 2>/dev/null || echo /opt/homebrew/opt/flang)
+
+############################################################
+# Link against libesmf.a using the F90 linker front-end
+#
+ESMF_F90LINKLIBS += -L$(FLANG_PREFIX)/lib -lflang_rt.runtime -lc++
+
+############################################################
+# Link against libesmf.a using the C++ linker front-end
+#
+ESMF_CXXLINKLIBS += -L$(FLANG_PREFIX)/lib -lflang_rt.runtime -lc++ -lm

Which, huh, is sort of the same thing? Not sure.

under Darwin Homebrew Flang installations.

theurich · 2026-06-18T00:18:17Z

@mathomp4 I pushed a change to this PR that should hopefully address the issue with the Homebrew Flang installation as I understand it at the moment. Please give it another try and let me know. Thanks.

mathomp4 · 2026-06-22T12:34:27Z

@mathomp4 I pushed a change to this PR that should hopefully address the issue with the Homebrew Flang installation as I understand it at the moment. Please give it another try and let me know. Thanks.

@theurich Sorry for the late reply. I'll test today when I can. I got caught in a Homebrew update that removed my gcc@15 install (comedy of errors). Since I can't use bottles, I need to build gcc from source each time I upgrade/install. Whee.

mathomp4 · 2026-06-22T20:07:41Z

@theurich Okay. It seems to have built. Huzzah! I did a check and got:

SYSTEM TESTS SUMMARY
Found 51 multi-processor system tests, 51 passed and 0 failed.



UNIT TESTS SUMMARY
Found 8801 non-exhaustive multi-processor unit tests, 7083 passed and 1718 failed.

So...less yay? I'll try and build MAPL tomorrow

theurich · 2026-06-22T21:04:57Z

@mathomp4 - well, a step forward at least. Do I recall that you had cleaner test results with your Darwin.llvm configuration? Maybe I am confusing something though, I don't find the issue/comment now.

As for making further progress, do you think you could set up gs6101-alderaan.ndc.nasa.gov to report results to the ESMF artifacts repo? I don't think it needs to run on a regular basis, just would be convenient to have the reports uploaded when you manually make a test run. Let me know what you think and we can iterate on it.

mathomp4 · 2026-06-22T21:49:06Z

@mathomp4 - well, a step forward at least. Do I recall that you had cleaner test results with your Darwin.llvm configuration? Maybe I am confusing something though, I don't find the issue/comment now.

As for making further progress, do you think you could set up gs6101-alderaan.ndc.nasa.gov to report results to the ESMF artifacts repo? I don't think it needs to run on a regular basis, just would be convenient to have the reports uploaded when you manually make a test run. Let me know what you think and we can iterate on it.

@theurich Maaaaaybe? I might need you or @oehmke to remind me how to add machines to the framework. Was this the link to look at:

https://github.com/esmf-org/esmf-test-scripts/tree/main/python_scripts#readme

theurich · 2026-06-22T21:53:43Z

Yes, that is the correct link. Would alderaan be the appropriate machine name for this purpose? If so, let me add a branch under the artifacts repo for it. That will make the initial checkout procedure a lot smoother - since the repo is pretty large at this point.

mathomp4 · 2026-06-22T21:57:03Z

Yes, that is the correct link. Would alderaan be the appropriate machine name for this purpose? If so, let me add a branch under the artifacts repo for it. That will make the initial checkout procedure a lot smoother - since the repo is pretty large at this point.

@theurich I have access to three macs. Alderaan is an M1, dagobah an M2, and my personal is an M3.

I suppose the M1 might be "best" in that it's the "lowest" of the three? So yes, let's go with alderaan.

theurich · 2026-06-22T22:12:29Z

Okay, branch is added, use the

git clone -c feature.manyFiles=true git@github.com:esmf-org/esmf-test-artifacts.git --branch alderaan --single-branch

option to clone and checkout the alderaan branch for the artifacts. Otherwise just follow the instructions to add new config. Let me know if you run into any issues.

Thanks for doing this. I think it will make life a lot easier in the long run to have the test artifacts from your Darwin machine available for review.

mathomp4 · 2026-06-23T15:31:19Z

@theurich Well, some good news, some bad. I think I got a semi-good setup for alderaan. But it dies in the build:

mpifort  -dynamiclib -o /Users/mathomp4/ESMF_Testing/llvm_22.1_openmpi_O_feature_darwin-work/esmf/lib/libO/Darwin.llvm.64.openmpi.default/libesmftrace_preload.dylib preload.o preload_mpi.o wrappers.o wrappers_mpi.o    -pthread -L/Users/mathomp4/ESMF_Testing/llvm_22.1_openmpi_O_feature_darwin-work/esmf/lib/libO/Darwin.llvm.64.openmpi.default  -L/Users/mathomp4/installed/Compiler/appleclang-flang-22/openmpi/5.0.10/lib -L/Users/mathomp4/installed/Compiler/appleclang-flang-22/openmpi/5.0.10/lib -Lopt/libomp/lib -L/Users/mathomp4/.homebrew/brew/opt/llvm/lib/clang/22/lib/darwin -L/Users/mathomp4/.homebrew/brew/Cellar/flang/22.1.8/lib -Wl,-rpath,/Users/mathomp4/ESMF_Testing/llvm_22.1_openmpi_O_feature_darwin-work/esmf/lib/libO/Darwin.llvm.64.openmpi.default  -Wl,-rpath,/Users/mathomp4/ESMF_Testing/llvm_22.1_openmpi_O_feature_darwin-work/esmf/lib/libO/Darwin.llvm.64.openmpi.default -Wl,-rpath,/Users/mathomp4/installed/Compiler/appleclang-flang-22/openmpi/5.0.10/lib -Wl,-rpath,/Users/mathomp4/installed/Compiler/appleclang-flang-22/openmpi/5.0.10/lib -Wl,-rpath,opt/libomp/lib -Wl,-rpath,/Users/mathomp4/.homebrew/brew/opt/llvm/lib/clang/22/lib/darwin -Wl,-rpath,/Users/mathomp4/.homebrew/brew/Cellar/flang/22.1.8/lib  -lc++ -lesmf
flang-22: error: unknown argument: '-dynamiclib'
make[2]: *** [tracelib_preload] Error 1
make[1]: *** [build_tracelibs] Error 2
make: *** [lib] Error 2

I pushed artifacts up for alderaan in this case. Maybe you can see something.

Note: I guess your test framework must build differently than I do in Baselibs, though I also only build ESMF_BOPT=g in my Baselibs testing. Still, this shouldn't care about optimization

theurich · 2026-06-23T16:26:33Z

I just pushed a change to the add_alderaan branch on esmf-test-scripts. I think the important difference with previous builds was missing of ESMF_TRACE_LIB_BUILD=OFF, which I added. Also there is no darwin-work branch for nuopc_branch so it should be develop.

Hopefully this will get us further again.

mathomp4 · 2026-06-23T17:06:08Z

I just pushed a change to the add_alderaan branch on esmf-test-scripts. I think the important difference with previous builds was missing of ESMF_TRACE_LIB_BUILD=OFF, which I added. Also there is no darwin-work branch for nuopc_branch so it should be develop.

Hopefully this will get us further again.

Oh. Actually, I never pushed up my changes. I had develop for both and then did:

--branch feature/darwin-work

but I guess that messed up NUOPC. So I've cleaned things up, pushed, and am trying again

theurich · 2026-06-23T17:08:15Z

Ahh, that makes sense how that would have changed it for both. But now your latest changes are not what we want, we need this:

test:
  esmf_branch: [feature/darwin-work]
  nuopc_branch: [develop]

theurich · 2026-06-23T17:10:01Z

Nevermind, I see your latest push now... all good.

mathomp4 · 2026-06-23T18:18:50Z

@theurich It seems a lot happier now. Hopefully "soon" it'll push up test logs and all. I'm running with --throttle 1 since, well, this is my interactive machine and I didn't know if it would try and do both the O and g builds at the same time.

So it might be a bit as it needs to finish O and then do g. But positive signs so far!

mathomp4 · 2026-06-23T18:20:16Z

Also, in re that trace flag I see in my Baselibs CHANGELOG:

Set ESMF_TRACE_LIB_BUILD=OFF when building ESMF on Darwin

Previously was only on NAG, but testing shows even GCC builds
can fail with this flag on

so, yep. You probably taught me that long ago and I've forgotten. 😄

mathomp4 · 2026-06-23T19:19:51Z

@theurich New update. Things got farther, which is nice, but then things fell apart:

+ ssh gs6101-alderaan.ndc.nasa.gov /Users/mathomp4/ESMF_Testing/llvm_22.1_openmpi_O_feature_darwin-work/esmpy_install.bat
** WARNING: connection is not using a post-quantum key exchange algorithm.
** This session may be vulnerable to "store now, decrypt later" attacks.
** The server may need to be upgraded. See https://openssh.com/pq.html
By accessing and using this information system, you acknowledge and consent to the following: You are accessing a U.S. Government information system, which includes: (1) this computer; (2) this computer network; (3) all computers connected to this network including end user systems; ( 4) all devices and storage media attached to this network or to any computer on this network; and (5) cloud and remote information services. This information system is provided for U.S. Government-authorized use only. This system contains Controlled Unclassified Information (CUI). You have no reasonable expectation of privacy regarding any communication transmitted through or data stored on this information system. At any time, and for any lawful purpose, the U.S. Government may monitor, intercept, search, and seize any communication or data transiting, stored on, or traveling to or from this information system. You are NOT authorized to process classified information on this information system. Unauthorized or improper use of this system may result in suspension or loss of access privileges, disciplinary action, and civil and/or criminal penalties.
sign_and_send_pubkey: signing failed for RSA "Key For PIV Authentication (Matthew Thompson (affiliate))\001" from agent: agent refused operation
mathomp4@gs6101-alderaan.ndc.nasa.gov: Permission denied (publickey).
+ cd /Users/mathomp4/ESMF_Testing/llvm_22.1_openmpi_O_feature_darwin-work
+ conda activate /Users/mathomp4/ESMF_Testing/conda_environments/esmf-test-scripts-environment-GEOSpyD/26.3.2-0/3.14
/Users/mathomp4/ESMF_Testing/llvm_22.1_openmpi_O_feature_darwin-work/test.bat: line 28: conda: command not found
+ . esmpy_venv/bin/activate
/Users/mathomp4/ESMF_Testing/llvm_22.1_openmpi_O_feature_darwin-work/test.bat: line 29: esmpy_venv/bin/activate: No such file or directory
+ cd /Users/mathomp4/ESMF_Testing/llvm_22.1_openmpi_O_feature_darwin-work/esmf/src/addon/esmpy
+ make test
+ tee /Users/mathomp4/ESMF_Testing/llvm_22.1_openmpi_O_feature_darwin-work/esmpy-test.log
+ deactivate
/Users/mathomp4/ESMF_Testing/llvm_22.1_openmpi_O_feature_darwin-work/test.bat: line 32: deactivate: command not found
+ conda deactivate
/Users/mathomp4/ESMF_Testing/llvm_22.1_openmpi_O_feature_darwin-work/test.bat: line 33: conda: command not found
Traceback (most recent call last):
  File "/Users/mathomp4/ESMF_Testing/esmf-test-scripts/python_scripts/./test_esmf.py", line 369, in <module>
    go(args)
    ~~^^^^^^
  File "/Users/mathomp4/ESMF_Testing/esmf-test-scripts/python_scripts/./test_esmf.py", line 310, in go
    test.start()
    ~~~~~~~~~~^^
  File "/Users/mathomp4/ESMF_Testing/esmf-test-scripts/python_scripts/./test_esmf.py", line 279, in start
    _submit_case(_c)  # may bock
    ~~~~~~~~~~~~^^^^
  File "/Users/mathomp4/ESMF_Testing/esmf-test-scripts/python_scripts/./test_esmf.py", line 207, in _submit_case
    _case.submit(no_artifacts=self.no_artifacts)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mathomp4/ESMF_Testing/esmf-test-scripts/python_scripts/case.py", line 119, in submit
    self.test_job_num = self.machine.scheduler.submit_job(script_file=self.test_script, after=self.build_job_num)
                        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mathomp4/ESMF_Testing/esmf-test-scripts/python_scripts/noscheduler.py", line 18, in submit_job
    cmd.runcmd(f"{script_file}")
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/Users/mathomp4/ESMF_Testing/esmf-test-scripts/python_scripts/cmd.py", line 20, in runcmd
    raise cpe
  File "/Users/mathomp4/ESMF_Testing/esmf-test-scripts/python_scripts/cmd.py", line 15, in runcmd
    out = subprocess.check_output(cmd, shell=True, stderr=_stderr).strip().decode("utf-8")
          ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mathomp4/installed/Core/GEOSpyD/26.3.2-0/2026-05-06/envs/py3.14/lib/python3.14/subprocess.py", line 473, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
           ~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
               **kwargs).stdout
               ^^^^^^^^^
  File "/Users/mathomp4/installed/Core/GEOSpyD/26.3.2-0/2026-05-06/envs/py3.14/lib/python3.14/subprocess.py", line 578, in run
    raise CalledProcessError(retcode, process.args,
                             output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '/Users/mathomp4/ESMF_Testing/llvm_22.1_openmpi_O_feature_darwin-work/test.bat' returned non-zero exit status 127.

I think the issue is I don't have conda in my Python stack, only mamba. For now, I've commented out the esmpy: bit of alderaan.yaml because I think that's how you "don't do" esmpy testing.

I've resubmitted the test command.

theurich · 2026-06-23T19:23:39Z

Yes, I agree with not having the esmpy key in this configuration. I was wondering about it earlier for a millisecond when I saw it, but was immediately sidetracked with something else...

theurich · 2026-06-23T19:25:52Z

BTW, I already see the artifacts I wanted to look at. So from my side okay to not re-submit again at this time.

mathomp4 · 2026-06-23T19:28:21Z

BTW, I already see the artifacts I wanted to look at. So from my side okay to not re-submit again at this time.

Ah. I did a ctrl-c so it won't upload. Let me know when you'd like me to run again.

ETA: Also I guess if you want ESMF_BOPT=g output

theurich · 2026-06-23T19:33:42Z

Yep, ESMF_BOPT=g would be nice to have. I just pushed a change to config/alderaan.yaml on the branch so it will only run the debug mode. Please let that one run when it's convenient. Thanks

mathomp4 · 2026-06-24T12:19:01Z

Yep, ESMF_BOPT=g would be nice to have. I just pushed a change to config/alderaan.yaml on the branch so it will only run the debug mode. Please let that one run when it's convenient. Thanks

Running now! Hopefully updates pushed "soon" (so a couple hours it seems)

mathomp4 · 2026-06-24T15:30:31Z

I think the BOPT=g should be pushed up!

theurich · 2026-06-25T23:48:35Z

@mathomp4 haven't had much of a chance to look more deeply at the test artifacts yet. Just on the low hanging fruit, it looks like all of the ESMX tests under the NUOPC app protos are currently failing because of how FC is set in the environment:

CMake Error at /Users/mathomp4/.homebrew/brew/share/cmake/Modules/CMakeDetermineFortranCompiler.cmake:33 (message):
  Could not find the compiler specified in the environment variable FC:

  bin/flang-22.

Does FC get set by any of the modules that are loaded?

Currently Loaded Modules:
  1) GEOSpyD/26.3.2-0/3.14   2) appleclang-flang/22   3) openmpi/5.0.10

Or where might this be coming from? One way of handling it would be to set FC to a valid setting inside the config/alderaan.yaml, under extra_env_vars.

mathomp4 · 2026-06-26T11:23:37Z

@theurich I have a theory. My modulefile is:

https://github.com/mathomp4/m1modulefiles/blob/main/Core/appleclang-flang/22.lua

and on alderaan that is evaluated to:

/Users/mathomp4/.homebrew/brew/bin/flang-22

which, well, is where it is.

I use the same (essentially) modulefile on my home mac and:

❯ echo $FC
/opt/homebrew/bin/flang-22

But, I bet BREWPATH isn't set in the environment that is being run by the scripts. My dotfiles on all my macs are consistent and I also try to my my modulefiles be as portable as well (since brew might be installed in other places).

So I'm guessing I should add:

extra_env_vars:
  BREWPATH: /path/to/brew

?

I'll take a look when I can though, note, the next test might come from my other mac. Weirdly, I can't seem to ssh into alderaan from home. Something...firewall-y is happening? Not sure. But, well, an M1 vs an M2 isn't a big change.

mathomp4 · 2026-06-26T16:57:19Z

@theurich I'm trying a test now with the BREWPATH in extra_env_vars on my laptop. Let's see how this goes...

mathomp4 · 2026-06-27T11:45:34Z

@theurich I'm trying a test now with the BREWPATH in extra_env_vars on my laptop. Let's see how this goes...

Nope. Didn't work. Okay. Hard code FC!

theurich added 3 commits June 16, 2026 10:00

Detect Apple Clang and switch OpenMP and OpenACC.

e4fc878

Align handling of OpenMP under Apple Clang.

8959f4e

Deal with linking differences LLVM vs Apple Clang.

035582d

theurich self-assigned this Jun 16, 2026

theurich added the porting Porting ESMF to new platform and/or compiler label Jun 16, 2026

theurich mentioned this pull request Jun 16, 2026

Need for Darwin.llvm build_config #529

Open

theurich added the who: NASA Originates from NASA label Jun 16, 2026

theurich added 2 commits June 17, 2026 16:23

Robustify script to account for phantom LTO library paths that occur

0678b9a

under Darwin Homebrew Flang installations.

Disable verbose info about Apple vs LLVM Clang being found.

272b638

Uh oh!

Conversation

theurich commented Jun 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mathomp4 commented Jun 16, 2026

Uh oh!

mathomp4 commented Jun 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

theurich commented Jun 16, 2026

Uh oh!

mathomp4 commented Jun 17, 2026

Uh oh!

theurich commented Jun 18, 2026

Uh oh!

mathomp4 commented Jun 22, 2026

Uh oh!

mathomp4 commented Jun 22, 2026

Uh oh!

theurich commented Jun 22, 2026

Uh oh!

mathomp4 commented Jun 22, 2026

Uh oh!

theurich commented Jun 22, 2026

Uh oh!

mathomp4 commented Jun 22, 2026

Uh oh!

theurich commented Jun 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mathomp4 commented Jun 23, 2026

Uh oh!

theurich commented Jun 23, 2026

Uh oh!

mathomp4 commented Jun 23, 2026

Uh oh!

theurich commented Jun 23, 2026

Uh oh!

theurich commented Jun 23, 2026

Uh oh!

mathomp4 commented Jun 23, 2026

Uh oh!

mathomp4 commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mathomp4 commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

theurich commented Jun 23, 2026

Uh oh!

theurich commented Jun 23, 2026

Uh oh!

mathomp4 commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

theurich commented Jun 23, 2026

Uh oh!

mathomp4 commented Jun 24, 2026

Uh oh!

mathomp4 commented Jun 24, 2026

Uh oh!

theurich commented Jun 25, 2026

Uh oh!

mathomp4 commented Jun 26, 2026

Uh oh!

mathomp4 commented Jun 26, 2026

Uh oh!

mathomp4 commented Jun 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

theurich commented Jun 16, 2026 •

edited

Loading

mathomp4 commented Jun 16, 2026 •

edited

Loading

theurich commented Jun 22, 2026 •

edited

Loading

mathomp4 commented Jun 23, 2026 •

edited

Loading

mathomp4 commented Jun 23, 2026 •

edited

Loading

mathomp4 commented Jun 23, 2026 •

edited

Loading