CASTEP 23.1.1 and CASTEP 24.1

[heatherkellyucl]

The existing recipe covers CASTEP 19.1.1.rc2 and 21.11.

I have added the checksum for 23.1.1 in repos/dev/packages/castep/package.py

Need to check if there are any flags or variants needed for this version, if it compiles and if it works.

[heatherkellyucl]

I've added these variants:

    variant("grimmed3", default=False, description="Enable Grimme DFT+D library")

    variant("grimmed4", default=False, when="@23:", description="Enable Grimme D4 library")

    variant(
        "libxc", default=False, when="@23:",
        description="Enable libxc library of additional XC functionals"
    )

and some depends_on, around the existing mpi one:

    depends_on("perl", type=("build", "run"))
    depends_on("mpi", type=("build", "link", "run"), when="+mpi")
    depends_on("libxc", type=("build", "link", "run"), when="+libxc")

(not sure if libxc is needed at runtime)
and this is where it adds them to the build options:

        # "system" for existing libxc, "compile" to build own
        if "+libxc" in spec:
            targetlist.append("LIBXC=system")

        if "+grimmed3" in spec:
            targetlist.append("GRIMMED3=compile")

        if "+grimmed4" in spec:
            targetlist.append("GRIMMED4=compile")

Can't enable the xml variant right now because it requires FoX: A Fortran library for XML processing which isn't in Spack right now. The only fox is a C++ toolkit. (Quantum Espresso builds its own FoX). So it would need a different name if I added it.

CASTEP has a cmake build system too - probably that one should be added as well (https://spack.readthedocs.io/en/latest/packaging_guide.html#writing-a-package-recipe) but not right now.

I don't think the recipe builds castep.serial right now, which we want it to do. You'd normally do a serial build as well as the mpi build and install them both into the same place. To check.

[heatherkellyucl]

Altered the new variants above to add the when="@23:" since they were added in that version.

I was hoping if our source mirror was there then it would look in it for the tar file automatically - that doesn't appear to be the case so another bit of config needs adjusting. For right now, the package recipe tells it to look in the current directory for the source, so copying in there works.

However, part of the filter is wrong for this version:

==> Installing castep-23.1.1-2h2cflxs6gt2prrzr7kzuod7tktms4d5 [96/100]
==> No binary for castep-23.1.1-2h2cflxs6gt2prrzr7kzuod7tktms4d5 found: installing from source
==> Fetching file:///lustre/scratch/scratch/ccspapp/spack/0.22/hpc-spack/castep-23.1.1.tar.bz2
==> No patches needed for castep
==> castep: Executing phase: 'edit'
==> Error: FileNotFoundError: [Errno 2] No such file or directory: 'LibSource/dl_mg-2.0.3/platforms/castep.inc
'

/lustre/scratch/scratch/ccspapp/spack/0.22/hpc-spack/repos/dev/packages/castep/package.py:67, in edit:
         64    def edit(self, spec, prefix):
         65        if spec.satisfies("%gcc"):
         66            dlmakefile = FileFilter("LibSource/dl_mg-2.0.3/platforms/castep.inc")
  >>     67            dlmakefile.filter(r"MPIFLAGS = -DMPI", "MPIFLAGS = -fallow-argument-mismatch -DMPI")
         68            if self.spec.satisfies("@20:"):
         69                platfile = FileFilter("obj/platforms/linux_x86_64_gfortran.mk")
         70            else:

See build log for details:
  /lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/build_stage/ccspapp/spack-stage-castep-23.1.1-2h2cflxs6gt2prrzr7kzuod7tktms4d5/spack-build-out.txt

It is because it is LibSource/dl_mg-3.0.0 in this version, need to add logic for that.

[heatherkellyucl]

I confirmed that dl_mg-3.0.0 does also need -fallow-argument-mismatch adding to the MPIFLAGS.

Changed to:

            if self.spec.satisfies("@19:21"):
                dlmakefile = FileFilter("LibSource/dl_mg-2.0.3/platforms/castep.inc")
            if self.spec.satisfies("@23:"):
                dlmakefile = FileFilter("LibSource/dl_mg-3.0.0/platforms/castep.inc")
            dlmakefile.filter(r"MPIFLAGS = -DMPI", "MPIFLAGS = -fallow-argument-mismatch -DMPI")

[heatherkellyucl]

Still got type mismatches.

==> Error: ProcessError: Command exited with status 2:
    'make' '-j6' 'PWD=/lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/build_stage/ccspapp/spack-stage-cast
ep-23.1.1-2h2cflxs6gt2prrzr7kzuod7tktms4d5/spack-src' 'COMMS_ARCH=mpi' 'FFTLIBDIR=/lustre/shared/ucl/apps/spac
k/0.22/hk-initial-stack/spack/opt/spack/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_pla
ceholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_place
holder__/linux-rhel7-cascadelake/gcc-12.3.0/fftw-3.3.10-3hrfzp6xsbvyfdebwta53swqul7tt3x4/lib' 'MATHLIBDIR=/lus
tre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/opt/spack/__spack_path_placeholder__/__spack_path_placeh
older__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placehol
der__/__spack_path_placeholder__/linux-rhel7-cascadelake/gcc-12.3.0/openblas-0.3.26-mtr3n5zbgfqoca4pjp5xx5gotc5dqvhy/lib' 'FFT=fftw3' 'MATHLIBS=openblas' 'ARCH=linux_x86_64_gfortran'

242 errors found in build log:
     262
     263     22418 |        call MPI_allreduce(local_array,z_array(bufpos),min(buflen,length-bufpos+1),&
     264           |                          1
     265     ......
     266     22517 |        call MPI_allreduce(local_array,d_array(bufpos),min(buflen,length-bufpos+1),&
     267           |                          2
  >> 268     Error: Type mismatch between actual argument at (1) and actual argument at (2) (COMPLEX(8)/REAL(
             8)).
     269     ../../../Source/Utility/comms.mpi.F90:22418:38:
...

  >> 2180    Error: Type mismatch between actual argument at (1) and actual argument at (2) (CHARACTER(12)/IN
             TEGER(4)).
  >> 2181    make[1]: *** [../../include_rules.mk:155: comms.mpi.anc] Error 1
...

See build log for details:
  /lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/build_stage/ccspapp/spack-stage-castep-23.1.1-2h2cflxs6gt2prrzr7kzuod7tktms4d5/spack-build-out.txt

[giordano]

You can try adding fflags=-fallow-argument-mismatch to the spec as an alternative, which forces the flag in the wrapper of the fortran compiler used by spack.

[heatherkellyucl]

I'll have a go like that temporarily.

It looks like it needs adding in the package.py to FFLAGS_FOR_LIBRARIES in obj/platforms/linux_x86_64_gfortran.mk since it already gets added to FFLAGS_E which are the general flags.

# Flags for building third-party libraries (DL_MG, D3, D4). Assign with "=" to include $(OPT) defined later
#
FFLAGS_FOR_LIBRARIES = -fconvert=big-endian -fopenmp -fPIC -ffpe-summary=none $(OPT)

[heatherkellyucl]

Though there is a linux_x86_64_gfortran10.mk so maybe we should be using that now.

[heatherkellyucl]

Including in spec worked, as castep@23.1.1 fflags=-fallow-argument-mismatch:

==> Installing castep-23.1.1-odprczkj44tekqlbv4rwbzpdfecfsvel [96/101]
==> No binary for castep-23.1.1-odprczkj44tekqlbv4rwbzpdfecfsvel found: installing from source
==> Using cached archive: /lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/var/spack/cache/_source-ca
che/archive/8d/8d922c641c99fc6e4f5b4f7f2478abab897065850e454fb4968154ddbb566388.tar.bz2
==> No patches needed for castep
==> castep: Executing phase: 'edit'
==> castep: Executing phase: 'build'
==> castep: Executing phase: 'install'
==> castep: Successfully installed castep-23.1.1-odprczkj44tekqlbv4rwbzpdfecfsvel
  Stage: 26.42s.  Edit: 0.20s.  Build: 4m 20.51s.  Install: 1m 2.18s.  Post-install: 3.08s.  Total: 6m 3.66s

[heatherkellyucl]

There's a linux_x86_64_gfortran10.mk in our 20.1 source as well, so it looks like it has existed since then and should probably be the one used in all versions after 19.x. Assuming that works.

[heatherkellyucl]

Working out how to include this lot of recommended and incompatible compilers is going to be interesting...

 USE: The following compilers have been tested and confirmed to
 correctly compile CASTEP 23.1.1

 *  Intel Fortran 18.0.2+, 19.0.0, 19.0.2
 *  Intel oneAPI Fortran (classic) 2021.2.0-2021.3.0, 2021.5.0-2021.6.0
    (packaged as oneAPI version 2021.2,2021.3,2022.0.1,2022.0.2,2022.1.0)
 *  GNU Fortran   7.2-7.5, 8.*, 9.2, 9.3.0, 10.1.0, 11.x, 12.x
 *  NVIDIA/PGI    nvhpc 20.7-21.X, 22.5, 22.11 (+)
 *  NAG Fortran   7.1
 *  Cray Fortran  11.0.4, 12.0.3, 13.0.2 (++)

 AVOID: The following compilers either have bugs which result in at least one
 failure, either during compilation or a crash or incorrect
 results at runtime, or do not support the language standards required.
 There may be more details of the seriousness of
 the failures in the platform-specific sections below.

 *  Intel Fortran < 18.0, 18.0.0, 18.0.1 (*), 19.1.0(**)
 *  Intel oneAPI - ifx (any version) The new flang-based compiler (ifx) 
    suffers an ICE on attempted compile.
 *  Intel oneAPI classic (ifort) 2021.4.0, 2021.7.0,2021.7.1
    packaged as oneAPI 2021.4, 2022.2.0, 2022.2.1
    Various run-time failures on the test suite.
 *  NVidia nvfortran 22.7-22.10
 *  AMD AOCC flang < 4.0 (***)
 *  PGI Fortran < 19.x
 *  Oracle/Sun fortran (all versions) (****)
 *  Cray Fortran  11.0.x
 *  AMD AOCC flang (++)

For footnotes see README.INSTALL.

[heatherkellyucl]

Looking at https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/quantum-espresso/package.py for examples because it has lots of different kinds of conflicts.

    # Internal compiler error gcc8 and a64fx, I check only 6.5 and 6.6
    conflicts(
        "@5.3:", when="%gcc@8 target=a64fx", msg="Internal compiler error with gcc8 and a64fx"
    )

    with when("%nvhpc"):
        variant("cuda", default=False, description="Build with CUDA Fortran")
        with when("+cuda"):
            conflicts(
                "%nvhpc@21.11:22.3",
                msg="bugs with NVHPCSDK from v21.11 to v22.3, OpenMP and GPU",
            )

So like this? (https://spack.readthedocs.io/en/latest/basic_usage.html#version-specifier)

with when ("%intel-oneapi-compilers-classic"):
    conflicts(
        "%intel-oneapi-compilers-classic@2021.4.0,2021.7",
        msg="Various run-time failures on the test suite with ifort 2021.4.0, 2021.7.0, 2021.7.1",
    )

[heatherkellyucl]

The build stage with the source and tests are in /shared/ucl/apps/spack/0.22/hk-initial-stack/build_stage/ccspapp/spack-stage-castep-23.1.1-2h2cflxs6gt2prrzr7kzuod7tktms4d5/spack-src/Test/ still at the moment.

To run test suite at its simplest, just run

make

in this directory or

make check

in the top-level directory. Tests will be run using the CASTEP executable
"obj/$ARCH/castep", ie the default compiled executable. "make check ARCH=xxx"
works as expected, running the alternative executable.

The module for this CASTEP install is at /shared/ucl/apps/spack/0.22/hk-initial-stack/modules/lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/var/spack/environments/myriad/linux-rhel7-cascadelake/castep/23.1.1-openmpi/gcc-12.3.0

(Need to fix that somewhere).

[heatherkellyucl]

module use /shared/ucl/apps/spack/0.22/hk-initial-stack/modules/lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/var/spack/environments/myriad/linux-rhel7-cascadelake

module avail castep
- /shared/ucl/apps/spack/0.22/hk-initial-stack/modules/lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/var/spack/environments/myriad/linux-rhel7-cascadelake -
castep/23.1.1-openmpi/gcc-12.3.0  

--------------------------------- /shared/ucl/apps/modulefiles/applications ----------------------------------
castep/17.2/intel-2017  castep/17.21/intel-2017  castep/19.1.1/intel-2019 

[heatherkellyucl]

Needed to make sure gcc@12.3.0 is explicitly added to the environment too, since only the gcc-runtime was there as a module.

Then regenerated modules.

spack -e myriad add gcc@12.3.0
spack -e myriad concretize
spack -e myriad install
spack -e myriad module tcl refresh

module avail gcc
- /shared/ucl/apps/spack/0.22/hk-initial-stack/modules/lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/var/spack/environments/myriad/linux-rhel7-cascadelake -
gcc-runtime/11.2.1/gcc-11.2.1  gcc-runtime/12.3.0/gcc-12.3.0  gcc/12.3.0/gcc-11.2.1  

----------------------------------- /shared/ucl/apps/modulefiles/libraries -----------------------------------
gcc-libs/4.9.2  gcc-libs/7.3.0  gcc-libs/8.3.0  gcc-libs/9.2.0  gcc-libs/10.2.0  

The gcc-runtimes are like our gcc-libs, and the gcc loads some other modules and sets up CC, CXX, FC, F77 like our compilers/gnu modules.