Import error of adios4dolfinx on HPC via spack installation

installation
,
1

Hello all,

I try to install adios4dolfinx on HPC besides dolfinx inside a spack env

spack debug report
* **Spack:** 1.2.0.dev0 (https://github.com/spack/spack/commit/f38e413f807f0e7a0de47b7e7da1221d17e0225b)
* **Builtin repo:** https://github.com/spack/spack-packages/commit/e79ba9fc76dd02568da737230bd5f570822fcbb9
* **Python:** 3.6.8
* **Platform:** linux-rocky8-cascadelake

spack add py-pip
spack add py-fenics-dolfinx+petsc4py+slepc4py cflags="-O3" fflags="-O3"
spack add py-adios4dolfinx ^py-fenics-dolfinx+petsc4py+slepc4py

while the installation runs succesfull, i get an import error when i try to use adios4dolfinx inside the env

>>> import adios4dolfinx
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/zhome/academic/HLRS/isd/isdsg/spack/var/spack/environments/examples/.spack-env/view/lib/python3.12/site-packages/adios4dolfinx/__init__.py", line 11, in <module>
    from .checkpointing import (
  File "/zhome/academic/HLRS/isd/isdsg/spack/var/spack/environments/examples/.spack-env/view/lib/python3.12/site-packages/adios4dolfinx/checkpointing.py", line 22, in <module>
    from .adios2_helpers import (
  File "/zhome/academic/HLRS/isd/isdsg/spack/var/spack/environments/examples/.spack-env/view/lib/python3.12/site-packages/adios4dolfinx/adios2_helpers.py", line 25, in <module>
    adios2 = resolve_adios_scope(adios2)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/zhome/academic/HLRS/isd/isdsg/spack/var/spack/environments/examples/.spack-env/view/lib/python3.12/site-packages/adios4dolfinx/adios2_helpers.py", line 20, in resolve_adios_scope
    if not scope.is_built_with_mpi:
           ^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'adios2' has no attribute 'is_built_with_mpi'

I dont have import errors with other packages like dolfinx. Also setting the PATH as recommended in one github issue didnt solve the problem.

export PYTHONPATH=/spack/var/spack/environments/examples/.spack-env/._view/dxuyvxjoj7rsxx4hg5ohrnh25vrfpn6z/lib/python3.12/site-packages:$PYTHONPATH

I also tried to set the system openmpi explicitly but it didnt help

spack add py-adios4dolfinx ^py-fenics-dolfinx+petsc4py+slepc4py ^adios2%gcc@14.3.0 ^openmpi@5.0.8

Any idea?

Best regards

2

Could you check what version of adios2 that is installed?
The Python binding should have this attribute: Code search results · GitHub
For instance by calling python3 -c "import adios2; print(adios2.__version__)"

3

I am using this version

python3 -c "import adios2; print(adios2.__version__)"
2.8.3

which also aligns with the spack package

spack find adios2

-- linux-rocky8-cascadelake / %c,cxx,fortran=gcc@14.3.0 ---------
adios2@2.8.3
4

That is a quite old version of adios2. 2.8.3 is from 2022 (Release ADIOS v2.8.3 Production Patch Release · ornladios/ADIOS2 · GitHub)

Could you try to put a lower bound at 2.9? (Spack Packages)

5

Thanks for the help @dokken.

Yes, using a newer version of adios2 is the solution.

I am not sure why my spack behaved that way, I think it was because i had this adios2 version from an openfoam installation via spack.

What was not trivial, when i tried to use a newer adios2 version via this command, i ended up with an old version of hdf5, which was too old for the compiler:

spack add py-adios4dolfinx ^py-fenics-dolfinx+petsc4py+slepc4py ^adios2@2.10.2 ^openmpi@5.0.8

spack concretize -f 

==> Concretized 1 spec:
 -   6opy24l  py-adios4dolfinx@0.9.4 build_system=python_pip platform=linux os=rocky8 target=cascadelake
 -   pr4lmmx      ^adios2@2.9.2~aws+blosc2+bzip2~cuda~dataspaces+fortran+hdf5~ipo~kokkos+libcatalyst~libpressio+mgard+mpi~pic+png+python~rocm+sst+sz+xrootd+zfp build_system=cmake build_type=Release generator=make patches:=4247a85,48766ac,5b56f4b,d854008 platform=linux os=rocky8 target=cascadelake %c,cxx,fortran=gcc@14.3.0
[+]  7viti2e          ^bzip2@1.0.8~debug~pic+shared build_system=generic platform=linux os=rocky8 target=cascadelake %c=gcc@14.3.0
[+]  louy4ba              ^diffutils@3.12 build_system=autotools platform=linux os=rocky8 target=cascadelake %c=gcc@14.3.0
 -   yrzegrw          ^c-blosc2@2.15.1+avx2~ipo+lizard+lz4+snappy+zlib+zstd build_system=cmake build_type=Release generator=make platform=linux os=rocky8 target=cascadelake %c=gcc@14.3.0
 -   r7pud7d              ^lizard@2.1 build_system=makefile platform=linux os=rocky8 target=cascadelake %c=gcc@14.3.0
[+]  rssmyta              ^lz4@1.10.0+pic build_system=makefile libs:=shared,static platform=linux os=rocky8 target=cascadelake %c,cxx=gcc@14.3.0
[+]  s53zcru              ^snappy@1.2.1~ipo+pic+shared build_system=cmake build_type=Release generator=make platform=linux os=rocky8 target=cascadelake %c,cxx=gcc@14.3.0
[+]  aik5zb7              ^zstd@1.5.7+programs build_system=makefile compression:=none libs:=shared,static platform=linux os=rocky8 target=cascadelake %c,cxx=gcc@14.3.0
[+]  6zno4j4          ^cmake@3.31.9~doc+ncurses+ownlibs~qtgui build_system=generic build_type=Release platform=linux os=rocky8 target=cascadelake %c,cxx=gcc@14.3.0
[+]  fmrvsn2              ^curl@8.17.0~gssapi~ldap~libidn2~librtmp~libssh~libssh2+nghttp2 build_system=autotools libs:=shared,static tls:=openssl platform=linux os=rocky8 target=cascadelake %c,cxx=gcc@14.3.0
[+]  xgzxdk6                  ^nghttp2@1.67.1 build_system=autotools platform=linux os=rocky8 target=cascadelake %c,cxx=gcc@14.3.0
[+]  iohyzhq          ^compiler-wrapper@1.0 build_system=generic platform=linux os=rocky8 target=cascadelake
[e]  3gfxddl          ^gcc@14.3.0~binutils+bootstrap~graphite~mold~nvptx~piclibs~profiled~strip build_system=autotools build_type=RelWithDebInfo languages:='c,c++,fortran' platform=linux os=rocky8 target=x86_64
[+]  a6bqkaz          ^gcc-runtime@14.3.0 build_system=generic platform=linux os=rocky8 target=cascadelake
[e]  vuczjrb          ^glibc@2.28 build_system=autotools platform=linux os=rocky8 target=x86_64
[+]  lblybtp          ^gmake@4.4.1~guile build_system=generic platform=linux os=rocky8 target=cascadelake %c=gcc@14.3.0
 -   boakafo          ^hdf5@1.8.23~cxx~fortran~hl~ipo+mpi+shared~szip~threadsafe+tools api=default build_system=cmake build_type=Release generator=make patches:=f42732a platform=linux os=rocky8 target=cascadelake %c=gcc@14.3.0

Finally, after adding adios2 as a spec with the right hdf5 version, everything could be solved.

spack add adios2@2.10.2 ^hdf5@1.12.3
spack concretize -f
1 Like