MUMPS solver does not work in complex mode

Hi,

I am using complex dolfinx 0.7.3 installed through coda in wsl/ubuntu. I have noticed that MUMPS solver is never able to solve a problem. Using other direct solvers such as the default LU or UMFPACK is able to solve the problem without any issues.

I am unsure if it is a problem with petsc, the conda package or the solver needs special configuration to work which I don’t know? I will appreciate if more experienced users could share their thoughts.

Below is an MWE that solves scalar Helmholtz equation in a box. The expected solution is a propagating wave through the geometry. At least on my machine switching into MUMPS never leads to a result.

import dolfinx, petsc4py, mpi4py, gmsh, ufl
from dolfinx.io import gmshio
import numpy as np

def create_geom(fov_x=0.2, fov_y=0.2, fov_z=1):
    gmsh.initialize()
    gmsh.option.setNumber("General.Terminal", 0) # disable output messages
    gmsh.clear()
    gdim = 3
    model_rank = 0

    fov = gmsh.model.occ.add_box(-fov_x/2, -fov_y/2, -fov_z/2, fov_x, fov_y, fov_z)
    gmsh.model.occ.synchronize()

    all_doms = gmsh.model.getEntities(gdim)
    for j, dom in enumerate(all_doms):
        gmsh.model.addPhysicalGroup(dom[0], [dom[1]], j + 1)  # create the main group/node

    # number all boundaries
    all_edges = gmsh.model.getEntities(gdim - 1)
    for j, edge in enumerate(all_edges):
        gmsh.model.addPhysicalGroup(edge[0], [edge[1]], j + 1)  # create the main group/node

    gmsh.model.mesh.generate(gdim)
    gmsh.model.mesh.refine()
    
    model_rank = 0
    mesh, ct, ft = gmshio.model_to_mesh(gmsh.model, mpi4py.MPI.COMM_WORLD, model_rank, gdim)
    # gmsh.fltk.run()
    gmsh.finalize()
    return mesh, ct, ft

#%% input paramters
port_bnd_idx = (6, 5) # incident, transmitted
k0 = 2*np.pi/0.5
fe_degree = 3

mesh, ct, ft = create_geom()

Omega = dolfinx.fem.FunctionSpace(mesh, ("CG", fe_degree))
u = ufl.TrialFunction(Omega)
ut = ufl.TestFunction(Omega)

# bilinear form terms
F = (-ufl.inner(ufl.grad(u), ufl.grad(ut)) + k0**2*ufl.inner(u, ut))*ufl.dx
F += -ufl.inner(dolfinx.fem.Constant(mesh, petsc4py.PETSc.ScalarType(0.0)), ut)*ufl.dx # source term

#%% Port boundary at the top (excitation) and bottom surfaces 
ds = ufl.Measure("ds", domain=mesh, subdomain_data=ft) # boundary differential element

n_3d = ufl.FacetNormal(mesh)

ki_3d = ufl.as_vector((0, 0, -k0)) # incident wave
kr_3d = ufl.as_vector((0, 0, k0)) # reflected wave, if any
kt_3d = ufl.as_vector((0, 0, -k0)) # transmitted wave

# incident port
bc_inc = ufl.inner(1j*kr_3d[2]*u, ut)*ds(port_bnd_idx[0])
# transmission port
bc_tr = ufl.inner(ufl.dot(n_3d, 1j*kt_3d*u), ut)*ds(port_bnd_idx[1])
F+= bc_inc + bc_tr
# excitation at the incident port
u_inc = 1
bc_exc = ufl.inner(ufl.dot(n_3d, 1j*(ki_3d-kr_3d)*u_inc), ut)*ds(port_bnd_idx[0])
F += bc_exc

a = ufl.lhs(F)
L = ufl.rhs(F)

rtol = 1e-3
petsc_options = [
    {"ksp_type": "preonly", "pc_type": "lu", "monitor_convergence": True},
    {"ksp_type": "preonly", "pc_type": "lu", "pc_factor_mat_solver_type":"umfpack", "monitor_convergence": True},
    {"ksp_type": "preonly", "pc_type": "lu", "pc_factor_mat_solver_type":"mumps", 
    #  "mat_mumps_icntl_2": "1",
    #  "mat_mumps_icntl_4": "4",
    #  "mat_mumps_icntl_22":"1", # out of core factorization
    #  "mat_mumps_icntl_24":"1", # detect null pivot rows
    #  "mat_mumps_icntl_28":1, # detect null pivot rows
        # 'mat_mumps_cntl_3':0.01, 
        # "mat_mumps_icntl_1": 1, 
        # "mat_mumps_icntl_10":15,
        # 'mat_mumps_icntl_24':1
    },
]

solver_idx = 0
from dolfinx.fem.petsc import LinearProblem
Lin_system = LinearProblem(a, L, bcs = [], petsc_options=petsc_options[solver_idx])

solver = Lin_system.solver
import time
t1 = time.perf_counter()
u_sol = Lin_system.solve()
t2 = time.perf_counter()
print("Solved in {}s".format(t2-t1))
it = solver.getIterationNumber()
print("Constrained solver iterations {0:d}".format(it))
solver.view()

with dolfinx.io.VTXWriter(mpi4py.MPI.COMM_WORLD, "scalar_wave.bp", [u_sol], engine="BP4") as vtx:
    vtx.write(0.0)

Hi.

Could you elaborate a bit on the error/problem you are observing?. I obtained the following by running your code using MUMPS in dolfinx v0.7.3 docker image

Solved in 0.22613129199999094s
Constrained solver iterations 1
KSP Object: (dolfinx_solve_281473539498352) 1 MPI process
  type: preonly
  maximum iterations=10000, initial guess is zero
  tolerances:  relative=1e-05, absolute=1e-50, divergence=10000.
  left preconditioning
  using NONE norm type for convergence test
PC Object: (dolfinx_solve_281473539498352) 1 MPI process
  type: lu
    out-of-place factorization
    tolerance for zero pivot 2.22045e-14
    matrix ordering: external
    factor fill ratio given 0., needed 0.
      Factored matrix follows:
        Mat Object: (dolfinx_solve_281473539498352) 1 MPI process
          type: mumps
          rows=11491, cols=11491
          package used to perform factorization: mumps
          total: nonzeros=3232473, allocated nonzeros=3232473
            MUMPS run parameters:
              Use -dolfinx_solve_281473539498352ksp_view ::ascii_info_detail to display information for all processes
              RINFOG(1) (global estimated flops for the elimination after analysis): 6.32398e+08
              RINFOG(2) (global estimated flops for the assembly after factorization): 4.45209e+06
              RINFOG(3) (global estimated flops for the elimination after factorization): 6.32398e+08
              (RINFOG(12) RINFOG(13))*2^INFOG(34) (determinant): (0.,0.)*(2^0)
              INFOG(3) (estimated real workspace for factors on all processors after analysis): 3232473
              INFOG(4) (estimated integer workspace for factors on all processors after analysis): 107766
              INFOG(5) (estimated maximum front size in the complete tree): 447
              INFOG(6) (number of nodes in the complete tree): 454
              INFOG(7) (ordering option effectively used after analysis): 5
              INFOG(8) (structural symmetry in percent of the permuted matrix after analysis): 100
              INFOG(9) (total real/complex workspace to store the matrix factors after factorization): 3232473
              INFOG(10) (total integer space store the matrix factors after factorization): 107766
              INFOG(11) (order of largest frontal matrix after factorization): 447
              INFOG(12) (number of off-diagonal pivots): 0
              INFOG(13) (number of delayed pivots after factorization): 0
              INFOG(14) (number of memory compress after factorization): 0
              INFOG(15) (number of steps of iterative refinement after solution): 0
              INFOG(16) (estimated size (in MB) of all MUMPS internal data for factorization after analysis: value on the most memory consuming processor): 76
              INFOG(17) (estimated size of all MUMPS internal data for factorization after analysis: sum over all processors): 76
              INFOG(18) (size of all MUMPS internal data allocated during factorization: value on the most memory consuming processor): 76
              INFOG(19) (size of all MUMPS internal data allocated during factorization: sum over all processors): 76
              INFOG(20) (estimated number of entries in the factors): 3232473
              INFOG(21) (size in MB of memory effectively used during factorization - value on the most memory consuming processor): 65
              INFOG(22) (size in MB of memory effectively used during factorization - sum over all processors): 65
              INFOG(23) (after analysis: value of ICNTL(6) effectively used): 0
              INFOG(24) (after analysis: value of ICNTL(12) effectively used): 1
              INFOG(25) (after factorization: number of pivots modified by static pivoting): 0
              INFOG(28) (after factorization: number of null pivots encountered): 0
              INFOG(29) (after factorization: effective number of entries in the factors (sum over all processors)): 3232473
              INFOG(30, 31) (after solution: size in Mbytes of memory used during solution phase): 66, 66
              INFOG(32) (after analysis: type of analysis done): 1
              INFOG(33) (value used for ICNTL(8)): 7
              INFOG(34) (exponent of the determinant if determinant is requested): 0
              INFOG(35) (after factorization: number of entries taking into account BLR factor compression - sum over all processors): 3232473
              INFOG(36) (after analysis: estimated size of all MUMPS internal data for running BLR in-core - value on the most memory consuming processor): 0
              INFOG(37) (after analysis: estimated size of all MUMPS internal data for running BLR in-core - sum over all processors): 0
              INFOG(38) (after analysis: estimated size of all MUMPS internal data for running BLR out-of-core - value on the most memory consuming processor): 0
              INFOG(39) (after analysis: estimated size of all MUMPS internal data for running BLR out-of-core - sum over all processors): 0
  linear system matrix = precond matrix:
  Mat Object: (dolfinx_solve_281473539498352) 1 MPI process
    type: seqaij
    rows=11491, cols=11491
    total: nonzeros=494299, allocated nonzeros=494299
    total number of mallocs used during MatSetValues calls=0
      using I-node routines: found 8261 nodes, limit used is 5
WARNING! There are options you set that were not used!
WARNING! could be spelling mistake, etc!
There is one unused database option. It is:
Option left: name:-dolfinx_solve_281473539498352monitor_convergence value: true source: code

1 Like

Thanks for your feedback! In my case, the solver keeps running, never able to terminate.

Just checked on dolfinx/stable and it indeed works fine. I also redid the conda installation but it did not help.

To developers, could you please look into this problem?

I can reproduce the problem with conda using:

mamba install -c conda-forge  fenics-dolfinx petsc=*=complex* mpich gmsh python-gmsh

Conda env export yields:

conda env export 
name: test_mumps
channels:
  - conda-forge
dependencies:
  - _libgcc_mutex=0.1=conda_forge
  - _openmp_mutex=4.5=2_gnu
  - aiohttp=3.9.3=py312h98912ed_0
  - aiosignal=1.3.1=pyhd8ed1ab_0
  - alsa-lib=1.2.11=hd590300_1
  - aom=3.8.1=h59595ed_0
  - attr=2.5.1=h166bdaf_1
  - attrs=23.2.0=pyh71513ae_0
  - binutils_impl_linux-64=2.40=hf600244_0
  - binutils_linux-64=2.40=hdade7a5_2
  - blosc=1.21.5=h0f2a231_0
  - bzip2=1.0.8=hd590300_5
  - c-ares=1.27.0=hd590300_0
  - c-blosc2=2.13.2=hb4ffafa_0
  - ca-certificates=2024.2.2=hbcca054_0
  - cairo=1.18.0=h3faef2a_0
  - cffi=1.16.0=py312hf06ca03_0
  - dav1d=1.2.1=hd590300_0
  - dbus=1.13.6=h5008d03_3
  - double-conversion=3.3.0=h59595ed_0
  - eigen=3.4.0=h00ab1b0_0
  - expat=2.6.1=h59595ed_0
  - fenics-basix=0.7.0=py312h5b9907d_1
  - fenics-basix-pybind11-abi=0.4.12=h5b9907d_1
  - fenics-dolfinx=0.7.3=py312h9d5b5d7_1
  - fenics-ffcx=0.7.0=pyh4af843d_0
  - fenics-libbasix=0.7.0=hfdc072b_1
  - fenics-libdolfinx=0.7.3=he8d8b1c_1
  - fenics-ufcx=0.7.0=h4af843d_0
  - fenics-ufl=2023.2.0=pyhd8ed1ab_0
  - ffmpeg=6.1.1=gpl_h38e077a_106
  - fftw=3.3.10=mpi_mpich_h5537406_8
  - fltk=1.3.9=hea138e6_0
  - font-ttf-dejavu-sans-mono=2.37=hab24e00_0
  - font-ttf-inconsolata=3.000=h77eed37_0
  - font-ttf-source-code-pro=2.038=h77eed37_0
  - font-ttf-ubuntu=0.83=h77eed37_1
  - fontconfig=2.14.2=h14ed4e7_0
  - fonts-conda-ecosystem=1=0
  - fonts-conda-forge=1=0
  - freeimage=3.18.0=h4b96d29_20
  - freetype=2.12.1=h267a509_2
  - fribidi=1.0.10=h36c2ea0_0
  - frozenlist=1.4.1=py312h98912ed_0
  - gcc_impl_linux-64=12.3.0=he2b93b0_5
  - gcc_linux-64=12.3.0=h6477408_2
  - gettext=0.21.1=h27087fc_0
  - gl2ps=1.4.2=h0708190_0
  - glew=2.1.0=h9c3ff4c_2
  - glib=2.78.4=hf2295e7_4
  - glib-tools=2.78.4=hde27a5a_4
  - gmp=6.3.0=h59595ed_1
  - gmsh=4.12.2=h6b98cf8_0
  - gnutls=3.7.9=hb077bed_0
  - graphite2=1.3.13=h58526e2_1001
  - gst-plugins-base=1.22.9=h8e1006c_0
  - gstreamer=1.22.9=h98fc4e7_0
  - gxx_impl_linux-64=12.3.0=he2b93b0_5
  - gxx_linux-64=12.3.0=h4a1b8e8_2
  - harfbuzz=8.3.0=h3d44ed6_0
  - hdf4=4.2.15=h2a13503_7
  - hdf5=1.14.3=mpi_mpich_ha2c2bf8_0
  - hypre=2.28.0=mpi_mpich_h716cb5e_0
  - icu=73.2=h59595ed_0
  - idna=3.6=pyhd8ed1ab_0
  - imath=3.1.11=hfc55251_0
  - jsoncpp=1.9.5=h4bd325d_1
  - jxrlib=1.1=hd590300_3
  - kahip=3.16=he427f57_0
  - kahip-python=3.16=py312h79e6366_0
  - kernel-headers_linux-64=2.6.32=he073ed8_17
  - keyutils=1.6.1=h166bdaf_0
  - krb5=1.21.2=h659d440_0
  - lame=3.100=h166bdaf_1003
  - lcms2=2.16=hb7c19ff_0
  - ld_impl_linux-64=2.40=h41732ed_0
  - lerc=4.0.0=h27087fc_0
  - libabseil=20240116.1=cxx17_h59595ed_2
  - libadios2=2.9.2=mpi_mpich_h69321e2_1
  - libaec=1.1.2=h59595ed_1
  - libass=0.17.1=h8fe9dca_1
  - libblas=3.9.0=21_linux64_openblas
  - libboost=1.82.0=h6fcfa73_6
  - libboost-devel=1.82.0=h00ab1b0_6
  - libboost-headers=1.82.0=ha770c72_6
  - libcap=2.69=h0f662aa_0
  - libcblas=3.9.0=21_linux64_openblas
  - libclang=15.0.7=default_hb11cfb5_4
  - libclang13=15.0.7=default_ha2b6cf4_4
  - libcups=2.3.3=h4637d8d_4
  - libcurl=8.5.0=hca28451_0
  - libdeflate=1.19=hd590300_0
  - libdrm=2.4.120=hd590300_0
  - libedit=3.1.20191231=he28a2e2_2
  - libev=4.33=hd590300_2
  - libevent=2.1.12=hf998b51_1
  - libexpat=2.6.1=h59595ed_0
  - libffi=3.4.2=h7f98852_5
  - libflac=1.4.3=h59595ed_0
  - libgcc-devel_linux-64=12.3.0=h8bca6fd_105
  - libgcc-ng=13.2.0=h807b86a_5
  - libgcrypt=1.10.3=hd590300_0
  - libgfortran-ng=13.2.0=h69a702a_5
  - libgfortran5=13.2.0=ha4646dd_5
  - libglib=2.78.4=hf2295e7_4
  - libglu=9.0.0=hac7e632_1003
  - libgomp=13.2.0=h807b86a_5
  - libgpg-error=1.48=h71f35ed_0
  - libhwloc=2.9.3=default_h554bfaf_1009
  - libiconv=1.17=hd590300_2
  - libidn2=2.3.7=hd590300_0
  - libjpeg-turbo=3.0.0=hd590300_1
  - liblapack=3.9.0=21_linux64_openblas
  - libllvm15=15.0.7=hb3ce162_4
  - libnetcdf=4.9.2=nompi_h9612171_113
  - libnghttp2=1.58.0=h47da74e_1
  - libnsl=2.0.1=hd590300_0
  - libogg=1.3.4=h7f98852_1
  - libopenblas=0.3.26=pthreads_h413a1c8_0
  - libopenvino=2024.0.0=h2e90f83_1
  - libopenvino-auto-batch-plugin=2024.0.0=hd5fc58b_1
  - libopenvino-auto-plugin=2024.0.0=hd5fc58b_1
  - libopenvino-hetero-plugin=2024.0.0=h3ecfda7_1
  - libopenvino-intel-cpu-plugin=2024.0.0=h2e90f83_1
  - libopenvino-intel-gpu-plugin=2024.0.0=h2e90f83_1
  - libopenvino-ir-frontend=2024.0.0=h3ecfda7_1
  - libopenvino-onnx-frontend=2024.0.0=h757c851_1
  - libopenvino-paddle-frontend=2024.0.0=h757c851_1
  - libopenvino-pytorch-frontend=2024.0.0=h59595ed_1
  - libopenvino-tensorflow-frontend=2024.0.0=hca94c1a_1
  - libopenvino-tensorflow-lite-frontend=2024.0.0=h59595ed_1
  - libopus=1.3.1=h7f98852_1
  - libpciaccess=0.18=hd590300_0
  - libpng=1.6.43=h2797004_0
  - libpq=16.2=h33b98f1_0
  - libprotobuf=4.25.3=h08a7969_0
  - libptscotch=7.0.4=h2376d02_1
  - libraw=0.21.1=h2a13503_2
  - libsanitizer=12.3.0=h0f45ef3_5
  - libscotch=7.0.4=h91e35bf_1
  - libsndfile=1.2.2=hc60ed4a_1
  - libsodium=1.0.18=h36c2ea0_1
  - libsqlite=3.45.1=h2797004_0
  - libssh2=1.11.0=h0841786_0
  - libstdcxx-devel_linux-64=12.3.0=h8bca6fd_105
  - libstdcxx-ng=13.2.0=h7e041cc_5
  - libsystemd0=255=h3516f8a_1
  - libtasn1=4.19.0=h166bdaf_0
  - libtheora=1.1.1=h7f98852_1005
  - libtiff=4.6.0=ha9c0a0a_2
  - libunistring=0.9.10=h7f98852_0
  - libuuid=2.38.1=h0b41bf4_0
  - libva=2.21.0=hd590300_0
  - libvorbis=1.3.7=h9c3ff4c_0
  - libvpx=1.14.0=h59595ed_0
  - libwebp-base=1.3.2=hd590300_0
  - libxcb=1.15=h0b41bf4_0
  - libxcrypt=4.4.36=hd590300_1
  - libxkbcommon=1.6.0=hd429924_1
  - libxml2=2.12.5=h232c23b_0
  - libzip=1.10.1=h2629f0a_3
  - libzlib=1.2.13=hd590300_5
  - loguru=0.7.2=py312h7900ff3_1
  - lz4-c=1.9.4=hcb278e6_0
  - metis=5.1.1=h59595ed_2
  - mpfr=4.2.1=h9458935_0
  - mpg123=1.32.4=h59595ed_0
  - mpi=1.0=mpich
  - mpi4py=3.1.5=py312h5256a87_1
  - mpich=4.2.0=h846660c_100
  - multidict=6.0.5=py312h98912ed_0
  - mumps-include=5.6.2=ha770c72_2
  - mumps-mpi=5.6.2=h5b51c9e_2
  - mysql-common=8.0.33=hf1915f5_6
  - mysql-libs=8.0.33=hca2cd23_6
  - ncurses=6.4=h59595ed_2
  - nettle=3.9.1=h7ab15ed_0
  - nlohmann_json=3.11.2=h27087fc_0
  - nspr=4.35=h27087fc_0
  - nss=3.98=h1d7d5a4_0
  - numpy=1.26.4=py312heda63a1_0
  - occt=7.7.2=all_h4c9f3c6_201
  - ocl-icd=2.3.2=hd590300_0
  - openexr=3.2.2=haf962dd_1
  - openh264=2.4.1=h59595ed_0
  - openjpeg=2.5.2=h488ebb8_0
  - openssl=3.2.1=hd590300_0
  - p11-kit=0.24.1=hc5aa10d_0
  - parmetis=4.0.3=h2a9763c_1005
  - pcre2=10.43=hcad00b1_0
  - petsc=3.20.5=complex_h3cbb423_0
  - petsc4py=3.20.5=complex_hf96ce65_0
  - pip=24.0=pyhd8ed1ab_0
  - pixman=0.43.2=h59595ed_0
  - pkg-config=0.29.2=h36c2ea0_1008
  - proj=9.3.1=h1d62c97_0
  - pthread-stubs=0.4=h36c2ea0_1001
  - ptscotch=7.0.4=h23d43cc_1
  - pugixml=1.14=h59595ed_0
  - pulseaudio-client=16.1=hb77b528_5
  - pybind11-abi=4=hd8ed1ab_3
  - pycparser=2.21=pyhd8ed1ab_0
  - python=3.12.2=hab00c5b_0_cpython
  - python-gmsh=4.12.2=h57928b3_0
  - python_abi=3.12=4_cp312
  - qt-main=5.15.8=h5810be5_19
  - rapidjson=1.1.0=he1b5a44_1002
  - readline=8.2=h8228510_1
  - scalapack=2.2.0=hd931219_1
  - scotch=7.0.4=h23d43cc_1
  - setuptools=69.1.1=pyhd8ed1ab_0
  - slepc=3.20.1=complex_he7d881e_2
  - slepc4py=3.20.1=complex_h62475ad_0
  - snappy=1.1.10=h9fff704_0
  - sqlite=3.45.1=h2c6b66d_0
  - suitesparse=5.10.1=h3ec001c_2
  - superlu=5.2.2=h00795ac_0
  - superlu_dist=8.2.1=h5902e8c_0
  - svt-av1=1.8.0=h59595ed_0
  - sysroot_linux-64=2.12=he073ed8_17
  - tbb=2021.11.0=h00ab1b0_1
  - tbb-devel=2021.11.0=h5ccd973_1
  - tk=8.6.13=noxft_h4845f30_101
  - tzdata=2024a=h0c530f3_0
  - utfcpp=4.0.5=ha770c72_0
  - vtk=9.2.6=qt_py312h1234567_220
  - vtk-base=9.2.6=qt_py312h1234567_220
  - vtk-io-ffmpeg=9.2.6=qt_py312h1234567_220
  - wheel=0.42.0=pyhd8ed1ab_0
  - wslink=1.12.4=pyhd8ed1ab_0
  - x264=1!164.3095=h166bdaf_2
  - x265=3.5=h924138e_3
  - xcb-util=0.4.0=hd590300_1
  - xcb-util-image=0.4.0=h8ee46fc_1
  - xcb-util-keysyms=0.4.0=h8ee46fc_1
  - xcb-util-renderutil=0.3.9=hd590300_1
  - xcb-util-wm=0.4.1=h8ee46fc_1
  - xkeyboard-config=2.41=hd590300_0
  - xorg-fixesproto=5.0=h7f98852_1002
  - xorg-kbproto=1.0.7=h7f98852_1002
  - xorg-libice=1.1.1=hd590300_0
  - xorg-libsm=1.2.4=h7391055_0
  - xorg-libx11=1.8.7=h8ee46fc_0
  - xorg-libxau=1.0.11=hd590300_0
  - xorg-libxdmcp=1.1.3=h7f98852_0
  - xorg-libxext=1.3.4=h0b41bf4_2
  - xorg-libxfixes=5.0.3=h7f98852_1004
  - xorg-libxmu=1.1.3=h7f98852_0
  - xorg-libxrender=0.9.11=hd590300_0
  - xorg-libxt=1.3.0=hd590300_1
  - xorg-renderproto=0.11.1=h7f98852_1002
  - xorg-xextproto=7.3.0=h0b41bf4_1003
  - xorg-xf86vidmodeproto=2.3.1=h7f98852_1002
  - xorg-xproto=7.0.31=h7f98852_1007
  - xz=5.2.6=h166bdaf_0
  - yaml=0.2.5=h7f98852_2
  - yarl=1.9.4=py312h98912ed_0
  - zeromq=4.3.5=h59595ed_1
  - zfp=0.5.5=h9c3ff4c_8
  - zlib=1.2.13=hd590300_5
  - zlib-ng=2.0.7=h0b41bf4_0
  - zstd=1.5.5=hfc55251_0
prefix: /home/dokken/src/mambaforge/envs/test_mumps

@minrk any ideas? Another debug session this week?:slight_smile:

This is the problematic metis 5.1.1, which had been rolled back. I’m not sure why it’s getting picked up, but if you

mamba install -c conda-forge metis=5.1.0

you will get more recent builds of mumps and others that depend on metis. For me:

(base) root@1fa93d3c6f8b:/# mamba install -n metis-unbound metis=5.1.0

Looking for: ['metis=5.1.0']

conda-forge/linux-aarch64                                   Using cache
conda-forge/noarch                                          Using cache

Pinned packages:
  - python 3.12.*


Transaction

  Prefix: /opt/conda/envs/metis-unbound

  Updating specs:

   - metis=5.1.0
   - ca-certificates
   - openssl


  Package          Version  Build            Channel           Size
─────────────────────────────────────────────────────────────────────
  Change:
─────────────────────────────────────────────────────────────────────

  - mumps-include    5.6.2  h8af1aa0_2       conda-forge     Cached
  + mumps-include    5.6.2  h8af1aa0_4       conda-forge       27kB
  - suitesparse     5.10.1  h75d914d_2       conda-forge     Cached
  + suitesparse     5.10.1  h852e1e7_3       conda-forge        1MB
  - superlu_dist     8.2.1  h1047447_0       conda-forge     Cached
  + superlu_dist     8.2.1  h23a2dd0_1       conda-forge      717kB
  - kahip             3.16  py312h11e7e70_0  conda-forge     Cached
  + kahip             3.16  py312hcb0dd72_1  conda-forge        3MB
  - mumps-mpi        5.6.2  hfd937cd_2       conda-forge     Cached
  + mumps-mpi        5.6.2  h6741428_4       conda-forge        2MB
  - kahip-python      3.16  py312h75a5955_0  conda-forge     Cached
  + kahip-python      3.16  py312h75a5955_1  conda-forge       70kB

  Downgrade:
─────────────────────────────────────────────────────────────────────

  - metis            5.1.1  h2f0025b_2       conda-forge     Cached
  + metis            5.1.0  h2f0025b_1007    conda-forge        4MB

  Summary:

  Change: 6 packages
  Downgrade: 1 packages

  Total download: 11MB

─────────────────────────────────────────────────────────────────────

I’ll have to investigate why newer metis is being preferred to newer mumps builds.

3 Likes

The mumps builds that depend on metis 5.1.1 have been marked as broken, so they won’t get picked up anymore.

mamba install -c conda-forge  fenics-dolfinx petsc=*=complex* mpich gmsh python-gmsh

should once again produce a working environment with metis 5.1.0 by default.

2 Likes

The problem seems to be back in 0.8.0. I am using the complex build obtained through conda and metis is 5.1.0

EDIT: Apologies, I was using a different solver choice somehow. All seems good and no alarms.

1 Like

I guess I am once again running into this problem. My model that solves vector Helmholtz equation works fine in dolfinx 0.7 but not when I switch into 0.8, both installed through conda.

If needed, here is an example code. Apologies for simplifying it not more

import dolfinx, ufl, gmsh, petsc4py, mpi4py, numpy as np
from dolfinx.io import gmshio
from dolfinx.fem.petsc import LinearProblem


def create_geom(x0=0.0, y0=0.0, z0=0.0, sigma_src=0.0125, R1:float=0.5, fov_R:float=1.0, dpml=0.2):

    gmsh.initialize()
    gmsh.option.setNumber("General.Terminal", 0) # disable output messages
    gmsh.clear()
    gdim = 3
    model_rank = 0

    occ = gmsh.model.occ

    fov = occ.addSphere(0, 0, 0, fov_R, angle3=np.pi/2)
    sub = occ.addSphere(0, 0, 0, R1, angle3=np.pi/2)
    pml = occ.addSphere(0, 0, 0, fov_R-dpml, angle3=np.pi/2)
    point_circ = occ.addSphere(x0, y0, z0, sigma_src, angle3=np.pi/2)
    point_src = occ.addPoint(x0, y0, 0)
    occ.synchronize()
    frag, _ = occ.fragment([(gdim, fov)], [(gdim, sub), (gdim, pml), (0, point_src), (gdim, point_circ)])
    occ.synchronize()
    box = occ.addBox(0, 0, 0, fov_R, fov_R, fov_R)
    cut, _ = occ.intersect(frag, [(gdim, box)])
    occ.synchronize()
    
    # number all domains
    all_doms = gmsh.model.getEntities(gdim)
    for j, dom in enumerate(all_doms):
        gmsh.model.addPhysicalGroup(dom[0], [dom[1]], j + 1)  # create the main group/node

    # number all boundaries
    all_edges = gmsh.model.getEntities(gdim - 1)
    for j, edge in enumerate(all_edges):
        gmsh.model.addPhysicalGroup(edge[0], [edge[1]], edge[1])  # create the main group/node

    gmsh.model.mesh.generate(gdim)
    gmsh.model.mesh.optimize("Netgen")

    model_rank = 0
    mesh, ct, ft = gmshio.model_to_mesh(gmsh.model, mpi4py.MPI.COMM_WORLD, model_rank, gdim)
    return mesh, ct, ft

fem_order = 3

mesh, ct, ft = create_geom()

Omega = dolfinx.fem.functionspace(mesh, ("N1curl", fem_order))

U = ufl.TrialFunction(Omega)
Ut = ufl.TestFunction(Omega)

dx = ufl.Measure('dx', domain=mesh, subdomain_data=ct, metadata={'quadrature_degree':8})

k0 = dolfinx.fem.Constant(mesh, petsc4py.PETSc.ScalarType(2*np.pi/0.2))

F = (ufl.inner(ufl.curl(U), ufl.curl(Ut)) - \
     k0**2*ufl.inner(U, Ut))*dx

def gaussian_source(x, y, z, x0=0, y0=0, z0=0, sigma=0.0125):
    '''Gaussian approximation of Dirac delta function'''
    r_vec = ufl.as_vector((x, y, z))
    r0_vec = ufl.as_vector((x0, y0, z0))

    return ufl.exp(-ufl.dot((r_vec-r0_vec),(r_vec-r0_vec))/(sigma**2))/(sigma*np.sqrt(np.pi))

Z0 = 376.730313412
x_ufl, y_ufl, z_ufl = ufl.SpatialCoordinate(mesh)

dirac_delta = gaussian_source(x_ufl, y_ufl, z_ufl)

J_src = -1j*k0*ufl.as_vector((0, 0, dirac_delta)) # assume c->1

F += -ufl.inner(1j*Z0*k0*J_src, Ut)*dx

a = ufl.lhs(F)
L = ufl.rhs(F)

#%% solve
U_sol = dolfinx.fem.Function(Omega)

petsc_options = {"ksp_type": "preonly", "pc_type": "lu", "pc_factor_mat_solver_type":"mumps"}
Lin_system = LinearProblem(a, L, bcs=[], u=U_sol, petsc_options=petsc_options)

Lin_system.solve()

Could you please add information about how you installed this environment, and give the output of conda env export so that we have any chance of debugging or replicating this issue? See for instance: MUMPS solver does not work in complex mode - #5 by dokken for an illustration of this procedure.

Thank you for your feedback.

I just installed a new environment as follows:

mamba install -c conda-forge fenics-dolfinx petsc=*=complex* mpich gmsh python-gmsh dolfinx_mpc matplotlib ipykernel scipy

I confirmed that the problem persists. Here is the output of conda env export

name: fenics-complex
channels:
  - conda-forge
  - defaults
dependencies:
  - _libgcc_mutex=0.1=conda_forge
  - _openmp_mutex=4.5=2_gnu
  - _sysroot_linux-64_curr_repodata_hack=3=h69a702a_16
  - alsa-lib=1.2.12=h4ab18f5_0
  - asttokens=2.4.1=pyhd8ed1ab_0
  - binutils_impl_linux-64=2.40=ha1999f0_7
  - binutils_linux-64=2.40=hb3c18ed_2
  - brotli=1.1.0=hb9d3cd8_2
  - brotli-bin=1.1.0=hb9d3cd8_2
  - bzip2=1.0.8=h4bc722e_7
  - c-ares=1.33.1=heb4867d_0
  - c-blosc2=2.15.1=hc57e6cf_0
  - ca-certificates=2024.8.30=hbcca054_0
  - cairo=1.18.0=hebfffa5_3
  - certifi=2024.8.30=pyhd8ed1ab_0
  - cffi=1.17.1=py312h06ac9bb_0
  - comm=0.2.2=pyhd8ed1ab_0
  - contourpy=1.3.0=py312h68727a3_1
  - cycler=0.12.1=pyhd8ed1ab_0
  - dbus=1.13.6=h5008d03_3
  - debugpy=1.8.5=py312h2ec8cdc_1
  - decorator=5.1.1=pyhd8ed1ab_0
  - dolfinx_mpc=0.8.1=py312h733789c_0
  - double-conversion=3.3.0=h59595ed_0
  - exceptiongroup=1.2.2=pyhd8ed1ab_0
  - executing=2.1.0=pyhd8ed1ab_0
  - expat=2.6.3=h5888daf_0
  - fenics-basix=0.8.0=py312hcb56500_3
  - fenics-basix-nanobind-abi=0.1.9.13=h19b5320_3
  - fenics-dolfinx=0.8.0=py312he77254e_6
  - fenics-ffcx=0.8.0=pyh4af843d_0
  - fenics-libbasix=0.8.0=h7cb7ce6_3
  - fenics-libdolfinx=0.8.0=h03c8ada_6
  - fenics-ufcx=0.8.0=h22f594c_0
  - fenics-ufl=2024.1.0=pyhd8ed1ab_0
  - fftw=3.3.10=mpi_mpich_hbcf76dd_10
  - fltk=1.3.9=h9305793_1
  - font-ttf-dejavu-sans-mono=2.37=hab24e00_0
  - font-ttf-inconsolata=3.000=h77eed37_0
  - font-ttf-source-code-pro=2.038=h77eed37_0
  - font-ttf-ubuntu=0.83=h77eed37_2
  - fontconfig=2.14.2=h14ed4e7_0
  - fonts-conda-ecosystem=1=0
  - fonts-conda-forge=1=0
  - fonttools=4.53.1=py312h66e93f0_1
  - freeimage=3.18.0=h4b96d29_20
  - freetype=2.12.1=h267a509_2
  - gcc_impl_linux-64=13.3.0=hfea6d02_1
  - gcc_linux-64=13.3.0=hc28eda2_2
  - gmp=6.3.0=hac33072_2
  - gmsh=4.12.2=h6b98cf8_0
  - graphite2=1.3.13=h59595ed_1003
  - gxx_impl_linux-64=13.3.0=hdbfa832_1
  - gxx_linux-64=13.3.0=h6834431_2
  - harfbuzz=9.0.0=hda332d3_1
  - hdf5=1.14.3=mpi_mpich_h0f54ddc_5
  - hypre=2.31.0=mpi_mpich_hd1da18f_1
  - icu=75.1=he02047a_0
  - imath=3.1.11=hfc55251_0
  - importlib-metadata=8.4.0=pyha770c72_0
  - importlib_metadata=8.4.0=hd8ed1ab_0
  - ipykernel=6.29.5=pyh3099207_0
  - ipython=8.27.0=pyh707e725_0
  - jedi=0.19.1=pyhd8ed1ab_0
  - jupyter_client=8.6.2=pyhd8ed1ab_0
  - jupyter_core=5.7.2=py312h7900ff3_0
  - jxrlib=1.1=hd590300_3
  - kahip=3.16=h2fbc463_4
  - kahip-python=3.16=py312ha7be871_4
  - kernel-headers_linux-64=3.10.0=h4a8ded7_16
  - keyutils=1.6.1=h166bdaf_0
  - kiwisolver=1.4.7=py312h68727a3_0
  - krb5=1.21.3=h659f571_0
  - lcms2=2.16=hb7c19ff_0
  - ld_impl_linux-64=2.40=hf3520f5_7
  - lerc=4.0.0=h27087fc_0
  - libadios2=2.10.1=mpi_mpich_hb885cfe_3
  - libaec=1.1.3=h59595ed_0
  - libblas=3.9.0=23_linux64_openblas
  - libboost=1.86.0=hb8260a3_2
  - libboost-devel=1.86.0=h1a2810e_2
  - libboost-headers=1.86.0=ha770c72_2
  - libbrotlicommon=1.1.0=hb9d3cd8_2
  - libbrotlidec=1.1.0=hb9d3cd8_2
  - libbrotlienc=1.1.0=hb9d3cd8_2
  - libcblas=3.9.0=23_linux64_openblas
  - libclang-cpp18.1=18.1.8=default_hf981a13_4
  - libclang13=18.1.8=default_h9def88c_4
  - libcups=2.3.3=h4637d8d_4
  - libcurl=8.9.1=hdb1bdb2_0
  - libdeflate=1.21=h4bc722e_0
  - libdolfinx_mpc=0.8.1=hc52a2bd_0
  - libdrm=2.4.123=hb9d3cd8_0
  - libedit=3.1.20191231=he28a2e2_2
  - libegl=1.7.0=ha4b6fd6_0
  - libev=4.33=hd590300_2
  - libexpat=2.6.3=h5888daf_0
  - libffi=3.4.2=h7f98852_5
  - libgcc=14.1.0=h77fa898_1
  - libgcc-devel_linux-64=13.3.0=h84ea5a7_101
  - libgcc-ng=14.1.0=h69a702a_1
  - libgfortran=14.1.0=h69a702a_1
  - libgfortran-ng=14.1.0=h69a702a_1
  - libgfortran5=14.1.0=hc5f4f2c_1
  - libgl=1.7.0=ha4b6fd6_0
  - libglib=2.80.3=h315aac3_2
  - libglu=9.0.0=ha6d2627_1004
  - libglvnd=1.7.0=ha4b6fd6_0
  - libglx=1.7.0=ha4b6fd6_0
  - libgomp=14.1.0=h77fa898_1
  - libhwloc=2.11.1=default_hecaa2ac_1000
  - libiconv=1.17=hd590300_2
  - libjpeg-turbo=3.0.0=hd590300_1
  - liblapack=3.9.0=23_linux64_openblas
  - libllvm18=18.1.8=h8b73ec9_2
  - libnghttp2=1.58.0=h47da74e_1
  - libnsl=2.0.1=hd590300_0
  - libopenblas=0.3.27=pthreads_hac2b453_1
  - libpciaccess=0.18=hd590300_0
  - libpng=1.6.43=h2797004_0
  - libpq=16.4=h2d7952a_1
  - libptscotch=7.0.4=h2376d02_5
  - libraw=0.21.1=h2a13503_2
  - libsanitizer=13.3.0=heb74ff8_1
  - libscotch=7.0.4=h3055ed5_5
  - libsodium=1.0.20=h4ab18f5_0
  - libsqlite=3.46.1=hadc24fc_0
  - libssh2=1.11.0=h0841786_0
  - libstdcxx=14.1.0=hc0a3c3a_1
  - libstdcxx-devel_linux-64=13.3.0=h84ea5a7_101
  - libstdcxx-ng=14.1.0=h4852527_1
  - libtiff=4.6.0=h46a8edc_4
  - libuuid=2.38.1=h0b41bf4_0
  - libwebp-base=1.4.0=hd590300_0
  - libxcb=1.16=hb9d3cd8_1
  - libxcrypt=4.4.36=hd590300_1
  - libxkbcommon=1.7.0=h2c5496b_1
  - libxml2=2.12.7=he7c6b58_4
  - libxslt=1.1.39=h76b75d6_0
  - libzlib=1.3.1=h4ab18f5_1
  - lz4-c=1.9.4=hcb278e6_0
  - matplotlib=3.9.2=py312h7900ff3_0
  - matplotlib-base=3.9.2=py312h854627b_0
  - matplotlib-inline=0.1.7=pyhd8ed1ab_0
  - metis=5.1.0=h59595ed_1007
  - mpfr=4.2.1=h90cbb55_3
  - mpi=1.0=mpich
  - mpi4py=4.0.0=py312h0068001_3
  - mpich=4.2.2=h4a7f18d_101
  - mumps-include=5.7.3=ha770c72_3
  - mumps-mpi=5.7.3=h7bb8a9e_3
  - munkres=1.1.4=pyh9f0ad1d_0
  - mysql-common=9.0.1=h70512c7_0
  - mysql-libs=9.0.1=ha479ceb_0
  - ncurses=6.5=he02047a_1
  - nest-asyncio=1.6.0=pyhd8ed1ab_0
  - numpy=2.1.1=py312h58c1407_0
  - occt=7.7.2=novtk_h130ccc2_101
  - openexr=3.2.2=haf962dd_1
  - openjpeg=2.5.2=h488ebb8_0
  - openssl=3.3.2=hb9d3cd8_0
  - packaging=24.1=pyhd8ed1ab_0
  - parmetis=4.0.3=h583469f_1006
  - parso=0.8.4=pyhd8ed1ab_0
  - pcre2=10.44=hba22ea6_2
  - petsc=3.21.5=complex_h89735a0_1
  - petsc4py=3.21.5=py312h2026a2d_1
  - pexpect=4.9.0=pyhd8ed1ab_0
  - pickleshare=0.7.5=py_1003
  - pillow=10.4.0=py312h287a98d_0
  - pip=24.2=pyh8b19718_1
  - pixman=0.43.2=h59595ed_0
  - pkg-config=0.29.2=h4bc722e_1009
  - platformdirs=4.3.2=pyhd8ed1ab_0
  - prompt-toolkit=3.0.47=pyha770c72_0
  - psutil=6.0.0=py312h66e93f0_1
  - pthread-stubs=0.4=h36c2ea0_1001
  - ptyprocess=0.7.0=pyhd3deb0d_0
  - pugixml=1.14=h59595ed_0
  - pure_eval=0.2.3=pyhd8ed1ab_0
  - pycparser=2.22=pyhd8ed1ab_0
  - pygments=2.18.0=pyhd8ed1ab_0
  - pyparsing=3.1.4=pyhd8ed1ab_0
  - pyside6=6.7.2=py312hb5137db_2
  - python=3.12.5=h2ad013b_0_cpython
  - python-dateutil=2.9.0=pyhd8ed1ab_0
  - python-gmsh=4.12.2=h57928b3_0
  - python_abi=3.12=5_cp312
  - pyzmq=26.2.0=py312hbf22597_2
  - qhull=2020.2=h434a139_5
  - qt6-main=6.7.2=hb12f9c5_5
  - rapidjson=1.1.0.post20240409=hac33072_1
  - readline=8.2=h8228510_1
  - scalapack=2.2.0=h417d24c_2
  - scipy=1.14.1=py312h7d485d2_0
  - setuptools=73.0.1=pyhd8ed1ab_0
  - six=1.16.0=pyh6c4a22f_0
  - slepc=3.21.1=complex_h6ae04c3_202
  - slepc4py=3.21.1=py312h2061eb7_104
  - stack_data=0.6.2=pyhd8ed1ab_0
  - suitesparse=7.8.2=hb42a789_0
  - superlu=5.2.2=h00795ac_0
  - superlu_dist=9.0.0=h3feb4ed_1
  - sysroot_linux-64=2.17=h4a8ded7_16
  - tbb=2021.13.0=h84d6215_0
  - tk=8.6.13=noxft_h4845f30_101
  - tornado=6.4.1=py312h66e93f0_1
  - traitlets=5.14.3=pyhd8ed1ab_0
  - typing_extensions=4.12.2=pyha770c72_0
  - tzdata=2024a=h8827d51_1
  - wayland=1.23.1=h3e06ad9_0
  - wcwidth=0.2.13=pyhd8ed1ab_0
  - wheel=0.44.0=pyhd8ed1ab_0
  - xcb-util=0.4.1=hb711507_2
  - xcb-util-cursor=0.1.4=h4ab18f5_2
  - xcb-util-image=0.4.0=hb711507_2
  - xcb-util-keysyms=0.4.1=hb711507_0
  - xcb-util-renderutil=0.3.10=hb711507_0
  - xcb-util-wm=0.4.2=hb711507_0
  - xkeyboard-config=2.42=h4ab18f5_0
  - xorg-fixesproto=5.0=h7f98852_1002
  - xorg-inputproto=2.3.2=h7f98852_1002
  - xorg-kbproto=1.0.7=h7f98852_1002
  - xorg-libice=1.1.1=hd590300_0
  - xorg-libsm=1.2.4=h7391055_0
  - xorg-libx11=1.8.9=hb711507_1
  - xorg-libxau=1.0.11=hd590300_0
  - xorg-libxdmcp=1.1.3=h7f98852_0
  - xorg-libxext=1.3.4=h0b41bf4_2
  - xorg-libxfixes=5.0.3=h7f98852_1004
  - xorg-libxi=1.7.10=h4bc722e_1
  - xorg-libxmu=1.1.3=h4ab18f5_1
  - xorg-libxrender=0.9.11=hd590300_0
  - xorg-libxt=1.3.0=hd590300_1
  - xorg-libxtst=1.2.5=h4bc722e_0
  - xorg-libxxf86vm=1.1.5=h4bc722e_1
  - xorg-recordproto=1.14.2=h7f98852_1002
  - xorg-renderproto=0.11.1=h7f98852_1002
  - xorg-xextproto=7.3.0=h0b41bf4_1003
  - xorg-xproto=7.0.31=h7f98852_1007
  - xz=5.2.6=h166bdaf_0
  - yaml=0.2.5=h7f98852_2
  - zeromq=4.3.5=ha4adb4c_5
  - zfp=0.5.5=h9c3ff4c_8
  - zipp=3.20.1=pyhd8ed1ab_0
  - zlib=1.3.1=h4ab18f5_1
  - zlib-ng=2.2.1=he02047a_0
  - zstd=1.5.6=ha6fb4c9_0
prefix: /home/user/miniconda3/envs/fenics-complex

If I could speculate, the problem perhaps arises when dolfinx_mpc is added to the mix. I am sorry but I am too tired at this hour to redo and confirm.