Fatal error in internal_Finalize

Errors
,
1

Hi FEniCS community,

I have encountered this problem for a bit, and I haven’t figured out what is happening. So I ran python3 myscript.py in the terminal, after the problem is solved, the program does not terminate and output the following error:

Abort(868846479): Fatal error in internal_Finalize: Other MPI error, error stack:
internal_Finalize(50)............: MPI_Finalize failed
MPII_Finalize(439)...............:
MPID_Finalize(803)...............:
MPIDI_OFI_mpi_finalize_hook(1065):
flush_send_queue(1024)...........:
MPIDI_OFI_handle_cq_error(788)...: OFI poll failed (default nic=utun6: Input/output error)

I looked at the forum and it seems like it has something to do with python garbage collection and/or MPI object destruction. The same issue is encountered when I run mpirun -n 1 python3 myscript.py This did not happen when I first installed FEniCSx using conda, but only recently. Could anybody explain what is going on and how to resolve this issue?

Appreciate your help in advance.

2

Without the actual script, it is quite hard to give any further help.

3

Sorry for the delay in reply, I was trying to see where is the problem, below is an MWE that reproduces the same error on my machine, I suppose there is another mpi installation on my machine under homebrew

which mpirun
/opt/homebrew/bin/mpirun

But even when I specifies the mpirun and python under the FEniCSx environment in Conda it still reproduces the same error.

import dolfinx
from petsc4py.PETSc import ScalarType  # type: ignore
from mpi4py import MPI
import numpy as np
import ufl
from dolfinx import fem, io, mesh, plot
from dolfinx.fem.petsc import LinearProblem

msh = mesh.create_rectangle(
    comm=MPI.COMM_WORLD,
    points=((0.0, 0.0), (2.0, 1.0)),
    n=(32, 16),
    cell_type=mesh.CellType.triangle,
)
V = fem.functionspace(msh, ("Lagrange", 1))


facets = mesh.locate_entities_boundary(
    msh,
    dim=(msh.topology.dim - 1),
    marker=lambda x: np.isclose(x[0], 0.0) | np.isclose(x[0], 2.0),
)

dofs = fem.locate_dofs_topological(V=V, entity_dim=1, entities=facets)

bc = fem.dirichletbc(value=ScalarType(0), dofs=dofs, V=V)

u = ufl.TrialFunction(V)
v = ufl.TestFunction(V)
x = ufl.SpatialCoordinate(msh)
f = 10 * ufl.exp(-((x[0] - 0.5) ** 2 + (x[1] - 0.5) ** 2) / 0.02)
g = ufl.sin(5 * x[0])
a = ufl.inner(ufl.grad(u), ufl.grad(v)) * ufl.dx
L = ufl.inner(f, v) * ufl.dx + ufl.inner(g, v) * ufl.ds

problem = LinearProblem(a, L, bcs=[bc], petsc_options={"ksp_type": "preonly", "pc_type": "lu"})
uh = problem.solve()
assert isinstance(uh, fem.Function)

try:
    import pyvista

    cells, types, x = plot.vtk_mesh(V)
    grid = pyvista.UnstructuredGrid(cells, types, x)
    grid.point_data["u"] = uh.x.array.real
    grid.set_active_scalars("u")
    plotter = pyvista.Plotter()
    plotter.add_mesh(grid, show_edges=True)
    warped = grid.warp_by_scalar()
    plotter.add_mesh(warped)
    if pyvista.OFF_SCREEN:
        pyvista.start_xvfb(wait=0.1)
        plotter.screenshot("uh_poisson.png")
    else:
        plotter.show()
except ModuleNotFoundError:
    print("'pyvista' is required to visualise the solution.")
    print("To install pyvista with pip: 'python3 -m pip install pyvista'.")
4

How exactly did you install DOLFINx using conda, and what is the output of
which mpirun and which python3?
For instance, on my system, where I get no errors, I use the following commands:

mamba create -n fenicsx-env
conda activate fenicsx-env
mamba install -c conda-forge fenics-dolfinx mpich pyvista

and I get:

(fenicsx-env) dokken@dokken-XPS-15-9560:~/Documents/debug$ which mpirun
/home/dokken/miniforge3/envs/fenicsx-env/bin/mpirun
(fenicsx-env) dokken@dokken-XPS-15-9560:~/Documents/debug$ which python3
/home/dokken/miniforge3/envs/fenicsx-env/bin/python3

and your example runs nicely.
Finally, what is the output of conda env export ?

5

I installed DOLFINx via

conda create -n fenicsx-env
conda activate fenicsx-env
conda install -c conda-forge fenics-dolfinx mpich pyvista dolfinx_mpc

After conda activate fenicsx-env, I have

❯ which mpirun
/opt/homebrew/bin/mpirun
❯ which python3
/opt/homebrew/bin/python3

and the output of conda env export is:

❯ conda env export
name: fenicsx-env
channels:
  - defaults
  - conda-forge
  - https://repo.anaconda.com/pkgs/main
  - https://repo.anaconda.com/pkgs/r
dependencies:
  - aiohappyeyeballs=2.6.1=pyhd8ed1ab_0
  - aiohttp=3.11.16=py313ha9b7d5b_0
  - aiosignal=1.3.2=pyhd8ed1ab_0
  - aom=3.9.1=h7bae524_0
  - appnope=0.1.4=pyhd8ed1ab_1
  - asttokens=3.0.0=pyhd8ed1ab_1
  - attrs=25.3.0=pyh71513ae_0
  - blosc=1.21.6=h7dd00d9_1
  - brotli=1.1.0=hd74edd7_2
  - brotli-bin=1.1.0=hd74edd7_2
  - brotli-python=1.1.0=py313h3579c5c_2
  - bzip2=1.0.8=h99b78c6_7
  - c-ares=1.34.5=h5505292_0
  - c-blosc2=2.17.1=h9cbb436_0
  - ca-certificates=2025.4.26=hbd8a1cb_0
  - cairo=1.18.4=h6a3b0d2_0
  - cctools_osx-arm64=1010.6=h3b4f5d3_6
  - certifi=2025.4.26=pyhd8ed1ab_0
  - cffi=1.17.1=py313hc845a76_0
  - charset-normalizer=3.4.1=pyhd8ed1ab_0
  - clang=18.1.8=default_h474c9e2_9
  - clang-18=18.1.8=default_hf90f093_9
  - clang_impl_osx-arm64=18.1.8=h2ae9ea5_24
  - clang_osx-arm64=18.1.8=h07b0088_24
  - clangxx=18.1.8=default_h1ffe849_9
  - cloudpickle=3.0.0=py313hca03da5_0
  - colorama=0.4.6=pyhd8ed1ab_1
  - comm=0.2.2=pyhd8ed1ab_1
  - compiler-rt=18.1.8=h856b3c1_1
  - compiler-rt_osx-arm64=18.1.8=h832e737_1
  - contourpy=1.3.1=py313h0ebd0e5_0
  - cycler=0.12.1=pyhd8ed1ab_1
  - cyrus-sasl=2.1.27=h60b93bd_7
  - dav1d=1.2.1=hb547adb_0
  - dbus=1.13.6=h3818c69_3
  - debugpy=1.8.14=py313h928ef07_0
  - decorator=5.2.1=pyhd8ed1ab_0
  - dolfinx_mpc=0.9.1=py313h93a942c_0
  - double-conversion=3.3.1=h286801f_0
  - exceptiongroup=1.2.2=pyhd8ed1ab_1
  - executing=2.2.0=pyhd8ed1ab_0
  - expat=2.7.0=h286801f_0
  - fenics-basix=0.9.0=py313hd7981f8_2
  - fenics-basix-nanobind-abi=0.2.1.18=h9b63b7c_2
  - fenics-dolfinx=0.9.0=py313haaef0bd_111
  - fenics-ffcx=0.9.0=pyh2e48890_0
  - fenics-libbasix=0.9.0=hf7ae0cd_2
  - fenics-libdolfinx=0.9.0=py313he999d83_111
  - fenics-ufcx=0.9.0=hb7f7608_0
  - fenics-ufl=2024.2.0=pyhd8ed1ab_1
  - ffmpeg=7.1.1=gpl_h583251a_104
  - fftw=3.3.10=mpi_mpich_hd148bab_10
  - fmt=11.1.4=h440487c_1
  - font-ttf-dejavu-sans-mono=2.37=hab24e00_0
  - font-ttf-inconsolata=3.000=h77eed37_0
  - font-ttf-source-code-pro=2.038=h77eed37_0
  - font-ttf-ubuntu=0.83=h77eed37_3
  - fontconfig=2.15.0=h1383a14_1
  - fonts-conda-ecosystem=1=0
  - fonts-conda-forge=1=0
  - fonttools=4.57.0=py313ha9b7d5b_0
  - freetype=2.13.3=h1d14073_0
  - fribidi=1.0.10=h27ca646_0
  - frozenlist=1.5.0=py313ha9b7d5b_1
  - gdk-pixbuf=2.42.12=h7ddc832_0
  - gl2ps=1.4.2=hc97c1ff_1
  - glew=2.1.0=h9f76cd9_2
  - glib=2.84.0=heee381b_0
  - glib-tools=2.84.0=h1dc7a0c_0
  - gmp=6.3.0=h7bae524_2
  - graphite2=1.3.13=hebf3989_1003
  - gst-plugins-base=1.24.7=hb49d354_0
  - gstreamer=1.24.7=hc3f5269_0
  - h2=4.2.0=pyhd8ed1ab_0
  - harfbuzz=11.0.1=h371b746_0
  - hdf4=4.2.15=h2ee6834_7
  - hdf5=1.14.3=mpi_mpich_h13a04de_9
  - hpack=4.1.0=pyhd8ed1ab_0
  - hyperframe=6.1.0=pyhd8ed1ab_0
  - hypre=2.32.0=mpi_mpich_h189fe77_1
  - icu=75.1=hfee45f7_0
  - idna=3.10=pyhd8ed1ab_1
  - imageio=2.37.0=pyhfb79c49_0
  - importlib-metadata=8.6.1=pyha770c72_0
  - iniconfig=2.0.0=pyhd8ed1ab_1
  - ipykernel=6.29.5=pyh57ce528_0
  - ipython=8.30.0=py313hca03da5_0
  - ipython_pygments_lexers=1.1.1=pyhd8ed1ab_0
  - jedi=0.19.2=pyhd8ed1ab_1
  - jsoncpp=1.9.6=h726d253_1
  - jupyter_client=8.6.3=pyhd8ed1ab_1
  - jupyter_core=5.7.2=pyh31011fe_1
  - kahip=3.18=py313hf2f5181_0
  - kiwisolver=1.4.7=py313hf9c7212_0
  - krb5=1.21.3=h237132a_0
  - lame=3.100=h1a8c8d9_1003
  - lcms2=2.17=h7eeda09_0
  - ld64_osx-arm64=951.9=hb6b49e2_6
  - lerc=4.0.0=h9a09cb3_0
  - libabseil=20250127.1=cxx17_h07bc746_0
  - libadios2=2.10.2=mpi_mpich_hfe2f02c_5
  - libaec=1.1.3=hebf3989_0
  - libamd=3.3.3=h5087772_7100102
  - libass=0.17.3=h68e5b86_2
  - libblas=3.9.0=31_h10e41b3_openblas
  - libboost=1.86.0=hc9fb7c5_3
  - libboost-devel=1.86.0=hf450f58_3
  - libboost-headers=1.86.0=hce30654_3
  - libbrotlicommon=1.1.0=hd74edd7_2
  - libbrotlidec=1.1.0=hd74edd7_2
  - libbrotlienc=1.1.0=hd74edd7_2
  - libbtf=2.3.2=h99b4a89_7100102
  - libcamd=3.3.3=h99b4a89_7100102
  - libcblas=3.9.0=31_hb3479ef_openblas
  - libccolamd=3.3.4=h99b4a89_7100102
  - libcholmod=5.3.1=hbba04d7_7100102
  - libclang-cpp17=17.0.6=default_hf90f093_8
  - libclang-cpp18.1=18.1.8=default_hf90f093_9
  - libclang13=20.1.2=default_h81d93ff_0
  - libcolamd=3.3.4=h99b4a89_7100102
  - libcurl=8.13.0=h73640d1_0
  - libcxx=20.1.2=ha82da77_0
  - libcxx-devel=18.1.8=h6dc3340_8
  - libdeflate=1.23=hec38601_0
  - libdolfinx_mpc=0.9.1=h2534925_0
  - libedit=3.1.20250104=pl5321hafb1f1b_0
  - libev=4.33=h93a5062_2
  - libexpat=2.7.0=h286801f_0
  - libfabric=2.1.0=hce30654_0
  - libfabric1=2.1.0=h5505292_0
  - libffi=3.4.6=h1da3d7d_1
  - libgfortran=14.2.0=heb5dd2a_105
  - libgfortran5=14.2.0=h2c44a93_105
  - libglib=2.84.0=hdff4504_0
  - libhwloc=2.11.2=default_hbce5d74_1001
  - libiconv=1.18=hfe07756_1
  - libintl=0.23.1=h493aca8_0
  - libintl-devel=0.23.1=h493aca8_0
  - libjpeg-turbo=3.0.0=hb547adb_1
  - libklu=2.3.5=h4370aa4_7100102
  - liblapack=3.9.0=31_hc9a63f6_openblas
  - libllvm17=17.0.6=hc4b4ae8_3
  - libllvm18=18.1.8=hc4b4ae8_3
  - libllvm20=20.1.2=hc4b4ae8_0
  - liblzma=5.8.1=h39f12f2_0
  - libmpdec=4.0.0=h99b78c6_0
  - libnetcdf=4.9.2=nompi_h610d594_116
  - libnghttp2=1.64.0=h6d7220d_0
  - libntlm=1.8=h5505292_0
  - libogg=1.3.5=h99b78c6_0
  - libopenblas=0.3.29=openmp_hf332438_0
  - libopenvino=2025.0.0=h3f17238_3
  - libopenvino-arm-cpu-plugin=2025.0.0=h3f17238_3
  - libopenvino-auto-batch-plugin=2025.0.0=h7f72211_3
  - libopenvino-auto-plugin=2025.0.0=h7f72211_3
  - libopenvino-hetero-plugin=2025.0.0=h718ad69_3
  - libopenvino-ir-frontend=2025.0.0=h718ad69_3
  - libopenvino-onnx-frontend=2025.0.0=h1ae5b81_3
  - libopenvino-paddle-frontend=2025.0.0=h1ae5b81_3
  - libopenvino-pytorch-frontend=2025.0.0=h286801f_3
  - libopenvino-tensorflow-frontend=2025.0.0=heb6e3e1_3
  - libopenvino-tensorflow-lite-frontend=2025.0.0=h286801f_3
  - libopus=1.4=h48c0fde_0
  - libpng=1.6.47=h3783ad8_0
  - libpq=17.4=h6896619_1
  - libprotobuf=5.29.3=hccd9074_0
  - libptscotch=7.0.6=hcfc2647_1
  - librsvg=2.58.4=h266df6f_3
  - libscotch=7.0.6=hd10c9a7_1
  - libsodium=1.0.20=h99b78c6_0
  - libspqr=4.3.4=h775d698_7100102
  - libsqlite=3.49.1=h3f77e49_2
  - libssh2=1.11.1=h9cc3647_0
  - libsuitesparseconfig=7.10.1=h4a8fc20_7100102
  - libtheora=1.1.1=h99b78c6_1006
  - libtiff=4.7.0=h551f018_3
  - libumfpack=6.3.5=h7c2c975_7100102
  - libusb=1.0.28=h5505292_0
  - libvorbis=1.3.7=h9f76cd9_0
  - libvpx=1.14.1=h7bae524_0
  - libwebp-base=1.5.0=h2471fea_0
  - libxcb=1.17.0=hdb1d25a_0
  - libxml2=2.13.7=h52572c6_1
  - libzip=1.11.2=h1336266_0
  - libzlib=1.3.1=h8359307_2
  - llvm-openmp=20.1.2=hdb05f8b_0
  - llvm-tools=18.1.8=hc4b4ae8_3
  - llvm-tools-18=18.1.8=hc4b4ae8_3
  - loguru=0.7.2=py313h8f79df9_2
  - lz4-c=1.10.0=h286801f_1
  - matplotlib-base=3.10.1=py313haaf02c0_0
  - matplotlib-inline=0.1.7=pyhd8ed1ab_1
  - metis=5.1.0=h15f6cfe_1007
  - mpi=1.0.1=mpich
  - mpi4py=4.0.3=py313h208a61b_0
  - mpich=4.3.0=h9d9a6ae_100
  - msgpack-python=1.1.0=py313hf9c7212_0
  - multidict=6.3.2=py313h6347b5a_0
  - mumps-include=5.7.3=h71ed9e6_9
  - mumps-mpi=5.7.3=h2d32478_9
  - munkres=1.1.4=pyh9f0ad1d_0
  - mysql-common=9.0.1=hd7719f6_6
  - mysql-libs=9.0.1=ha8be5b7_6
  - ncurses=6.5=h5e97a16_3
  - nest-asyncio=1.6.0=pyhd8ed1ab_1
  - nlohmann_json=3.11.3=h00cdb27_1
  - nspr=4.36=h5833ebf_0
  - nss=3.110=ha3c76ea_0
  - numpy=2.2.4=py313h41a2e72_0
  - openh264=2.6.0=hb5b2745_0
  - openjpeg=2.5.3=h8a3d83b_0
  - openldap=2.6.9=hbe55e7a_0
  - openssl=3.5.0=h81ee809_1
  - packaging=24.2=pyhd8ed1ab_2
  - pango=1.56.3=h5fd7515_1
  - parmetis=4.0.3=ha4b917a_1007
  - parso=0.8.4=pyhd8ed1ab_1
  - pcre2=10.44=h297a79d_2
  - petsc=3.22.3=real_h8f1400a_103
  - petsc4py=3.22.4=np2py313h2f7c7fa_0
  - pexpect=4.9.0=pyhd8ed1ab_1
  - pickleshare=0.7.5=pyhd8ed1ab_1004
  - pillow=11.1.0=py313hb37fac4_0
  - pip=25.0.1=pyh145f28c_0
  - pixman=0.44.2=h2f9eb0b_0
  - pkg-config=0.29.2=hde07d2e_1009
  - platformdirs=4.3.7=pyh29332c3_0
  - pluggy=1.6.0=pyhd8ed1ab_0
  - ply=3.11=pyhd8ed1ab_3
  - pooch=1.8.2=pyhd8ed1ab_1
  - proj=9.5.1=h1318a7e_0
  - prompt-toolkit=3.0.43=py313hca03da5_0
  - prompt_toolkit=3.0.43=hd3eb1b0_0
  - propcache=0.2.1=py313ha9b7d5b_1
  - psutil=7.0.0=py313h90d716c_0
  - pthread-stubs=0.4=hd74edd7_1002
  - ptyprocess=0.7.0=pyhd8ed1ab_1
  - pugixml=1.15=hd3d436d_0
  - pure_eval=0.2.3=pyhd8ed1ab_1
  - pycparser=2.22=pyh29332c3_1
  - pygments=2.19.1=pyhd8ed1ab_0
  - pyparsing=3.2.3=pyhd8ed1ab_1
  - pyqt=5.15.10=py313hffa79ec_1
  - pyqt5-sip=12.13.0=py313h928ef07_1
  - pysocks=1.7.1=pyha55dd90_7
  - pytest=8.3.5=pyhd8ed1ab_0
  - python=3.13.2=h81fe080_101_cp313
  - python-dateutil=2.9.0.post0=pyhff2d567_1
  - python_abi=3.13=6_cp313
  - pyvista=0.44.2=pyhd8ed1ab_1
  - pyvistaqt=0.11.2=pyhdecd6ff_0
  - pyzmq=26.4.0=py313he6960b1_0
  - qhull=2020.2=h420ef59_5
  - qt-main=5.15.15=h67564f6_3
  - qt6-main=6.8.3=hca13b62_1
  - qtpy=2.4.3=pyhd8ed1ab_0
  - readline=8.2=h1d1bf99_2
  - requests=2.32.3=pyhd8ed1ab_1
  - scalapack=2.2.0=h71a4f75_4
  - scipy=1.15.2=py313h9a24e0a_0
  - scooby=0.10.0=pyhd8ed1ab_1
  - sdl2=2.32.54=h994913f_0
  - sdl3=3.2.10=he842692_0
  - setuptools=78.1.0=pyhff2d567_0
  - sigtool=0.1.3=h44b9a77_0
  - sip=6.10.0=py313h928ef07_0
  - six=1.17.0=pyhd8ed1ab_0
  - slepc=3.22.2=real_hf0b33f3_300
  - slepc4py=3.22.2=py313hac86436_0
  - snappy=1.2.1=h98b9ce2_1
  - spdlog=1.15.2=h008cadb_0
  - spyder-kernels=3.0.5=py313h7eb115d_0
  - sqlite=3.49.1=hd7222ec_2
  - stack_data=0.6.3=pyhd8ed1ab_1
  - superlu=7.0.0=h742a510_1
  - superlu_dist=9.1.0=h89afcdd_0
  - svt-av1=3.0.2=h8ab69cd_0
  - tapi=1300.6.5=h03f4b80_0
  - tbb=2022.1.0=h9541205_0
  - tk=8.6.13=h5083fa2_1
  - toml=0.10.2=pyhd8ed1ab_1
  - tomli=2.2.1=pyhd8ed1ab_1
  - tornado=6.4.2=py313h90d716c_0
  - tqdm=4.67.1=pyhd8ed1ab_1
  - traitlets=5.14.3=pyhd8ed1ab_1
  - typing-extensions=4.13.1=hf5ce1d7_0
  - typing_extensions=4.13.1=pyh29332c3_0
  - tzdata=2025b=h78e105d_0
  - urllib3=2.3.0=pyhd8ed1ab_0
  - utfcpp=4.0.6=h54c0426_0
  - vtk=9.3.1=qt_py313he4b582b_216
  - vtk-base=9.3.1=qt_py313hb5b88ee_216
  - vtk-io-ffmpeg=9.3.1=qt_py313he4b582b_216
  - wcwidth=0.2.13=pyhd8ed1ab_1
  - wslink=2.3.3=pyhd8ed1ab_0
  - wurlitzer=3.0.2=py313hca03da5_0
  - x264=1!164.3095=h57fd34a_2
  - x265=3.5=hbc6ce65_3
  - xorg-libxau=1.0.12=h5505292_0
  - xorg-libxdmcp=1.1.5=hd74edd7_0
  - yaml=0.2.5=h3422bc3_2
  - yarl=1.18.3=py313ha9b7d5b_1
  - zeromq=4.3.5=hc1bb282_7
  - zfp=1.0.1=h1c5d8ea_2
  - zipp=3.21.0=pyhd8ed1ab_1
  - zlib=1.3.1=h8359307_2
  - zlib-ng=2.2.4=h13dfb9a_0
  - zstandard=0.23.0=py313h90d716c_1
  - zstd=1.5.7=h6491c7d_2
  - pip:
      - gmsh==4.13.1
      - h5py==3.13.0
prefix: /Users/tweng/miniconda3/envs/fenicsx-env

I’m not exactly sure if this is due to I’m using tmux to edit and run scripts together, but when I’m not using tmux and just in terminal, the example runs nicely and the output is good.

❯ which mpirun
/Users/tweng/miniconda3/envs/fenicsx-env/bin/mpirun
❯ which python3
/Users/tweng/miniconda3/envs/fenicsx-env/bin/python3

Would you elaborate more on what’s possibly going on?

(Edit: I cannot reproduce the error now even within tmux, no idea what’s going wrong or right, the only clue is that I have had this other error before, which may be the cause of the error in the issue, and this is due to a misidentification of the physical groups of the boundary when implementing the boundary conditions.)

/Users/tweng/miniconda3/envs/fenicsx-env/lib/python3.13/multiprocessing/resource_tracker.py:277: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown: {'/mp-r2g37dg_'}
  warnings.warn(

===================================================================================
=   BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
=   PID 44727 RUNNING AT Daniels-MBP-6.lan
=   EXIT CODE: 11
=   CLEANING UP REMAINING PROCESSES
=   YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
===================================================================================
YOUR APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault: 11 (signal 11)
This typically refers to a problem with your application.
Please see the FAQ page for debugging suggestions
6

Apparently, also for future reference, this error has something to do with my mesh generation function using Gmsh. One has to be careful with it for improper use of gmsh.initialize() without gmsh.finalize() may occupy the process without the system even realizing it, and resulting in the aforementioned errors with processes run after that (even for scripts that does not call Gmsh like the Poisson example script). @dokken Thank you very much for your help!