Error running nonlinear mixed problem in Apptainer container (PETSc / dolfinx environment issue?)

I am encountering an error when running this minimal working example (see below) of a nonlinear mixed FEniCSx problem inside an HPC environment using Apptainer. I built the FEniCSx image inside the container, but I am unsure whether the issue is related to PETSc configuration, MPI, or how dolfinx was built inside the container.

Issue : I repeatedly encounter Runtime Error PETSc code 63, and AttributeError of ‘NonlinearProblem’ object having no attribute ‘_snes’. This is when trying to initialize the nonlinear problem instance in the code line

problem = NonlinearProblem(F, U, J=fem.form(J), bcs=[bc], petsc_options_prefix=petsc_options)

The FEniCSx configuration I have inside the container is :

Python 3.12.3
/dolfinx-env/bin/python
fenics-basix            0.10.0
fenics-dolfinx          0.10.0
fenics-ufl              2025.2.0
mpi4py                  4.1.1
petsc4py                3.24.0

Here is the MWE :

from petsc4py import PETSc
from mpi4py import MPI
import ufl
import numpy as np
from dolfinx import mesh, fem, io, nls, log, io
from dolfinx.fem.petsc import NonlinearProblem
from dolfinx.nls.petsc import NewtonSolver
import os
import basix

a = 10  # domain size
b = 6 # height of initial gaussian 
c = 5e-2 # stdev of initial gaussian
kappa = 0.1 # diffusion coefficient
Pe = 0.5 # Desired Peclet Number (to compute mesh size, always <1)
cfl = 0.5
alpha = 1e-3  # reaction term
q = 1 # constant in the beta-term, equal to q/epsilon where q=1
epsilon = 0.3 # constant in the reaction term
u_pc = 3 # limit in the step-function
t = 0  # Start time
T = 30  # End time

max_wind = 1
nx, ny = 200,200
h = (a/200)
dt = 0.1
num_steps = np.int64(T//dt)

# initialize domain
bottom_left = np.array([-a,-a])
top_right = np.array([a,a])
domain = mesh.create_rectangle(MPI.COMM_WORLD, [bottom_left, top_right], [nx, nx], mesh.CellType.triangle)
el_u = basix.ufl.element("Lagrange", domain.basix_cell(), 1)
el_beta = basix.ufl.element("Lagrange", domain.basix_cell(), 1)
el_mixed = basix.ufl.mixed_element([el_u, el_u])
mixed_space = fem.functionspace(domain, el_mixed)

# set homogenous dirichlet bcs
w_D = fem.Function(mixed_space)
w_D.x.array[:] = 0.0
fdim = domain.topology.dim - 1
boundary_facets = mesh.locate_entities_boundary(domain, fdim, lambda x: np.full(x.shape[1], True, dtype=bool))
wh = fem.Function(mixed_space)
dofs = fem.locate_dofs_topological(mixed_space, fdim, boundary_facets)
bc = fem.dirichletbc(w_D, dofs)
bc.set(wh.x.array)
wh.x.scatter_forward()

# setup variational form
# Test functions and unknown
V0 = mixed_space.sub(0).collapse()[0]
V1 = mixed_space.sub(1).collapse()[0]
v_, w_ = ufl.TestFunctions(mixed_space)
U = fem.Function(mixed_space)          # unknown solution
u_, beta_ = ufl.split(U)

U_prev = fem.Function(mixed_space)
a, b, c = 20, 6, 5e-2
def ic_func_u(x):
    y = 6 * np.exp( -0.05* ((x[0]+9)**2 + (x[1]+9)**2))
    return y

def ic_func_beta(x):
    # x has shape (gdim, npoints)
    npoints = x.shape[1]
    return np.random.uniform(0.0, 1.0, size=npoints)

U_prev.sub(0).interpolate(ic_func_u)
U_prev.sub(1).interpolate(ic_func_beta)

u_prev_fn, beta_prev_fn = U_prev.split()

u_pc_const = fem.Constant(domain, PETSc.ScalarType(u_pc))
delta_const = fem.Constant(domain, PETSc.ScalarType(0.05))
t = (u_ - (u_pc - delta_const)) / (2.0 * delta_const)
# polynomial ramp
ramp = 3*t**2 - 2*t**3
H = ufl.conditional(ufl.lt(u_, u_pc - delta_const), 0.0, ufl.conditional(ufl.gt(u_, u_pc + delta_const), 1.0, ramp))

dt_const = fem.Constant(domain, PETSc.ScalarType(dt))
epsilon_const = fem.Constant(domain, PETSc.ScalarType(epsilon))
q_const = fem.Constant(domain, PETSc.ScalarType(q))
kappa_const = fem.Constant(domain, PETSc.ScalarType(kappa))
alpha_const = fem.Constant(domain, PETSc.ScalarType(alpha))

u_term = u_ / (1 + epsilon_const * u_)

F1 = ((-H * (epsilon_const / q_const) * ufl.exp(u_term) * beta_) * dt_const * v_
        - beta_ * v_
        + beta_prev_fn * v_) * ufl.dx

val_c = fem.Constant(domain, PETSc.ScalarType(float(np.sqrt(2)/(2))))
wind_vec = ufl.as_vector([val_c, val_c])

F2 = ((kappa_const * ufl.dot(ufl.grad(u_), ufl.grad(w_))
        + w_ * ufl.dot(wind_vec, ufl.grad(u_))
        - H * beta_ * ufl.exp(u_term) * w_
        + alpha_const * u_ * w_) * dt_const
        + u_ * w_ - u_prev_fn * w_) * ufl.dx

F = F1 + F2

u_sol   = fem.Function(V0)
beta_sol = fem.Function(V1)

# setup solver
problem = NonlinearProblem(F, U, bcs=[bc])
solver = NewtonSolver(MPI.COMM_WORLD, problem)

# configure linear solver 
solver.report = False
solver.convergence_criterion = "incremental"
solver.error_on_nonconvergence = True
solver.rtol = np.sqrt(np.finfo(float).eps) * 1e-2

ksp = solver.krylov_solver
opts = PETSc.Options()
prefix = ksp.getOptionsPrefix()

option_prefix = ksp.getOptionsPrefix()
opts[f"{option_prefix}ksp_type"] = "preonly"
opts[f"{option_prefix}pc_type"] = "lu"
opts[f"{prefix}ksp_rtol"] = 1e-8
opts[f"{prefix}ksp_max_it"] = 1000
sys = PETSc.Sys()

if sys.hasExternalPackage("superlu_dist"):
    opts[f"{option_prefix}pc_factor_mat_solver_type"] = "superlu_dist"
elif sys.hasExternalPackage("mumps"):
    opts[f"{option_prefix}pc_factor_mat_solver_type"] = "mumps"

ksp.setFromOptions()

solver.solve(U) 

Give you provide the full error traceback? This is usually a symptom of another issue.

For instance, running your code on 0.10 through docker, I get:

Exception ignored in: <function NonlinearProblem.__del__ at 0x740e5a340360>
Traceback (most recent call last):
  File "/usr/local/dolfinx-real/lib/python3.12/dist-packages/dolfinx/fem/petsc.py", line 1361, in __del__
    lambda obj: obj is not None, (self._snes, self._A, self._b, self._x, self._P_mat)
                                  ^^^^^^^^^^
AttributeError: 'NonlinearProblem' object has no attribute '_snes'
Traceback (most recent call last):
  File "/root/shared/mwe_solve.py", line 108, in <module>
    problem = NonlinearProblem(F, U, bcs=[bc])
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: NonlinearProblem.__init__() missing 1 required keyword-only argument: 'petsc_options_prefix'

This is due to an API change, specified in Release notes — DOLFINx 0.10.0.post5 documentation
i.e. using the solver setup:


petsc_options = {"ksp_type":"preonly",
                 "pc_type":"lu",
                 "snes_monitor": None,
                 "snes_linesearch_type": "none",
                 "snes_error_if_not_converged": True,
                 "snes_atol": 0,
                 "snes_rtol": 0,
                 "snes_stol": np.sqrt(np.finfo(float).eps) * 1e-2}
sys = PETSc.Sys()
if sys.hasExternalPackage("superlu_dist"):
    petsc_options["pc_factor_mat_solver_type"] = "superlu_dist"
elif sys.hasExternalPackage("mumps"):
    petsc_options["pc_factor_mat_solver_type"] = "mumps"

problem = NonlinearProblem(F, U, bcs=[bc],
                           petsc_options=petsc_options, petsc_options_prefix="nonlinearsolver")

problem.solve()

gives you a running code with the following output:

  0 SNES Function norm 2.464739905341e+00
  1 SNES Function norm 1.577760486205e-01
  2 SNES Function norm 8.817142191449e-02
  3 SNES Function norm 2.780274518105e-03
  4 SNES Function norm 1.143643031644e-04
  5 SNES Function norm 8.061773263418e-07
  6 SNES Function norm 5.996084022451e-09
  7 SNES Function norm 8.083087974506e-11
  8 SNES Function norm 3.059821852641e-12

The full error traceback is as follows :

Traceback (most recent call last):
  File "/home/develop/mwe.py", line 123, in <module>
    problem = NonlinearProblem(F, U, bcs=[bc],
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/dolfinx-real/lib/python3.12/dist-packages/dolfinx/fem/petsc.py", line 1272, in __init__
    self._A = create_matrix(self.J, kind=kind)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/dolfinx-real/lib/python3.12/dist-packages/dolfinx/fem/petsc.py", line 198, in create_matrix
    return _cpp.fem.petsc.create_matrix(a._cpp_object, kind)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Failed to successfully call PETSc function 'MatXIJSetPreallocation'. PETSc error code is: 63, Argument out of range
Exception ignored in: <function NonlinearProblem.__del__ at 0x7fdd4063c4a0>
Traceback (most recent call last):
  File "/usr/local/dolfinx-real/lib/python3.12/dist-packages/dolfinx/fem/petsc.py", line 1361, in __del__
    lambda obj: obj is not None, (self._snes, self._A, self._b, self._x, self._P_mat)
                                  ^^^^^^^^^^
AttributeError: 'NonlinearProblem' object has no attribute '_snes'

Even using the solver setup you have suggested, I seem to get this same error when running the MWE in my container.

This seems more like an issue with apptainer.
I would have to have the full set of instructions you used for installation to try to reproduce
(i.e. what docker container is the singularity/apptainer based on, and how are you using it on the HPC system)

Could you try running:

from mpi4py import MPI
from petsc4py import PETSc
import ufl
import numpy as np
from dolfinx import mesh, fem
import basix
import dolfinx.fem.petsc

a = 10  # domain size
b = 6  # height of initial gaussian
c = 5e-2  # stdev of initial gaussian
kappa = 0.1  # diffusion coefficient
Pe = 0.5  # Desired Peclet Number (to compute mesh size, always <1)
cfl = 0.5
alpha = 1e-3  # reaction term
q = 1  # constant in the beta-term, equal to q/epsilon where q=1
epsilon = 0.3  # constant in the reaction term
u_pc = 3  # limit in the step-function
t = 0  # Start time
T = 30  # End time

max_wind = 1
nx, ny = 200, 200
h = a / 200
dt = 0.1
num_steps = np.int64(T // dt)

# initialize domain
bottom_left = np.array([-a, -a])
top_right = np.array([a, a])
domain = mesh.create_rectangle(
    MPI.COMM_WORLD, [bottom_left, top_right], [nx, nx], mesh.CellType.triangle
)
el_u = basix.ufl.element("Lagrange", domain.basix_cell(), 1)
el_beta = basix.ufl.element("Lagrange", domain.basix_cell(), 1)
el_mixed = basix.ufl.mixed_element([el_u, el_u])
mixed_space = fem.functionspace(domain, el_mixed)

# set homogenous dirichlet bcs
w_D = fem.Function(mixed_space)
w_D.x.array[:] = 0.0
fdim = domain.topology.dim - 1
boundary_facets = mesh.locate_entities_boundary(
    domain, fdim, lambda x: np.full(x.shape[1], True, dtype=bool)
)
wh = fem.Function(mixed_space)
dofs = fem.locate_dofs_topological(mixed_space, fdim, boundary_facets)
bc = fem.dirichletbc(w_D, dofs)
bc.set(wh.x.array)
wh.x.scatter_forward()

# setup variational form
# Test functions and unknown
V0 = mixed_space.sub(0).collapse()[0]
V1 = mixed_space.sub(1).collapse()[0]
v_, w_ = ufl.TestFunctions(mixed_space)
U = fem.Function(mixed_space)  # unknown solution
u_, beta_ = ufl.split(U)

U_prev = fem.Function(mixed_space)
a, b, c = 20, 6, 5e-2


def ic_func_u(x):
    y = 6 * np.exp(-0.05 * ((x[0] + 9) ** 2 + (x[1] + 9) ** 2))
    return y


def ic_func_beta(x):
    # x has shape (gdim, npoints)
    npoints = x.shape[1]
    return np.random.uniform(0.0, 1.0, size=npoints)


U_prev.sub(0).interpolate(ic_func_u)
U_prev.sub(1).interpolate(ic_func_beta)

u_prev_fn, beta_prev_fn = U_prev.split()

u_pc_const = fem.Constant(domain, PETSc.ScalarType(u_pc))
delta_const = fem.Constant(domain, PETSc.ScalarType(0.05))
t = (u_ - (u_pc - delta_const)) / (2.0 * delta_const)
# polynomial ramp
ramp = 3 * t**2 - 2 * t**3
H = ufl.conditional(
    ufl.lt(u_, u_pc - delta_const),
    0.0,
    ufl.conditional(ufl.gt(u_, u_pc + delta_const), 1.0, ramp),
)

dt_const = fem.Constant(domain, PETSc.ScalarType(dt))
epsilon_const = fem.Constant(domain, PETSc.ScalarType(epsilon))
q_const = fem.Constant(domain, PETSc.ScalarType(q))
kappa_const = fem.Constant(domain, PETSc.ScalarType(kappa))
alpha_const = fem.Constant(domain, PETSc.ScalarType(alpha))

u_term = u_ / (1 + epsilon_const * u_)

F1 = (
    (-H * (epsilon_const / q_const) * ufl.exp(u_term) * beta_) * dt_const * v_
    - beta_ * v_
    + beta_prev_fn * v_
) * ufl.dx

val_c = fem.Constant(domain, PETSc.ScalarType(float(np.sqrt(2) / (2))))
wind_vec = ufl.as_vector([val_c, val_c])

F2 = (
    (
        kappa_const * ufl.dot(ufl.grad(u_), ufl.grad(w_))
        + w_ * ufl.dot(wind_vec, ufl.grad(u_))
        - H * beta_ * ufl.exp(u_term) * w_
        + alpha_const * u_ * w_
    )
    * dt_const
    + u_ * w_
    - u_prev_fn * w_
) * ufl.dx

F = F1 + F2

J = ufl.derivative(F, U)

A = dolfinx.fem.petsc.assemble_matrix(dolfinx.fem.form(J))
A.assemble()

The code that you have suggested also doesn’t run, the full error traceback it gives is :

Traceback (most recent call last):
  File "/home/develop/jsdokken_test.py", line 124, in <module>
    A = dolfinx.fem.petsc.assemble_matrix(dolfinx.fem.form(J))
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/functools.py", line 909, in wrapper
    return dispatch(args[0].__class__)(*args, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/dolfinx-real/lib/python3.12/dist-packages/dolfinx/fem/petsc.py", line 399, in assemble_matrix
    A = create_matrix(a, kind)
        ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/dolfinx-real/lib/python3.12/dist-packages/dolfinx/fem/petsc.py", line 198, in create_matrix
    return _cpp.fem.petsc.create_matrix(a._cpp_object, kind)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Failed to successfully call PETSc function 'MatXIJSetPreallocation'. PETSc error code is: 63, Argument out of range

The instructions I use to create the FEniCSx image in my container are

` apptainer build images/fenicsx.sif utils/sshd_support_fenicsx.def `

and the ssdh_supprt_fenicsx.def file is as follows :

Bootstrap: docker
From: dolfinx/dolfinx:stable

%files
    # Copying container startup script
    utils/startup.sh /startup.sh

%post
    # Allowing entrypoint execution
    chmod +x /startup.sh

    # Optional: installing sshd (useful for VSCode)
    apt update && apt install -y ssh vim
    apt clean

    # Adjust to your username
    MY_USER=""
    # In non fakeroot, making user have a password entry to satisfy
    # pam-auth.so
    echo "${MY_USER}:*:19774:0:99999:7:::" >> /etc/shadow
    # Removing tty group, preventing sshd from chgrp tty for /dev/pty
    # with unprivileged user
    mv /etc/group /etc/group.save
    grep -v -i "tty:x:5" /etc/group.save > /etc/group

    # Fix for rbnics images
    echo "export PYTHONPATH="/usr/local/dolfinx-real/lib/python3.12/dist-packages:/usr/local/lib::$PYTHONPATH"" >> /etc/profile.d/my_definition.sh ; \
    echo "export PATH=/dolfinx-env/bin:$PATH" >> /etc/profile.d/my_definition.sh ; \


%runscript

    source /etc/profile

    # Change the entrypoint
    exec /startup.sh "$@"

Let me know if you need any more information as well.

And it works with the same apptainer on a non HPC system (ie your own computer)?

I could not reproduce this with apptainer and a slightly modified definition from you:

Bootstrap: docker
From: dolfinx/dolfinx:stable

%files
    # Copying container startup script
    script.py /script.py

%post
    # Allowing entrypoint execution
    #chmod +x /startup.sh

    # Optional: installing sshd (useful for VSCode)
    apt update && apt install -y ssh vim
    apt clean

    # Adjust to your username
    MY_USER=""
    # In non fakeroot, making user have a password entry to satisfy
    # pam-auth.so
    echo "${MY_USER}:*:19774:0:99999:7:::" >> /etc/shadow
    # Removing tty group, preventing sshd from chgrp tty for /dev/pty
    # with unprivileged user
    mv /etc/group /etc/group.save
    grep -v -i "tty:x:5" /etc/group.save > /etc/group

    # Fix for rbnics images
    echo "export PYTHONPATH="/usr/local/dolfinx-real/lib/python3.12/dist-packages:/usr/local/lib::$PYTHONPATH"" >> /etc/profile.d/my_definition.sh ; \
    echo "export PATH=/dolfinx-env/bin:$PATH" >> /etc/profile.d/my_definition.sh ; \


%runscript

    source /etc/profile

    python3 script.py
    mpirun -n 2 python3 script.py

and

from mpi4py import MPI
from petsc4py import PETSc
import ufl
import numpy as np
from dolfinx import mesh, fem
import basix
import dolfinx.fem.petsc

a = 10  # domain size
b = 6  # height of initial gaussian
c = 5e-2  # stdev of initial gaussian
kappa = 0.1  # diffusion coefficient
Pe = 0.5  # Desired Peclet Number (to compute mesh size, always <1)
cfl = 0.5
alpha = 1e-3  # reaction term
q = 1  # constant in the beta-term, equal to q/epsilon where q=1
epsilon = 0.3  # constant in the reaction term
u_pc = 3  # limit in the step-function
t = 0  # Start time
T = 30  # End time

max_wind = 1
nx, ny = 200, 200
h = a / 200
dt = 0.1
num_steps = np.int64(T // dt)

# initialize domain
bottom_left = np.array([-a, -a])
top_right = np.array([a, a])
domain = mesh.create_rectangle(
    MPI.COMM_WORLD, [bottom_left, top_right], [nx, nx], mesh.CellType.triangle
)
el_u = basix.ufl.element("Lagrange", domain.basix_cell(), 1)
el_beta = basix.ufl.element("Lagrange", domain.basix_cell(), 1)
el_mixed = basix.ufl.mixed_element([el_u, el_u])
mixed_space = fem.functionspace(domain, el_mixed)

# set homogenous dirichlet bcs
w_D = fem.Function(mixed_space)
w_D.x.array[:] = 0.0
fdim = domain.topology.dim - 1
boundary_facets = mesh.locate_entities_boundary(
    domain, fdim, lambda x: np.full(x.shape[1], True, dtype=bool)
)
wh = fem.Function(mixed_space)
dofs = fem.locate_dofs_topological(mixed_space, fdim, boundary_facets)
bc = fem.dirichletbc(w_D, dofs)
bc.set(wh.x.array)
wh.x.scatter_forward()

# setup variational form
# Test functions and unknown
V0 = mixed_space.sub(0).collapse()[0]
V1 = mixed_space.sub(1).collapse()[0]
v_, w_ = ufl.TestFunctions(mixed_space)
U = fem.Function(mixed_space)  # unknown solution
u_, beta_ = ufl.split(U)

U_prev = fem.Function(mixed_space)
a, b, c = 20, 6, 5e-2


def ic_func_u(x):
    y = 6 * np.exp(-0.05 * ((x[0] + 9) ** 2 + (x[1] + 9) ** 2))
    return y


def ic_func_beta(x):
    # x has shape (gdim, npoints)
    npoints = x.shape[1]
    return np.random.uniform(0.0, 1.0, size=npoints)


U_prev.sub(0).interpolate(ic_func_u)
U_prev.sub(1).interpolate(ic_func_beta)

u_prev_fn, beta_prev_fn = U_prev.split()

u_pc_const = fem.Constant(domain, PETSc.ScalarType(u_pc))
delta_const = fem.Constant(domain, PETSc.ScalarType(0.05))
t = (u_ - (u_pc - delta_const)) / (2.0 * delta_const)
# polynomial ramp
ramp = 3 * t**2 - 2 * t**3
H = ufl.conditional(
    ufl.lt(u_, u_pc - delta_const),
    0.0,
    ufl.conditional(ufl.gt(u_, u_pc + delta_const), 1.0, ramp),
)

dt_const = fem.Constant(domain, PETSc.ScalarType(dt))
epsilon_const = fem.Constant(domain, PETSc.ScalarType(epsilon))
q_const = fem.Constant(domain, PETSc.ScalarType(q))
kappa_const = fem.Constant(domain, PETSc.ScalarType(kappa))
alpha_const = fem.Constant(domain, PETSc.ScalarType(alpha))

u_term = u_ / (1 + epsilon_const * u_)

F1 = (
    (-H * (epsilon_const / q_const) * ufl.exp(u_term) * beta_) * dt_const * v_
    - beta_ * v_
    + beta_prev_fn * v_
) * ufl.dx

val_c = fem.Constant(domain, PETSc.ScalarType(float(np.sqrt(2) / (2))))
wind_vec = ufl.as_vector([val_c, val_c])

F2 = (
    (
        kappa_const * ufl.dot(ufl.grad(u_), ufl.grad(w_))
        + w_ * ufl.dot(wind_vec, ufl.grad(u_))
        - H * beta_ * ufl.exp(u_term) * w_
        + alpha_const * u_ * w_
    )
    * dt_const
    + u_ * w_
    - u_prev_fn * w_
) * ufl.dx

F = F1 + F2

J = ufl.derivative(F, U)

A = dolfinx.fem.petsc.assemble_matrix(dolfinx.fem.form(J))
A.assemble()
print(MPI.COMM_WORLD.rank, MPI.COMM_WORLD.size, "Done", flush=True)

yielding

sudo ./install-dir/bin/apptainer run images/fenicsx.sif 
/.singularity.d/runscript: 4: source: not found
0 1 Done

0 2 Done
1 2 Done

Thus it is likely a mismatch with MPI (on your HPC system) and within singularity. Which MPI version is installed on the HPC system?