JIT compilation timed out in DOLFINx v0.8.0

dolfinx
1

Met RuntimeError: Failed just-in-time compilation of form: JIT compilation timed out, probably due to a failed previous compile... when submitted a large number of cases on HPC.

Say I have code in main.py. I submitted n cases with command line argument in order by

$ mpirun -n 12 python main.py -case 1
$ mpirun -n 12 python main.py -case 2
...
$ mpirun -n 12 python main.py -case n-1
$ mpirun -n 12 python main.py -case n

Each above execution command generates a running folder and all output data are stored over there (case1, case2,… casen). However, just case1 has normal output, while others get the runtime error as mentioned above.

I tried to modified the cache_dir of jit.DOLFINX_DEFAULT_JIT_OPTIONS, but the following code returns RuntimeError: Failed just-in-time compilation of form: Compilation failed on root node. Any suggestion?

MWE:

from mpi4py import MPI
import ufl
from dolfinx import jit, fem, mesh

# Get case ID from CLI and mkdir, say "case3" for example.
# ...

jit.DOLFINX_DEFAULT_JIT_OPTIONS.update({"cache_dir": "case3"})

msh = mesh.create_unit_square(MPI.COMM_WORLD, 4, 4)
V = fem.functionspace(msh, ("Lagrange", 1, (msh.topology.dim, )))
u = fem.Function(V)
L = fem.form(ufl.div(u) * ufl.dx)
2

A less invasive way to reach the same goal could be to set the XDG_CACHE_HOME environment variable, which is then used to form the cache_dir. Can you check if that makes any difference?

3

No, I did the following, but the cache files libffcx_* are still created at $HOME/.cache/fenics folder.

import os
os.environ["XDG_CACHE_HOME"] = "/path/to/case3"
4

You haven’t provided a reproducible example of your last attempt. I can only guess that you import os and dolfinx, and afterwards set the environment variable. That is wrong: you must import os, set the environment variable, and then import dolfinx.