Unable to found JIT module

[MPenaR]

Well, the title is the message error. Basically I have dolfin-x 0.3.0 installed (it is the one that comes with debian testing right now as far as I know), and I was looking into the documentation for this version for the poisson example:

I didn’t managed to replicate it as the command V = FunctionSpace(mesh, ("Lagrange", 1)) fails giving the next error message:

---------------------------------------------------------------------------
FileExistsError                           Traceback (most recent call last)
/usr/lib/python3/dist-packages/ffcx/codegeneration/jit.py in get_cached_module(module_name, object_names, cache_dir, timeout)
     61         # Create C file with exclusive access
---> 62         open(c_filename, "x")
     63         return None, None

FileExistsError: [Errno 17] File exists: '/home/manuel/.cache/fenics/libffcx_elements_0ba90af30174bac8aa16375eb5840f3f5018d54c.c'

During handling of the above exception, another exception occurred:

ModuleNotFoundError                       Traceback (most recent call last)
/tmp/ipykernel_469262/3084727201.py in <module>
      5     CellType.triangle, dolfinx.cpp.mesh.GhostMode.none)
      6 
----> 7 V = FunctionSpace(mesh, ("Lagrange", 1))

/usr/lib/petsc/lib/python3/dist-packages/dolfinx/fem/function.py in __init__(self, mesh, element, cppV, form_compiler_parameters, jit_parameters)
    383 
    384         # Compile dofmap and element and create DOLFIN objects
--> 385         (self._ufc_element, self._ufc_dofmap), module, code = jit.ffcx_jit(
    386             mesh.mpi_comm(), self.ufl_element(), form_compiler_parameters=form_compiler_parameters,
    387             jit_parameters=jit_parameters)

/usr/lib/petsc/lib/python3/dist-packages/dolfinx/jit.py in mpi_jit(comm, *args, **kwargs)
     60         # Just call JIT compiler when running in serial
     61         if comm.size == 1:
---> 62             return local_jit(*args, **kwargs)
     63 
     64         # Default status (0 == ok, 1 == fail)

/usr/lib/petsc/lib/python3/dist-packages/dolfinx/jit.py in ffcx_jit(ufl_object, form_compiler_parameters, jit_parameters)
    216         r = ffcx.codegeneration.jit.compile_forms([ufl_object], parameters=p_ffcx, **p_jit)
    217     elif isinstance(ufl_object, ufl.FiniteElementBase):
--> 218         r = ffcx.codegeneration.jit.compile_elements([ufl_object], parameters=p_ffcx, **p_jit)
    219     elif isinstance(ufl_object, ufl.Mesh):
    220         r = ffcx.codegeneration.jit.compile_coordinate_maps(

/usr/lib/python3/dist-packages/ffcx/codegeneration/jit.py in compile_elements(elements, parameters, cache_dir, timeout, cffi_extra_compile_args, cffi_verbose, cffi_debug, cffi_libraries)
    105     if cache_dir is not None:
    106         cache_dir = Path(cache_dir)
--> 107         obj, mod = get_cached_module(module_name, names, cache_dir, timeout)
    108         if obj is not None:
    109             # Pair up elements with dofmaps

/usr/lib/python3/dist-packages/ffcx/codegeneration/jit.py in get_cached_module(module_name, object_names, cache_dir, timeout)
     73                 spec = finder.find_spec(module_name)
     74                 if spec is None:
---> 75                     raise ModuleNotFoundError("Unable to find JIT module.")
     76                 compiled_module = importlib.util.module_from_spec(spec)
     77                 spec.loader.exec_module(compiled_module)

ModuleNotFoundError: Unable to find JIT module.

I don’t seem to find many people with this problem.

PS: I would like to add that the poisson example with dolfin (not the dolfin-x) seems to work properly. So for some reason I don’t understand dolfin finds the JIT but dolfin-x does not.

[dparsons]

@MPenaR

FileExistsError: [Errno 17] File exists: '/home/manuel/.cache/fenics/libffcx_elements_0ba90af30174bac8aa16375eb5840f3f5018d54c.c'

Your cache is probably corrupted, possibly due to version upgrades. Try deleting ~/.cache/fenics and try it again.

[MPenaR]

Thanks, works fine now. Not sure if I understood the problem although.

[dparsons]

The hash in the file name is intended to established uniqueness of the jit file within the context of the build environment. i.e. to mark whether it needs to be regenerated or not (the fenics run is faster if it doesn’t). But the hash signature doesn’t yet capture every nuance in the build environment, leading to the problem you experienced. cf. increasing the entropy of the jit hash · Issue #290 · FEniCS/ffcx · GitHub

[MPenaR]

I understand, thank you very much!

[Pasuta]

Could you please explain me in detail how to delete the cache, i couldn’t find where is cache located

[dparsons]

With dolfinx (ffcx), it’s now usually located at ~/.cache/fenics/. With older versions you might find it at ~/.ffcx_cache/ (or maybe that’s just my setup).

You can configure the location via the jit_options argument in LinearProblem or NonlinearProblem, which takes a dict of values (the cache location has keyname "cache_dir")
cf. Set JIT cache directory

[Pasuta]

I coudn’t find those files. I don’t know if everyone is like me but mine is coded on the Jupiter Lab website. And I can’t save those codes on my computer. I must save and export notebook as… and whenever i closed the Jupiter Lab i had to copy those code and patse it again.

[Pasuta]

this might be requesting too much but I will greatly appreciate it If you can show me how to do it and record the screen for me (forgot to mention that I ran it on Docker.) Another thing is that coding is something very new for me and I might be asking a lot.Thank you.

[dparsons]

I don’t use Jupyter or docker myself. I use a Linux environment.

[dokken]

If you run jupyterlab with docker, you should be able to open the terminal inside jupyter lab (it has a terminal option), and call dijitso clean.

[Pasuta]

I put the first line in terminal Jupyterlab. It says Permission denied.

~/.cache/fenics/libffcx_forms_5eb899f22d9b05f26c47e06ab454b4561b19ac5a.c

and when i call dijitso clean.

bash: dijitso: command not found

[dokken]

What docker image are you using and what commands are you using to launch Jupyterlab with docker?

[Pasuta]

I run with this command on window powershell, the command is from GitHub - FEniCS/dolfinx: Next generation FEniCS problem solving environment

docker run --init -ti -p 8888:8888 dolfinx/lab:stable

then i copy the URL after run ‘docker run’ command and paste it on google chrome to open it

http://127.0.0.1:8888/lab?token=95fb25d4089f88d63894dd83345d8d018b7c80fad1aad8b4

[dokken]

then it is no Wonder dijitso isn’t there, as dolfinx doesn’t use dijitso.

@Pasuta

~/.cache/fenics

Did you try rm -rf /root/.cache/fenics?

[Pasuta]

nothing happen

[Pasuta]

but the error is change from (you can see my problem from Calculate sensitivity equation of reaction-diffusion systems with dolfinx)

"TimeoutError: JIT compilation timed out, probably due to a failed previous compile.Try cleaning cache (e.g. remove /root/.cache/fenics/libffcx_forms_5eb899f22d9b05f26c47e06ab454b4561b19ac5a.c) or increase timeout option."

to this

---------------------------------------------------------------------------
ArityMismatch                             Traceback (most recent call last)
Cell In[121], line 1
----> 1 dJdm_form = dolfinx.fem.form(ufl.derivative(J, eps))

File /usr/local/dolfinx-complex/lib/python3.10/dist-packages/dolfinx/fem/forms.py:188, in form(form, dtype, form_compiler_options, jit_options)
    185         return list(map(lambda sub_form: _create_form(sub_form), form))
    186     return form
--> 188 return _create_form(form)

File /usr/local/dolfinx-complex/lib/python3.10/dist-packages/dolfinx/fem/forms.py:183, in form.<locals>._create_form(form)
    180 """Recursively convert ufl.Forms to dolfinx.fem.Form, otherwise
    181 return form argument"""
    182 if isinstance(form, ufl.Form):
--> 183     return _form(form)
    184 elif isinstance(form, collections.abc.Iterable):
    185     return list(map(lambda sub_form: _create_form(sub_form), form))

File /usr/local/dolfinx-complex/lib/python3.10/dist-packages/dolfinx/fem/forms.py:141, in form.<locals>._form(form)
    139 if mesh is None:
    140     raise RuntimeError("Expecting to find a Mesh in the form.")
--> 141 ufcx_form, module, code = jit.ffcx_jit(mesh.comm, form,
    142                                        form_compiler_options=form_compiler_options,
    143                                        jit_options=jit_options)
    145 # For each argument in form extract its function space
    146 V = [arg.ufl_function_space()._cpp_object for arg in form.arguments()]

File /usr/local/dolfinx-complex/lib/python3.10/dist-packages/dolfinx/jit.py:56, in mpi_jit_decorator.<locals>.mpi_jit(comm, *args, **kwargs)
     51 @functools.wraps(local_jit)
     52 def mpi_jit(comm, *args, **kwargs):
     53 
     54     # Just call JIT compiler when running in serial
     55     if comm.size == 1:
---> 56         return local_jit(*args, **kwargs)
     58     # Default status (0 == ok, 1 == fail)
     59     status = 0

File /usr/local/dolfinx-complex/lib/python3.10/dist-packages/dolfinx/jit.py:204, in ffcx_jit(ufl_object, form_compiler_options, jit_options)
    202 # Switch on type and compile, returning cffi object
    203 if isinstance(ufl_object, ufl.Form):
--> 204     r = ffcx.codegeneration.jit.compile_forms([ufl_object], options=p_ffcx, **p_jit)
    205 elif isinstance(ufl_object, ufl.FiniteElementBase):
    206     r = ffcx.codegeneration.jit.compile_elements([ufl_object], options=p_ffcx, **p_jit)

File /usr/local/lib/python3.10/dist-packages/ffcx/codegeneration/jit.py:190, in compile_forms(forms, options, cache_dir, timeout, cffi_extra_compile_args, cffi_verbose, cffi_debug, cffi_libraries)
    187     for name in form_names:
    188         decl += form_template.format(name=name)
--> 190     impl = _compile_objects(decl, forms, form_names, module_name, p, cache_dir,
    191                             cffi_extra_compile_args, cffi_verbose, cffi_debug, cffi_libraries)
    192 except Exception as e:
    193     try:
    194         # remove c file so that it will not timeout next time

File /usr/local/lib/python3.10/dist-packages/ffcx/codegeneration/jit.py:260, in _compile_objects(decl, ufl_objects, object_names, module_name, options, cache_dir, cffi_extra_compile_args, cffi_verbose, cffi_debug, cffi_libraries)
    256 import ffcx.compiler
    258 # JIT uses module_name as prefix, which is needed to make names of all struct/function
    259 # unique across modules
--> 260 _, code_body = ffcx.compiler.compile_ufl_objects(ufl_objects, prefix=module_name, options=options)
    262 ffibuilder = cffi.FFI()
    263 ffibuilder.set_source(module_name, code_body, include_dirs=[ffcx.codegeneration.get_include_path()],
    264                       extra_compile_args=cffi_extra_compile_args, libraries=cffi_libraries)

File /usr/local/lib/python3.10/dist-packages/ffcx/compiler.py:97, in compile_ufl_objects(ufl_objects, object_names, prefix, options, visualise)
     95 # Stage 1: analysis
     96 cpu_time = time()
---> 97 analysis = analyze_ufl_objects(ufl_objects, options)
     98 _print_timing(1, time() - cpu_time)
    100 # Stage 2: intermediate representation

File /usr/local/lib/python3.10/dist-packages/ffcx/analysis.py:88, in analyze_ufl_objects(ufl_objects, options)
     85     else:
     86         raise TypeError("UFL objects not recognised.")
---> 88 form_data = tuple(_analyze_form(form, options) for form in forms)
     89 for data in form_data:
     90     elements += [convert_element(e) for e in data.unique_sub_elements]

File /usr/local/lib/python3.10/dist-packages/ffcx/analysis.py:88, in <genexpr>(.0)
     85     else:
     86         raise TypeError("UFL objects not recognised.")
---> 88 form_data = tuple(_analyze_form(form, options) for form in forms)
     89 for data in form_data:
     90     elements += [convert_element(e) for e in data.unique_sub_elements]

File /usr/local/lib/python3.10/dist-packages/ffcx/analysis.py:163, in _analyze_form(form, options)
    160 complex_mode = "_Complex" in options["scalar_type"]
    162 # Compute form metadata
--> 163 form_data = ufl.algorithms.compute_form_data(
    164     form,
    165     do_apply_function_pullbacks=True,
    166     do_apply_integral_scaling=True,
    167     do_apply_geometry_lowering=True,
    168     preserve_geometry_types=(ufl.classes.Jacobian,),
    169     do_apply_restrictions=True,
    170     do_append_everywhere_integrals=False,  # do not add dx integrals to dx(i) in UFL
    171     complex_mode=complex_mode)
    173 # If form contains a quadrature element, use the custom quadrature scheme
    174 custom_q = None

File /usr/local/lib/python3.10/dist-packages/ufl/algorithms/compute_form_data.py:415, in compute_form_data(form, do_apply_function_pullbacks, do_apply_integral_scaling, do_apply_geometry_lowering, preserve_geometry_types, do_apply_default_restrictions, do_apply_restrictions, do_estimate_degrees, do_append_everywhere_integrals, complex_mode)
    412 preprocessed_form = reconstruct_form_from_integral_data(self.integral_data)
    414 # TODO: Test how fast this is
--> 415 check_form_arity(preprocessed_form, self.original_form.arguments(), complex_mode)
    417 # TODO: This member is used by unit tests, change the tests to
    418 # remove this!
    419 self.preprocessed_form = preprocessed_form

File /usr/local/lib/python3.10/dist-packages/ufl/algorithms/check_arities.py:198, in check_form_arity(form, arguments, complex_mode)
    196 """Check the arity of a form."""
    197 for itg in form.integrals():
--> 198     check_integrand_arity(itg.integrand(), arguments, complex_mode)

File /usr/local/lib/python3.10/dist-packages/ufl/algorithms/check_arities.py:190, in check_integrand_arity(expr, arguments, complex_mode)
    188 for arg, conj in arg_tuples:
    189     if arg.number() == 0 and not conj:
--> 190         raise ArityMismatch("Failure to conjugate test function in complex Form")
    191     elif arg.number() > 0 and conj:
    192         raise ArityMismatch(f"Argument {arg} is spuriously conjugated in complex Form")

ArityMismatch: Failure to conjugate test function in complex Form

[dokken]

Well, so this changes things.

The issue is with how one differentiates a complex valued problem.
I would try:
dJdm = dolfinx.fem.form(ufl.derivative(J, eps, ufl.conj(ufl.TestFunction(eps.function_space))
given that eps is a function in some functionspace.

[Pasuta]

It worked now, Thank you so much! It’s very useful.