How to properly clear and control JIT cache in dolfinx

dolfinx
1

Can somebody explain how to properly clear the cache at ~/.cache/fenics when using dolfinx? I’m seeing some suggestions (here and here) to execute dijitso clean at the command line but I don’t have the dijitso command available within my conda environment–maybe this procedure has changed or I need to install additional utilities?

On a related note, I have a workflow where I’m running a FEniCSx problem repeatedly with an embarrassingly parallel workflow. The inputs (mesh, constants values) are the different each time, but the code/forms remain the same. That said, I noticed that my ~/.cache/fenics directory contains over 26,000 files (nearly 5 GB worth). Is that expected? I am fighting some issues with I/O overhead and it seems this could be contributing. Does this indicate that something isn’t working as intended with the cache?

2

dijitso is part of legacy FEniCS. The python layer of dolfinx does not use dijitso, instead it uses CFFI. And the default cache directory is the one you’ve referred to.

Regarding many files in your cache. Make sure you’re properly generating dolfinx.fem.form objects outside of for/while loops. See, e.g., Speeding up time-dependent diffusion equation - #6 by nate

3

Thanks for the response, @nate. Regarding taking the fem.form call “outside of a loop”, this is frankly not practical in the near-term within my workflow since the “loop” is controlled externally by different software. I see how this is not technically leveraging FEniCS optimally, but I expect I will be stuck with situation for a while. Is there a way to simply accept that I’ll need to recompile each time (it is reasonably fast) and avoid digging into the cache every time?

Helpful to know that dolfinx is using a different approach for the compilation, thanks for that as well. Do you know if there is a “proper” method to clear the cache within CFFI or if I can simply delete those files without issue? A quick look into CFFI documentation was not helpful…

4

I would like to see how you do this, as there are many ways of writing code with external input that would use a single cached form.

Just remove the files in that folder.

5

Thanks, I marked your response as the solution as clearing the cache was the first key here! To follow up on the related issues…

I would like to see how you do this, as there are many ways of writing code with external input that would use a single cached form.

Without getting into too much detail, the workflow is something like (from Matlab):

for ii = 1 : nIters
    
    % prep mesh .xdmf file, other inputs
    
    % execute fenics analysis for this iteration
    system('python3 run.py`)

    % read and process outputs

end

In this case, run.py is a structural analysis that reads the mesh/inputs, constructs the relevant equations, solves, then outputs results, similar to this.

Indeed, clearing the cache drastically improved my issues with I/O overhead. And it appears I do have a new entry in the cache every time I run run.py, which explains why the cache had become so bloated over time. Thus, I’d be very curious as to how this workflow could be retrofitted to either (1) use a single cached form for every iteration or (2) simply ignore the cache (accepting that we will need to compile each time) so as to eliminate issues with I/O and cache bloat (obviously not the elegant solution but possibly pragmatic).

6

If you provide your code it would be easier to point out what to do. It usually boils down to material parameters and discretization parameters (such as theta in time splitting schemes) not being wrapped as Constant.

7

Okay, thanks. I cannot share my current code in its entirety due to IP issues unfortunately, but I may be able to put together a representative example later.

It usually boils down to material parameters and discretization parameters (such as theta in time splitting schemes) not being wrapped as Constant.

This makes sense–I can review my code and investigate the issue with this in mind.