Saving and loading dolfinx function in parallel

I/O
1

Hi,

I am currently trying to find a setup of the following nature. I run a large number of simulations of a time-dependent PDE. The solution at the end of simulation i is supposed to be the initial condition for simulation i+1. In order to do that, I need a way of writing the solution to a file and reading it back in again afterwards. All simulations are run in parallel with mesh partitioning.

Unfortunately, the IT-staff of the supercomputer that I am using did not manage to install adios. Is there any way of doing this differently than with adios? I feel like it could be possible with XDMF, but this does not seem to be recommended. Alternatively, I could just write the end solution to a numpy array using np.save, but then I have problems due to inconsistent mesh partitioning.

Any pointers are welcomed. Thanks in advance.

Kind regards,
Maarten

2

ADIOS2 shouldn’t be hard for IT staff at a super-computer facility to install.
Could you be more specific with what particular issues they had?

It is for instance on spack (Spack Packages) and similarly adios4dolfinx: Spack Packages

XDMF only stores data at mesh nodes, limiting what function spaces one can read and write data with.

I don’t know what you imply by “inconsistent” mesh partitioning. There are several posts on the forum on how to gather data to a single process, see for instance:

if you want to store everything as pure arrays (you have to take into account what function space you are using to consider this as a viable approach or not).

3

Hi Jørgen,

Thank you very much for your reply.

I am only using a 1st order lagrange functions, so I believe xdmf should work then? If so, I guess that I am only missing a function to read the function in at the beginning of the next experiment, something in the shape of dolfinx.io.XDMFFile.read_function(). Or should I use the read_mesh() function for that purpose?

FYI 1: With inconsistent, I meant that the same mesh was partitioned differently at different instances. This means I could not create a numpy array per process, but this issue should be resolved by gathering all data on a single process.

FYI 2: for the issues regarding ADIOS2, I was told that some tests fail, particularly some of the MPI write tests (see easybuilders/easybuild-easyconfigs github repo issue # 23215)

Thank you again,
Maarten

4

As I don’t use easybuild myself, I can’t really help much with solving that one: ({tools}[gompi/2024a] ADIOS2 v2.10.2 by alainvanhoof · Pull Request #23215 · easybuilders/easybuild-easyconfigs · GitHub)

You could try running spack on the HPC system and see if that resolves it.

If you want something like dolfinx.io.XDMFFile.read_function, you can consider the code I have in:

which doesn’t actually depend on adios2 (it uses some library features from adios4dolfinx, but not adios as IO).
I will hopefully make a h5py version of the library soon.