Parallel function saving and reading it back using PETSc or adios4dolfinx

[dokken]

@francesco-ballarin

If I look at adios_write.py there is instead a call to

adios4dolfinx.write_mesh(mesh, path_out, engine="BP4")
adios4dolfinx.write_function(kappa, path_out, engine="BP4")

i.e., writing again the same mesh, and in addition also writing the function. Does that mean that write_function has side effects that change the mesh that was written in the previous call?

The issue is that the mesh has been re-ordered, as it has been repartitioned when executed on more than 1 process.

A long term goal would maybe be to write the data out in its original order (however, I am not 100 % certain how easy that will be, due to dof transformations/cell permutations). Another note is that this would increase the runtime of write_function, as data is no longer written in a contiguous fashion.