# Saving and loading dolfinx function in parallel **URL:** \ **Category:** I/O\ **Created:** [September 29, 2025, 8:10am UTC](https://fenicsproject.discourse.group/t/saving-and-loading-dolfinx-function-in-parallel/18235 "2025-09-29T08:10:05Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![MaartenV](https://avatars.discourse-cdn.com/v4/letter/m/a4c791/32.png) [@MaartenV](https://fenicsproject.discourse.group/u/MaartenV)\ **Post date:** [September 29, 2025, 8:10am UTC](https://fenicsproject.discourse.group/t/saving-and-loading-dolfinx-function-in-parallel/18235/1 "2025-09-29T08:10:05Z")
Hi, I am currently trying to find a setup of the following nature. I run a large number of simulations of a time-dependent PDE. The solution at the end of simulation i is supposed to be the initial condition for simulation i+1. In order to do that, I need a way of writing the solution to a file and reading it back in again afterwards. All simulations are run in parallel with mesh partitioning. Unfortunately, the IT-staff of the supercomputer that I am using did not manage to install adios. Is there any way of doing this differently than with adios? I feel like it could be possible with XDMF, but this does not seem to be recommended. Alternatively, I could just write the end solution to a numpy array using np.save, but then I have problems due to inconsistent mesh partitioning. Any pointers are welcomed. Thanks in advance. Kind regards, Maarten ---
**Author:** ![dokken](https://yyz2.discourse-cdn.com/free1/user_avatar/fenicsproject.discourse.group/dokken/32/1560_2.png) [@dokken](https://fenicsproject.discourse.group/u/dokken)\ **Post date:** [September 29, 2025, 10:53am UTC](https://fenicsproject.discourse.group/t/saving-and-loading-dolfinx-function-in-parallel/18235/2 "2025-09-29T10:53:04Z")
ADIOS2 shouldn’t be hard for IT staff at a super-computer facility to install. Could you be more specific with what particular issues they had? It is for instance on spack ([Spack Packages](https://packages.spack.io/package.html?name=adios2)) and similarly adios4dolfinx: [Spack Packages](https://packages.spack.io/package.html?name=py-adios4dolfinx) > [@MaartenV](#): > > feel like it could be possible with XDMF, but this does not seem to be recommended. XDMF only stores data at mesh nodes, limiting what function spaces one can read and write data with. > [@MaartenV](#): > > Alternatively, I could just write the end solution to a numpy array using np.save, but then I have problems due to inconsistent mesh partitioning. I don’t know what you imply by “inconsistent” mesh partitioning. There are several posts on the forum on how to gather data to a single process, see for instance: > [@How to collect the global mesh without writing a file](https://fenicsproject.discourse.group/t/how-to-collect-the-global-mesh-without-writing-a-file/12445/4): > > Here is an example code of how to do it: # Copyright (C) 2023 Jørgen S. Dokken # # SPDX-License-Identifier: MIT # # Code to plot mesh with pyvista in parallel by gather data on a root process # # Last changed: 2023-10-23 import dolfinx from mpi4py import MPI import pyvista import numpy as np pyvista.start\_xvfb(0.2) mesh = dolfinx.mesh.create\_unit\_square(MPI.COMM\_WORLD, 10, 10) V = dolfinx.fem.functionspace(mesh, ("Lagrange", 2)) u = dolfinx.fem.Function(V) u.interpolate(lambda x: x[0] + 2\*… if you want to store everything as pure arrays (you have to take into account what function space you are using to consider this as a viable approach or not). ---
**Author:** ![MaartenV](https://avatars.discourse-cdn.com/v4/letter/m/a4c791/32.png) [@MaartenV](https://fenicsproject.discourse.group/u/MaartenV)\ **Post date:** [September 29, 2025, 12:53pm UTC](https://fenicsproject.discourse.group/t/saving-and-loading-dolfinx-function-in-parallel/18235/3 "2025-09-29T12:53:24Z")
Hi Jørgen, Thank you very much for your reply. > [@dokken](#): > > XDMF only stores data at mesh nodes, limiting what function spaces one can read and write data with. I am only using a 1st order lagrange functions, so I believe xdmf should work then? If so, I guess that I am only missing a function to read the function in at the beginning of the next experiment, something in the shape of dolfinx.io.XDMFFile.read\_function(). Or should I use the read\_mesh() function for that purpose? > [@dokken](#): > > I don’t know what you imply by “inconsistent” mesh partitioning. There are several posts on the forum on how to gather data to a single process, see for instance: FYI 1: With inconsistent, I meant that the same mesh was partitioned differently at different instances. This means I could not create a numpy array per process, but this issue should be resolved by gathering all data on a single process. > [@dokken](#): > > ADIOS2 shouldn’t be hard for IT staff at a super-computer facility to install. > Could you be more specific with what particular issues they had? FYI 2: for the issues regarding ADIOS2, I was told that some tests fail, particularly some of the MPI write tests (see easybuilders/easybuild-easyconfigs github repo issue # 23215) Thank you again, Maarten ---
**Author:** ![dokken](https://yyz2.discourse-cdn.com/free1/user_avatar/fenicsproject.discourse.group/dokken/32/1560_2.png) [@dokken](https://fenicsproject.discourse.group/u/dokken)\ **Post date:** [September 29, 2025, 3:07pm UTC](https://fenicsproject.discourse.group/t/saving-and-loading-dolfinx-function-in-parallel/18235/4 "2025-09-29T15:07:46Z")
> [@MaartenV](#): > > FYI 2: for the issues regarding ADIOS2, I was told that some tests fail, particularly some of the MPI write tests (see easybuilders/easybuild-easyconfigs github repo issue # 23215) As I don’t use easybuild myself, I can’t really help much with solving that one: ([{tools}[gompi/2024a] ADIOS2 v2.10.2 by alainvanhoof · Pull Request #23215 · easybuilders/easybuild-easyconfigs · GitHub](https://github.com/easybuilders/easybuild-easyconfigs/pull/23215)) You could try running spack on the HPC system and see if that resolves it. If you want something like `dolfinx.io.XDMFFile.read_function`, you can consider the code I have in: > > > quick and "dirty" python version of reading data at geometry nodes. > Should be p…roperly implemented with ADIOS2. > \`\`\`python > from pathlib import Path > > import adios4dolfinx > import dolfinx > import numpy as np > from mpi4py import MPI > > with dolfinx.io.XDMFFile(MPI.COMM\_WORLD, "idealized\_LV\_ref\_0.xdmf", "r") as xdmf: > mesh = xdmf.read\_mesh(name="mesh", xpath="Xdmf/Domain/Grid") > > import basix > import h5py > > > def read\_mesh\_data(file\_path:Path, mesh: dolfinx.mesh.Mesh, data\_path:str): > assert file\_path.is\_file(), f"File {file\_path} does not exist" > infile = h5py.File(file\_path, "r", driver="mpio", comm=mesh.comm) > num\_nodes\_global = mesh.geometry.index\_map().size\_global > assert data\_path in infile.keys(), f"Data {data\_path} does not exist" > dataset = infile\[data\_path\] > shape = dataset.shape > assert shape\[0\] == num\_nodes\_global, f"Got data of shape {shape}, expected {num\_nodes\_global, shape\[1\]}" > dtype = dataset.dtype > # Read data locally on each process > local\_input\_range = adios4dolfinx.comm\_helpers.compute\_local\_range(mesh.comm, num\_nodes\_global) > local\_input\_data = dataset\[local\_input\_range\[0\]:local\_input\_range\[1\]\] > > # Create appropriate function space (based on coordinate map) > assert len(mesh.geometry.cmaps) == 1, "Mixed cell-type meshes not supported" > element = basix.ufl.element( > basix.ElementFamily.P, > mesh.topology.cell\_name(), > mesh.geometry.cmaps\[0\].degree, > mesh.geometry.cmaps\[0\].variant, > shape=(shape\[1\],), > gdim=mesh.geometry.dim) > > # Assumption: Same doflayout for geometry and function space, cannot test in python > V = dolfinx.fem.FunctionSpace(mesh, element) > uh = dolfinx.fem.Function(V, name=data\_path) > # Assume that mesh is first order for now > assert mesh.geometry.cmaps\[0\].degree == 1, "Only linear meshes supported" > x\_dofmap = mesh.geometry.dofmap > igi = np.array(mesh.geometry.input\_global\_indices, dtype=np.int64) > global\_geom\_input = igi\[x\_dofmap\] > global\_geom\_owner = adios4dolfinx.utils.index\_owner(mesh.comm, global\_geom\_input.reshape(-1), num\_nodes\_global) > for i in range(shape\[1\]): > arr\_i = adios4dolfinx.comm\_helpers.send\_dofs\_and\_recv\_values(global\_geom\_input.reshape(-1), global\_geom\_owner, mesh.comm, local\_input\_data\[:,i\], local\_input\_range\[0\]) > dof\_pos = x\_dofmap.reshape(-1)\*shape\[1\]+i > uh.x.array\[dof\_pos\] = arr\_i > infile.close() > return uh > infile = Path("idealized\_LV\_ref\_0.h5") > > > for data in \["f0", "n0", "phi", "psi", "s0"\]: > uh = read\_mesh\_data(infile, mesh, data) > with dolfinx.io.XDMFFile(mesh.comm, f"{data}.xdmf", "w") as xdmf: > xdmf.write\_mesh(mesh) > xdmf.write\_function(uh) > \`\`\` > > ref: > https://fenicsproject.discourse.group/t/i-o-from-xdmf-hdf5-files-in-dolfin-x/3122/47 which doesn’t actually depend on adios2 (it uses some library features from adios4dolfinx, but not adios as IO). I will hopefully make a h5py version of the library soon.