Use write_function of xdmf in timeloop in parallel

I/O
,
1

Version:
fenics-dolfinx 0.9

installed via spack

spack add fenics-dolfinx+petsc+adios2 py-fenics-dolfinx cflags="-O3" fflags="-O3"

I tried to use fenicsx in parallel and i am confused how to use the write_function of xdmf the right way on mutiple ranks.

While the write_function works in parallel outside of the time loop, inside the time loop i get only results from rank 0 written to the file (see pictures).

rank4
rank4755×629 9.66 KB

rank0
rank0783×670 18 KB

MWE:

from dolfinx import fem, mesh, log, io
from mpi4py import MPI
import ufl
import numpy as np
from basix.ufl import element, mixed_element
from petsc4py import PETSc


domain = mesh.create_unit_square(MPI.COMM_WORLD, nx=4, ny=4, cell_type=mesh.CellType.triangle, ghost_mode=mesh.GhostMode.none)

#FunctionSpace Output
DG0 = fem.functionspace(domain, ("DG", 0))

rank_mpi = fem.Function(DG0)
F = fem.Function(DG0)
rank_mpi.x.array[:] = MPI.COMM_WORLD.Get_rank()
F.x.array[:] = 1.0


t = 0

with io.XDMFFile(MPI.COMM_WORLD, "mpi_test.xdmf", "w") as xdmf:
     xdmf.write_mesh(domain)
     F.name = "test"
     rank_mpi.name = "rank"
     xdmf.write_function(rank_mpi)
     
for n in range(10):
     
     t += 0.1
     xdmf.write_function(F, t)
     xdmf.write_function(rank_mpi, t)
     print("rank", rank_mpi.x.array[:])

Am I missing something fundamental here?

best regards

2

I think you close the file before the loop (with statement in Python | GeeksforGeeks)

Consider:

from dolfinx import fem, mesh, log, io
from mpi4py import MPI
import ufl
import numpy as np
from basix.ufl import element, mixed_element
from petsc4py import PETSc


domain = mesh.create_unit_square(
    MPI.COMM_WORLD,
    nx=4,
    ny=4,
    cell_type=mesh.CellType.triangle,
    ghost_mode=mesh.GhostMode.none,
)

# FunctionSpace Output
DG0 = fem.functionspace(domain, ("DG", 0))

rank_mpi = fem.Function(DG0)
F = fem.Function(DG0)
rank_mpi.x.array[:] = MPI.COMM_WORLD.Get_rank()
F.x.array[:] = 1.0


t = 0

xdmf = io.XDMFFile(MPI.COMM_WORLD, "mpi_test.xdmf", "w")
xdmf.write_mesh(domain)
F.name = "test"
rank_mpi.name = "rank"
xdmf.write_function(rank_mpi, t)

for n in range(10):

    t += 0.1
    xdmf.write_function(F, t)
    xdmf.write_function(rank_mpi, t)
    print("rank", rank_mpi.x.array[:])

xdmf.close()
3 Likes
3

@evdkmv thanks a lot for the catch, this solved the problem:D

1 Like