Assemble global coordinates

dolfinx
1

After meshing in gmsh, I do the following to access the nodal coordinates

from dolfinx import mesh, fem, plot, io, default_scalar_type

domain, markers, facets = io.gmshio.model_to_mesh(gmsh.model, MPI.COMM_WORLD, 0, gdim=3)

print(domain.geometry.x)

Serially, len(domain.geometry.x) gives me 27840.
But when i run it in parallel, sum of len(domain.geometry.x) from all the processes gives me 28536

I need the mesh coordinates to plot graphs.

2

When the mesh is distributed over multiple processes, some points are present on multiple processes (along inter-process boundaries).
Use mesh.geometry.index_map.size_local to get the local number (the mesh nodes owned by the current process).

Secondly, you can use mesh.geometry.input_global_indices to map the local node i (mesh.geometry.x[i,:]) to the global input node in the GMSH model

3

error

Secondly,
correct me if i am wrong,
domain.geometry.input_global_indices will give me a num_local sized array with global indices
and then to get the coordinates I need to read the mesh again on a single process and use domain.geometry.x[i,:] on global domain ???

4

domain.geometry.index_map().size_local will give you the number of owned points on the current process.
i.e.

x = mesh.geometry.x
num_owned_nodes = mesh.geometry.index_map().size_local
x_local = x[:num_owned_nodes, :]

Then to get the global indices of these coordinates you call

global_input_index = mesh.geometry.input_global_indices()[:num_owned_nodes]

tells you where these nodes should be put in a global array.

5

I tried,

g_coord = domain.comm.gather(domain.geometry.x[:domain.geometry.index_map().size_local,1])
g_indices = domain.comm.gather(domain.geometry.input_global_indices[:domain.geometry.index_map().size_local])

if rank == 0:
    gl_coor = np.full(domain.geometry.index_map().size_global,None)

    for j in range(len(g_coord)):
        gl_coor[g_indices[j]] = g_coord[j]

I got this,

Screenshot 2024-08-26 185551

Next, I tried

g_coord = domain.comm.gather(domain.geometry.x[:domain.geometry.index_map().size_local,1])
g_indices = domain.comm.gather(domain.geometry.input_global_indices[:domain.geometry.index_map().size_local])

if rank == 0:
     gl_coor = np.concatenate(g_coord)

I got this,
Screenshot 2024-08-26 185505

This was the output from serial code which I was trying to recreate in parallel,
Screenshot 2024-08-26 190119

The second approach seems closer to expected output.

6

As you have not provided a reproducible example, I have no idea how you are gathering the deformation across multiple processes, and therefore it is very hard to give you any guidance.
Please consider making a reproducible example.