# Assign values at DOF in parallel using MPI

**URL:** <https://fenicsproject.discourse.group/t/assign-values-at-dof-in-parallel-using-mpi/13672>\
**Category:** General\
**Tags:** dolfinx, mpi\
**Created:** [February 21, 2024, 9:32am UTC](https://fenicsproject.discourse.group/t/assign-values-at-dof-in-parallel-using-mpi/13672 "2024-02-21T09:32:27Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![dongjun](https://avatars.discourse-cdn.com/v4/letter/d/e47c2d/32.png) [@dongjun](https://fenicsproject.discourse.group/u/dongjun)\
**Post date:** [February 21, 2024, 9:32am UTC](https://fenicsproject.discourse.group/t/assign-values-at-dof-in-parallel-using-mpi/13672/1 "2024-02-21T09:32:27Z")

</div>

Hi there,  
I’m trying to define a Function by setting the values at the DOFs. So to say, I want to define u on the unit cube as

u(t,x,y)= \begin{cases} 2.0 & \max\{|x-0.7|,|y-0.3|\}\<0.2 \\ 0 & \text{else} \end{cases}

I used the following code:

```auto
from mpi4py import MPI
import numpy as np
from dolfinx import mesh, io
from dolfinx.fem import FunctionSpace,Function
from petsc4py import PETSc

nt, nx, ny = 20, 40, 40
comm = MPI.COMM_WORLD
rank = comm.Get_rank()
msh = mesh.create_box(comm, points=np.array([[0,0,0], [1,1, 1]]), n = [nt, nx, ny], ghost_mode=mesh.GhostMode.shared_facet)

W = FunctionSpace(msh, ("Lagrange", 1))
dof_coord = W.tabulate_dof_coordinates()

u = Function(W)

for k in range(u.vector.array[:].size):
    if max(abs(dof_coord[k][1] - 0.7), abs(dof_coord[k][2] - 0.3)) < 0.2:
        u.vector.array[k] = 2.0

u.vector.ghostUpdate(addv=PETSc.InsertMode.INSERT, mode=PETSc.ScatterMode.FORWARD)
u.x.scatter_forward()

with io.XDMFFile(msh.comm, "u.xdmf", "w") as xdmf:
    xdmf.write_mesh(msh)
    xdmf.write_function(u)

```

But I get different results when using MPI. If I run `mpirun -n 1 python3 test.py`, it works perfect. Below the images of u(0.1,x,y) and u(0.5,x,y), which look perfect:  
[slice 1]  
 ![0.1](https://global.discourse-cdn.com/free1/uploads/fenicsproject1/original/2X/d/d0fac60857929ebcff6a515568a14c4b28f6d684.png)  
[slice 2]  
 ![0.6](https://global.discourse-cdn.com/free1/uploads/fenicsproject1/original/2X/8/888dbfb7d87a4de66ffa6d3e767c41f682160e3e.png)

But If I run `mpirun -n 4 python3 test.py`, the same slices look like follows:  
[slice 1]  
 ![mpi_0.1](https://global.discourse-cdn.com/free1/uploads/fenicsproject1/original/2X/a/afdeb5d147956e1611c2876f92e81db3b66a8903.png)  
[slice 2]  
 ![mpi_0.6](https://global.discourse-cdn.com/free1/uploads/fenicsproject1/original/2X/c/c338d9c1480d70820641b479ea169eaf9ef3284b.png)

I guess the problem is related to ghost points, but don’t know how to correct this issue. I can see that the lengths of dof\_coord and that of u.vector.array[:] are different when running in parallel, while they are the same when specifying -n 1. So, if I’m to define a Function as above by specifying values at the DOFs, how should I do when running in parallel?

---

<div class="post-metadata">

**Author:** ![dokken](https://yyz2.discourse-cdn.com/free1/user_avatar/fenicsproject.discourse.group/dokken/32/1560_2.png) [@dokken](https://fenicsproject.discourse.group/u/dokken)\
**Post date:** [February 21, 2024, 11:46am UTC](https://fenicsproject.discourse.group/t/assign-values-at-dof-in-parallel-using-mpi/13672/2 "2024-02-21T11:46:39Z")

</div>

> [@dongjun](#):
>
> `u.vector.array[:].size`

Use `u.x.array` instead and add a tolerance for floating point issues:

```python
from mpi4py import MPI
import numpy as np
from dolfinx import mesh, io
from dolfinx.fem import FunctionSpace,Function
from petsc4py import PETSc

nt, nx, ny = 20, 40, 40
comm = MPI.COMM_WORLD
rank = comm.Get_rank()
msh = mesh.create_box(comm, points=np.array([[0,0,0], [1,1, 1]]), n = [nt, nx, ny], ghost_mode=mesh.GhostMode.shared_facet)

W = FunctionSpace(msh, ("Lagrange", 1))
dof_coord = W.tabulate_dof_coordinates()

u = Function(W)

for k in range(u.x.array[:].size):
    if max(abs(dof_coord[k][1] - 0.7), abs(dof_coord[k][2] - 0.3)) < 0.2+ 1e-14:
        u.x.array[k] = 2.0
u.x.scatter_forward()

with io.XDMFFile(msh.comm, "u.xdmf", "w") as xdmf:
    xdmf.write_mesh(msh)
    xdmf.write_function(u)

```

---

<div class="post-metadata">

**Author:** ![dongjun](https://avatars.discourse-cdn.com/v4/letter/d/e47c2d/32.png) [@dongjun](https://fenicsproject.discourse.group/u/dongjun)\
**Post date:** [February 21, 2024, 2:26pm UTC](https://fenicsproject.discourse.group/t/assign-values-at-dof-in-parallel-using-mpi/13672/3 "2024-02-21T14:26:14Z")

</div>

Problem solved! Great thanks!