# PointSource definition in parallel leads to deadlock

**URL:** <https://fenicsproject.discourse.group/t/pointsource-definition-in-parallel-leads-to-deadlock/4729>\
**Category:** Uncategorized\
**Created:** [December 14, 2020, 11:24am UTC](https://fenicsproject.discourse.group/t/pointsource-definition-in-parallel-leads-to-deadlock/4729 "2020-12-14T11:24:18Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![IshaanDesai](https://yyz2.discourse-cdn.com/free1/user_avatar/fenicsproject.discourse.group/ishaandesai/32/1369_2.png) [@IshaanDesai](https://fenicsproject.discourse.group/u/IshaanDesai)\
**Post date:** [December 14, 2020, 11:24am UTC](https://fenicsproject.discourse.group/t/pointsource-definition-in-parallel-leads-to-deadlock/4729/1 "2020-12-14T11:24:18Z")

</div>

Hello,  
I am using `fenics-2019.1.0` on Ubuntu 18.04.  
I have a vertical 2D beam on which I want to define PointSource loads at specific points on the mesh.  
 ![Screenshot from 2020-12-14 12-16-07](https://global.discourse-cdn.com/free1/uploads/fenicsproject1/original/2X/6/6c993771699eca5104f31b7d3ff44b226eae0e68.png)  
When the code is run in parallel (for example 2 processors), FEniCS decomposes the domain in such a way:  
 ![Screenshot from 2020-12-14 12-16-20](https://global.discourse-cdn.com/free1/uploads/fenicsproject1/original/2X/c/c310b6a9747118efb03c2463ddd33ff7c0a8a804.png)

I define PointSource’s on select points of the mesh using this minimum working example code:

```auto
from fenics import Point, PointSource, SubDomain, RectangleMesh, vertices, VectorFunctionSpace
import numpy as np
from mpi4py import MPI

H = 1
W = 0.1
tol = 1E-14

class neumannBoundary(SubDomain):
    def inside(self, x, on_boundary):
        tol = 1E-14
        if on_boundary and ((abs(x[1] - 1) < tol) or abs(abs(x[0]) - W / 2) < tol):
            return True
        else:
            return False

comm = MPI.COMM_WORLD
rank = comm.Get_rank()

# create Mesh
n_x_Direction = 1
n_y_Direction = 10
mesh = RectangleMesh(Point(-W / 2, 0), Point(W / 2, H), n_x_Direction, n_y_Direction)
boundary = neumannBoundary()

fenics_coords = []
for v in vertices(mesh):
    if boundary.inside(v.point(), True):
        fenics_coords.append([v.x(0), v.x(1)])

fenics_coords = np.array(fenics_coords)
n_vertices, _ = fenics_coords.shape

# create Function Space
V = VectorFunctionSpace(mesh, 'P', 2)

# Dummy nodal data to assign as PointSource value
nodal_data = np.zeros_like(fenics_coords)

# Define PointSource for all points on coupling boundary
vertices_x = fenics_coords[:, 0]
vertices_y = fenics_coords[:, 1]

x_forces = dict()
y_forces = dict()

for i in range(n_vertices):
    px, py = vertices_x[i], vertices_y[i]
    key = (px, py)
    x_forces[key] = PointSource(V.sub(0), Point(px, py), nodal_data[i, 0])
    y_forces[key] = PointSource(V.sub(1), Point(px, py), nodal_data[i, 1])

print("Rank {}: Definition of PointSource's is successful".format(rank))

```

The code runs fine in serial by: `python3 <filename>` but when run in parallel: `mpirun -np 2 python3 <filename>` the code hangs. I observed that if I define the mesh with only one element in the X-direction then the code runs fine. As soon as I have multiple elements in the X-direction and I try to define a PointSource on them, the code hangs.  
What could be the possible problem?

Thanks for your help!

---

<div class="post-metadata">

**Author:** ![nate](https://yyz2.discourse-cdn.com/free1/user_avatar/fenicsproject.discourse.group/nate/32/17_2.png) [@nate](https://fenicsproject.discourse.group/u/nate)\
**Post date:** [December 17, 2020, 6:49pm UTC](https://fenicsproject.discourse.group/t/pointsource-definition-in-parallel-leads-to-deadlock/4729/2 "2020-12-17T18:49:39Z")

</div>

Your example is a little too convoluted for me to examine at a glance. But I’m assuming you’re applying `PointSource`s on process boundaries. This has been buggy in the past with old DOLFIN and some developments were made in 2016/2017 to fix these issues. Sadly there’s little motivation to go back and fix this in old DOLFIN as people have moved on to DOLFIN-x. Your best bet is to read the `PointSource` [source](https://bitbucket.org/fenics-project/dolfin/src/master/dolfin/fem/PointSource.cpp) and make sure you’re generating them correctly in parallel.

---

<div class="post-metadata">

**Author:** ![IshaanDesai](https://yyz2.discourse-cdn.com/free1/user_avatar/fenicsproject.discourse.group/ishaandesai/32/1369_2.png) [@IshaanDesai](https://fenicsproject.discourse.group/u/IshaanDesai)\
**Post date:** [December 18, 2020, 12:16pm UTC](https://fenicsproject.discourse.group/t/pointsource-definition-in-parallel-leads-to-deadlock/4729/3 "2020-12-18T12:16:57Z")

</div>

Hi @nate! You assumed correctly that I am applying `PointSources` on process boundaries. I have a vertical beam as shown in the images above and I am applying `PointSources` on the outer edge of this beam.  
I will go through the `PointSource` source file and try to get it working. If that fails then the solution would be to move to DOLFIN-X?

---

<div class="post-metadata">

**Author:** ![fpradelli](https://yyz2.discourse-cdn.com/free1/user_avatar/fenicsproject.discourse.group/fpradelli/32/3291_2.png) [@fpradelli](https://fenicsproject.discourse.group/u/fpradelli)\
**Post date:** [May 25, 2021, 12:29pm UTC](https://fenicsproject.discourse.group/t/pointsource-definition-in-parallel-leads-to-deadlock/4729/4 "2021-05-25T12:29:10Z")

</div>

Hi Ishaasn,  
Did you eventually manage to solve the issue? I’m stuck with the same problem.  
Thank you!

---

<div class="post-metadata">

**Author:** ![IshaanDesai](https://yyz2.discourse-cdn.com/free1/user_avatar/fenicsproject.discourse.group/ishaandesai/32/1369_2.png) [@IshaanDesai](https://fenicsproject.discourse.group/u/IshaanDesai)\
**Post date:** [May 25, 2021, 1:18pm UTC](https://fenicsproject.discourse.group/t/pointsource-definition-in-parallel-leads-to-deadlock/4729/5 "2021-05-25T13:18:24Z")

</div>

Hi @fpradelli  
unfortunately I did not manage to resolve this issue. I guess the solution would be to port the code to DOLFIN-X. Sorry I cant help you more here…