Hi, I want to run my code with mpi4py, but I’m having some troubles with vertex_to_dof_map(dolfin.VectorFunctionSpace). Here is part of my code:
from dolfin import *
from mpi4py import MPI as pyMPI
comm = pyMPI.COMM_WORLD
nx = 100
nz = 65
mesh = RectangleMesh(comm, Point(0.0, 0.0), Point(1.0, 0.65), nx-1, nz-1)
VF = VectorFunctionSpace(mesh, "Lagrange", 1)
vf_dim = VF.dim()
theta = TrialFunction(VF)
v2d = vertex_to_dof_map(VF)
theta_vec = theta.vector()[v2d]
...
When I run with:
mpiexec -n 2 python3 .py
VF.dim() returns 13000 points, witch is perfect and correct. But theta.vector()[:].shape has only 6502 positions. Where is the other half of theta.vector()[:]? Besides, vertex_to_dof_map(VF) also returns 6560 dof positions and i don’t know how to get the other dofs.
But when i run with:
mpiexec -n 1 python3 .py
Everything works fine and returns all the 13000 points.
I know this questions can be extremely basic, but I am a mathematician and I’m trying to use FEniCS to compute shape optimization algorithms. Anyway, if you can help me, (please) be patience and explain each step in details 
When you run your code in parallel, your mesh is partitioned and distributed over multiple processors. Consider the following illustration:
from dolfin import *
from mpi4py import MPI as pyMPI
comm = pyMPI.COMM_WORLD
nx = 100
nz = 65
mesh = RectangleMesh(comm, Point(0.0, 0.0), Point(1.0, 0.65), nx-1, nz-1)
VF = VectorFunctionSpace(mesh, "Lagrange", 1)
vf_dim = VF.dim()
theta = Function(VF)
v2d = vertex_to_dof_map(VF)
theta_vec = theta.vector()
print(comm.rank, theta_vec.local_size(), theta_vec.local_range())
This prints:
0 6502 (0, 6502)
1 6498 (6502, 13000)
Meaning that processor one holds 6502 dofs, which globally are the first 6502 dofs in the dof map. The second processor holds the remaining 6498 dofs.
However, in general, as you consider shape optimization problems, you should consider using dolfin-adjoint, which has examples (Stokes-problem and time dependent heat equation of automatically deriving discretely consistent shape derivatives. For even more details, see this arxiv preprint.
1 Like
Thank you sir for the precise answer! I’m reading carefully all literature you recommended. It is nice to know that you made a recent contribution on shape optimization. I’m PhD candidate in Applied Mathematics at University of Sao Paulo - Brazil and Shape Optimization is one of my area of interest which I’m in focus now. I sure that this article you recommended will help me a lot.