Strange error in parallel mode but fine in serial mode

Hello,

I wrote a function with fenics, I don’t think it is necessary to post the full code for this issue, let’s call it model(parameter) in the script “stress.py”, such that it takes in a parameter and solves for a stress field and evaluates the scalar value at the midpoint. I wrapped it in another python script to loop it through different input parameters:

As a serial example:

from stress import model
from numpy import linspace, savetxt
parameters = linspace(0,1,100)

for param in parameters:
    output = model(param)
    savetxt(str(param)+'.txt',output,delimiter=',')

This worked perfectly fine, except I wanted to run it with MPI to speed up the tasks since each evaluation is independent of the others, my parallel example:

from mpi4py import MPI
comm = MPI.COMM_WORLD
rank = comm.Get_rank()
size = comm.Get_size()

from stress import model
from numpy import linspace, savetxt
parameters = linspace(0,1,100)

for i in range(len(parameters)/size):
    output = model( parameter(i+rank) )
    savetxt(str(parameter(i+rank))+'.txt',output,delimiter=',')

The above script was executed on the terminal with mpirun -n 5 python3 wrapper script.py

However, this parallel attempt gave me this following error on all processes:

RuntimeError: 

*** -------------------------------------------------------------------------
*** DOLFIN encountered an error. If you are not able to resolve this issue
*** using the information listed below, you can ask for help at
***
***     fenics-support@googlegroups.com
***
*** Remember to include the error message listed below and, if possible,
*** include a *minimal* running example to reproduce the error.
***
*** -------------------------------------------------------------------------
*** Error:   Unable to evaluate function at point.
*** Reason:  The point is not inside the domain. Consider calling "Function::set_allow_extrapolation(true)" on this Function to allow extrapolation.
*** Where:   This error was encountered inside Function.cpp.
*** Process: 4
*** 
*** DOLFIN version: 2019.1.0
*** Git changeset:  15b823f2c45d3036b6af931284d0f8e3c77b6845
*** -------------------------------------------------------------------------

Basically I think it is telling me that it couldn’t evaluate the stress at the midpoint because it is not in the domain? But it was totally fine in serial mode? Does anyone have a clue on what’s wrong here? Thank you!

See this thread: Problem with evaluation at a point in parallel

1 Like