PETSc matrix-matrix multiplication got signal number 11 error in parallel

dolfinx
1

I got signal number 11 SEGV: Segmentation Violation in doing PETSc matrix-matrix multiplication in parallel. It works fine in serial.

In my FSI problem, I defined two non-matching meshes meshf and meshs. Then I defined a weak form on the smaller mesh meshs and assembled it to As (PETSc.Mat). I introduced a matrix I as an interpolation matrix, which makes the following transformation with matrix-matrix multiplication:

\Delta A = I^T A_s I

It fails in parallel runs but passes in serial runs.

MWE:

# dolfinx.__version__ == "0.7.3"

# mpirun -n >1 this.py # error
# mpirun -n  1 this.py # OK

from mpi4py import MPI
from petsc4py import PETSc
from dolfinx import mesh, fem
from dolfinx.fem.petsc import assemble_matrix
import ufl

# Mesh
nf = 20
ns = 10
meshf = mesh.create_unit_cube(MPI.COMM_WORLD, nf, nf, nf)
meshs = mesh.create_unit_cube(MPI.COMM_WORLD, ns, ns, ns)
meshs.geometry.x[:, :] *= 0.50
meshs.geometry.x[:, :] += 0.25

# Functionspaces
Vf = fem.functionspace(meshf, ("Lagrange", 1))
Vs = fem.functionspace(meshs, ("Lagrange", 1))

# Trail and test functions
us = ufl.TrialFunction(Vs)
vs = ufl.TestFunction(Vs)
As = assemble_matrix(fem.form(ufl.inner(us, vs) * ufl.dx))

# `I` for interpolation matrix
local_rows     = Vs.dofmap.index_map.size_local
global_rows    = Vs.dofmap.index_map.size_global
local_columns  = Vf.dofmap.index_map.size_local
global_columns = Vf.dofmap.index_map.size_global
matrix_sizes = [[local_rows,    global_rows   ], 
                [local_columns, global_columns]]
I = PETSc.Mat().create(Vs.mesh.comm)
I.setSizes(matrix_sizes)
I.setUp()

# Set values examples
I[0, 0] = 1

I.assemble()

# The following is with error:
#  PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, 
# probably memory access out of range
delta_A = I.transposeMatMult(As).matMult(I)
2

To answer my own question:

The above return value PETSc.Mat As is not finalized as stated:

To make it completed, it should be followed by

As.assemble()

N.B. For a assembled vector (PETSc.Vec), its ghost info in parallell should be specified with

bs = assemble_vector(L)
b.ghostUpdate(...)

See