Use of Submesh in parallel

1

Hi,

So I was planning to solve a problem which distributes a mesh and its submesh to different processes. I am aware that the FEniCS in-built SubMesh function is not yet compatible with mpi. However, I have been busy trying to find a way around it. I tried doing it myself and mapping dofs and vertices between processors and then gathering and broadcasting but I had no luck. The shared vertices and dofs make it very confusing. Then I came across this which was written in 2016. I tried to fix all the version errors and write the C++ codes in python, but this has way too many bugs and I feel like I am changing things that shouldn’t be changed. My questions are:
1- Is there a 2019 method to deal with this problem?
2- Is anyone working on this currently and when can we expect a working version?
3- Are there any other alternatives that you know of?

Best,

2

I would suggest using MeshView, see for instance: https://bitbucket.org/fenics-project/dolfin/src/master/python/demo/undocumented/meshview-3D2D/demo_meshview-3D2D.py

3

Thank you. This sure seems like it will solve my problem. But is this a part of the 2019 stable version of FEniCS? Because I have it and it says this function doesn’t exist.

1 Like
4

No, it is part of the master branch of fenics, Which can be installed with either docker (the dev/lastest image) or with apt-get as explained in this post No module named 'dolfin' ubuntu 20.04

5

Thank you again Dokken. We are trying to get this to work on a cluster through Singularity. Our cluster expert has tried building from multiple branches that include meshview, including the latest ‘master’ branch, and ‘cecile/mixed-dimensional’ , they fail at an identical place while compiling dolfin.

[ 81%] Building CXX object dolfin/CMakeFiles/dolfin.dir/fem/LocalAssembler.cpp.o
[ 83%] Building CXX object dolfin/CMakeFiles/dolfin.dir/fem/LocalSolver.cpp.o
[ 83%] Building CXX object dolfin/CMakeFiles/dolfin.dir/fem/MixedAssembler.cpp.o
/temporary/dolfin/dolfin/fem/MixedAssembler.cpp: In member function 'void dolfin::MixedAssembler::assemble_cells(dolfin::GenericTensor&, const dolfin::Form&, dolfin::UFC&, std::shared_ptr<const dolfin::MeshFu
nction<long unsigned int> >, std::vector<double>*)':
/temporary/dolfin/dolfin/fem/MixedAssembler.cpp:243:22: error: no matching function for call to 'ufc::cell_integral::tabulate_tensor(double*, double**, double*, int&, __gnu_cxx::__alloc_traits<std::allocator<
int> >::value_type&)'
       local_facets[j]); // local index of the lower dim entity involved
                      ^
In file included from /temporary/dolfin/dolfin/mesh/Cell.h:34:0,
                 from /temporary/dolfin/dolfin/fem/MixedAssembler.cpp:28:
/usr/include/ufc.h:607:18: note: candidate: virtual void ufc::cell_integral::tabulate_tensor(double*, const double* const*, const double*, int) const
     virtual void tabulate_tensor(double * A,
                  ^
/usr/include/ufc.h:607:18: note:   candidate expects 4 arguments, 5 provided
dolfin/CMakeFiles/dolfin.dir/build.make:4598: recipe for target 'dolfin/CMakeFiles/dolfin.dir/fem/MixedAssembler.cpp.o' failed
make[2]: *** [dolfin/CMakeFiles/dolfin.dir/fem/MixedAssembler.cpp.o] Error 1
CMakeFiles/Makefile2:267: recipe for target 'dolfin/CMakeFiles/dolfin.dir/all' failed
make[1]: *** [dolfin/CMakeFiles/dolfin.dir/all] Error 2
Makefile:138: recipe for target 'all' failed
make: *** [all] Error 2

compiling ‘2019.1.0.post0’ does not have the same issue, but it doesn’t appear to include ‘meshview’. Do you know why this happens?

1 Like
6

You sure you’ve got the right version of FFC (and implicity ufc)?

7

We have tried a few different iterations with the same results, but perhaps it’s not picking up the correct version.

pip3 install fenics">=2019.1.0,<2019.2.0"

As well as:

add-apt-repository ppa:fenics-packages/fenics && \

apt-get update && \

apt-get -y install fenics

pip3 install fenics

Do you think this is the case? How can we make it pick the correct version?

8

Why are you installing fenics with pip3?

I have made a script below that shows the minimal instructions you need (Using a clean ubuntu 20.04 container)

docker run -it  --rm  ubuntu:20.04
apt-get update
apt-get install  -y sudo
sudo apt-get install -y software-properties-common
sudo add-apt-repository ppa:fenics-packages/fenics
sudo apt-get install -y fenics
python3 -c "import dolfin; print(dolfin.MeshView)"
9

Hello,
I’ve been working with nami attempting to install fenics with multimesh on our hpc cluster. Thank you for the support to date.
Due to kernel compatibility issues, we’re running rhel6, the latest ubuntu we’re able to containerize is xenial (16.04). I tried the suggested steps and installed fenics from the ppa repository, but that does not seem to include meshview.

python3 -c “import dolfin; print(dolfin.MeshView)”
Traceback (most recent call last):
File “”, line 1, in
AttributeError: module ‘dolfin’ has no attribute ‘MeshView’

Any suggestions?

10

If you use singularity, can’t you use singularity and docker images?


with this image:
11

In general, yes, however the fenics docker image is based off of a kernel that is too new for our rhel6 compute nodes.

12

As rhel6 has reached end of life (https://access.redhat.com/discussions/4768501)
I would strongly suggest getting your compute nodes up to date. Edit there has been posted an extension of life cycle. But it is still quite outdated (latest minor release in 2018).

If you do not want to update the compute nodes, I think you would have to create a singularity/docker container from scratch.
You can find the dependencies here:


and where to pull fenics packages from in fenics-pull:
13

Regarding rhel6, fully agreed and a newer version is something we will be rolling out in the near future, the sticking point being all the previously compiled software that our user base depends on.
I did find the dependencies earlier and it’s what I based the singularity recipe off of, so we should have a good starting point.
The fenics-pull may be the key piece that was missing. Thank you for that, I’ll give it a try and hopefully that does the trick.

14

Which fenics branch should I pull to include meshview? I tried ‘2019.1.0.post0’ but it doesn’t seem to include it and ‘cecile/mixed-dimensional’ is still giving me compile errors.

15

You should use master

1 Like
16

Thank you! I believe we have it working.

1 Like
17

Thank you all for all your help.