Dijitso: libmpicxx.so.12: cannot open shared object file: No such file or directory

installation
1

Background

I am trying to build a docker image that contains JupyterHub and a FEniCS installation, so that students can work with some FEniCS demos without going through the installation procedure. Within this container, FEniCS (2019.1) is installed via conda.

Behaviour

After installing fenics with conda, I run the following MWE:

from fenics import *
u_D = Expression('1 + x[0]*x[0] + 2*x[1]*x[1]', degree=2)

This results in the following error message:

dijitso failed to load existing file:
	/home/admin/.cache/dijitso/lib/libdijitso-dolfin_expression_c49ebd8fd6c25a3c31b612f2a1752a2c.so
error is:
	libmpicxx.so.12: cannot open shared object file: No such file or directory
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
/opt/conda/lib/python3.7/site-packages/dolfin/jit/jit.py in compile_class(cpp_data, mpi_comm)
    167                                         mpi_comm=mpi_comm)
--> 168         submodule = dijitso.extract_factory_function(module, "create_" + module_name)()
    169     except Exception:

/opt/conda/lib/python3.7/site-packages/dijitso/jit.py in extract_factory_function(lib, name)
     46     """
---> 47     function = getattr(lib, name)
     48     function.restype = ctypes.c_void_p

AttributeError: 'NoneType' object has no attribute 'create_dolfin_expression_c49ebd8fd6c25a3c31b612f2a1752a2c'

During handling of the above exception, another exception occurred:

RuntimeError                              Traceback (most recent call last)
<ipython-input-1-28e2f3e6f919> in <module>
      1 from fenics import *
----> 2 u_D = Expression('1 + x[0]*x[0] + 2*x[1]*x[1]', degree=2)
      3 
      4 # Define boundary condition

/opt/conda/lib/python3.7/site-packages/dolfin/function/expression.py in __init__(self, cpp_code, *args, **kwargs)
    398                     raise KeyError("User parameter key must be a string")
    399 
--> 400             self._cpp_object = jit.compile_expression(cpp_code, params)
    401             self._parameters = ExpressionParameters(self._cpp_object, params)
    402 

/opt/conda/lib/python3.7/site-packages/dolfin/function/jit.py in compile_expression(statements, properties)
    156                 'name': 'expression', 'jit_generate': jit_generate}
    157 
--> 158     expression = compile_class(cpp_data, mpi_comm=mpi_comm)
    159     return expression

/opt/conda/lib/python3.7/site-packages/dolfin/jit/jit.py in compile_class(cpp_data, mpi_comm)
    168         submodule = dijitso.extract_factory_function(module, "create_" + module_name)()
    169     except Exception:
--> 170         raise RuntimeError("Unable to compile C++ code with dijitso")
    171 
    172     if name == 'expression':

RuntimeError: Unable to compile C++ code with dijitso

Observations:

libmpicxx.so.12 is present on my system. In fact, looking closer at the dijitso log-file, I can see the command it should execute:

c++ -Wall -shared -fPIC -std=c++11 -O3 -fno-math-errno -fno-trapping-math -ffinite-math-only \
-I/opt/conda/include -I/opt/conda/include/eigen3 -I/home/admin/.cache/dijitso/include dolfin_expression_c49ebd8fd6c25a3c31b612f2a1752a2c.cpp \
-L/opt/conda/lib -L/opt/conda/opt/conda/lib -L/home/admin/.cache/dijitso/lib \
-Wl,-rpath,/home/admin/.cache/dijitso/lib -lmpi -lmpicxx -lpetsc -lslepc -lz -lhdf5 -lboost_timer \
-ldolfin -olibdijitso-dolfin_expression_c49ebd8fd6c25a3c31b612f2a1752a2c.so

which contains -L/opt/conda/lib, and a find command reveals /opt/conda/lib/libmpicxx.so.12

Things I have tried:

  • Copy /opt/conda/lib/libmpicxx.so.12 into .cache/lib: dijitso successfully compiles the code and FEniCS successfully runs the complete example script, but this is not a sustainable solution.
  • Manually run the compilation command the dijitso module displayed in the log-file: the compilation succeeds, but the resulting library file is not picked up by dijitso.

Other information:

  • FEniCS version: 2019.1.0
  • OS: Debian GNU/Linux 10

Has anyone encountered something like this before? Any help is much appreciated: even if I can circumvent this by using a different setup.

2

Hi,
I generally use a singularity container to run FEniCS. It supports building containers from existing Docker images and you can pretty much seamlessly deploy these on HPC clusters as well.

I recently built a container with matplotlib and other necessary dependenies (for instance NetGen/NGSolve to generate simplical meshes) installed. Though JupyterHub is not installed you can access the recipe file here or I can build another image with JupyterHub installed on it.

You can install Singularity and shell-in the image using singularity shell. After that you should be good to go. If you need to test the above code you can pull the image using:

 singularity pull --name fenics2019.simg shub://bhaveshshrimali/singularitFEniCS:recipe

and then shell in using

singularity shell fenics2019.simg

and run

python3 -c "from dolfin import *"

to test if it is able to import the libraries correctly.

1 Like
3

Hi bhaveshshrimali,

Thanks, that looks useful. I’ve never used Singularity before but it seems promising.

4

I found out how to fix this particular case: the environment variable LD_LIBRARY_PATH was not set.
To be honest, it seemed redundant to me, looking at the library paths in the compilation commands, but maybe I am missing something.

For future people encountering problems with the same setup, I resolved it by adding the following variables to kernel.json (location: /opt/conda/share/jupyter/kernels/python3/kernel.json).

"env":{
 "LD_LIBRARY_PATH": "/opt/conda/lib:"
 }

Afterwards, the libraries were found as expected.