Strange JIT compilation error

installation
1

I’m compiling fenics 2019.2.0 on a cluster, and seeing some errors during compilation that I never saw before. When I run a script for the first time, and fenics has to compile all forms, it fails with this message:

Calling FFC just-in-time (JIT) compiler, this may take some time.
------------------- Start compiler output ------------------------
/usr/bin/ld: symbol lookup error: /usr/lib64/power9/libpthread.so.0: undefined symbol: pthread_mutex_lock
collect2: error: ld returned 127 exit status

-------------------  End compiler output  ------------------------

...

raise DijitsoError("Dijitso JIT compilation failed, see '%s' for details"
dijitso.jit.DijitsoError: Dijitso JIT compilation failed, see '~/jitfailure-ffc_element_5a01f62d0f2c0fe5f4320b0433b4b3fb72e2b207' for details

if i run readelf -a /usr/lib64/power9/libpthread.so.0 I get the following output:

   215: 000000000000c090  1120 FUNC    WEAK   DEFAULT   12 pthread_mutex_lock@@GLIBC_2.17 	[<localentry>: 8] 
   244: 000000000000c090  1120 FUNC    GLOBAL DEFAULT   12 __pthread_mutex_lock@@GLIBC_2.17 	[<localentry>: 8] 
   724: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS pthread_mutex_lock.c
   736: 000000000000b860  2092 FUNC    LOCAL  DEFAULT   12 __pthread_mutex_lock_full 	[<localentry>: 8] 
  3321: 000000000000c090  1120 FUNC    LOCAL  DEFAULT   12 __GI___pthread_mutex_lock 	[<localentry>: 8] 
  3576: 000000000000c090  1120 FUNC    WEAK   DEFAULT   12 pthread_mutex_lock [<localentry>: 8] 
  3696: 000000000000c090  1120 FUNC    GLOBAL DEFAULT   12 __pthread_mutex_lock 	[<localentry>: 8] 
    Applies to region from 0xb862 to 0xc4f0 (.annobin_pthread_mutex_lock.c)

Interestingly, if I run recompile.sh from the jitfailure folder, it just runs.

Anyway, looking for this error online, it seems that a solution should be to add the flag -pthread at the compilation stage.

However, this compilation step is hidden to me, since it’s done by ffc automatically if I’m not mistaken. It also seems that the -pthread flag should be added at the end of the compilation command, that is, the position where the flag is added counts.

So I guess ffc should run something like this example:

g++ -Wall -shared -fPIC -std=c++11 -O2 -I./lib/python3.8/site-packages/ffc/backends/ufc -I~/.cache/dijitso/include ffc_element_5a01f62d0f2c0fe5f4320b0433b4b3fb72e2b207.cpp 
 -L~/.cache/dijitso/lib -Wl,-rpath,~/.cache/dijitso/lib  
-olibdijitso-ffc_element_5a01f62d0f2c0fe5f4320b0433b4b3fb72e2b207.so  -pthread

where I abbreviated the paths. I also got this same error while building fenics itself from source, and solved it by adding -pthread to CXXFLAGS.

I know all this is very heuristic, but I’m not familiar with this compilation procedures and c++.

So, is there any way to modify ffc or dijitso in order to change the jit compilation phase, by adding a flag to the very end of the compilation input to the compiler (or do you know any other way to solve this)??