Superlu_dist.cc compilation error when using OpenMP

installation
1

hello,
I got a compilation error during the treatment of cpp/dolfinx/la/superlu_dist.cpp while I was compiling dolfinx@0.11. In my context, I use OpenMP. The extern "C" block is used in the code to include superlu_ddefs.h, superlu_sdefs.h and superlu_zdefs.h. All those headers include superlu_defs.h, which includes omp.h. The problem is that omp.h contains C++ stuff and thus can’t be inside an extern "C" block.
The following patch is a simple workaround:

 diff --git a/cpp/dolfinx/la/superlu_dist.cpp b/cpp/dolfinx/la/superlu_dist.cpp 
index 5c24eff434..b1eafaf9fa 100644 
--- a/cpp/dolfinx/la/superlu_dist.cpp 
+++ b/cpp/dolfinx/la/superlu_dist.cpp 
@@ -8,6 +8,7 @@ 
 #ifdef HAS_SUPERLU_DIST 
  
 #include "superlu_dist.h" 
+#include <superlu_defs.h> 
 extern "C" 
 { 
 #include <superlu_ddefs.h>

This compiles, but I have not yet tested the runtime behaviour, as I am unsure whether to keep this OpenMP context. I don’t know if it is the cleanest way to treat this OpenMP context failure but if it helps …

2

Thanks for the report and the patch. I can’t reproduce this: with GCC 13/14 and LLVM OpenMP 22, omp.h compiles fine inside extern "C". Could you share:

  1. The first compiler error, copied in full (the first ~20 lines of the failing superlu_dist.cpp compile, starting from the first error:).
  2. Your compiler and version, the output of $CXX --version (or mpicxx --version).
  3. Your OpenMP implementation (GCC’s libgomp, LLVM libomp, Intel, …) and how you turn OpenMP on, e.g. -fopenmp in CMAKE_CXX_FLAGS, or a Spack variant.
  4. Your SuperLU_DIST version, how it was installed (Spack, PETSc --download-superlu_dist, conda, from source), and whether it was built with OpenMP or with GPU support (CUDA/HIP).
  5. Your full CMake configure command for DOLFINx.
3

Hello Dokken,
my context is:

  1. spack install on linux-sapphirerapids
  2. gcc@15.3.0 (external for spack i.e. from the Red Hat Enterprise Linux release 8.10 (Ootpa))
  3. OpenMP implementation : GCC’s libgomp from gcc-runtime@15.3.0
  4. superlu-dist@9.2.1+openmp
  5. The error msg without the extra include (compile cmd appears in first line):
....
  cd /tmp/../spack-stage/spack-stage-fenics-dolfinx-0.11.0.post0-tkssfclsdqvothcsrpmqsaj4jomfi3gs/spack-build-tkssfcl/dolfinx && /.../spack_store/linux-sapphirerapids/compiler-wrapper-1.1.0-ytyo7cmimzhghtafrf6nzqgbsqond4rp/libexec/spack/gcc/g++ -DADIOS2_USE_MPI -DCOMMON_RANDOM_FIXED_SEED -DDOLFINX_GIT_COMMIT_HASH=\"\" -DDOLFINX_VERSION=\"0.11.0\" -DHAS_ADIOS2 -DHAS_KAHIP -DHAS_PARMETIS -DHAS_PETSC -DHAS_PTSCOTCH -DHAS_SLEPC -DHAS_SUPERLU_DIST -DMDSPAN_USE_BRACKET_OPERATOR=0 -DMDSPAN_USE_PAREN_OPERATOR=1 -DSPDLOG_COMPILED_LIB -DSPDLOG_FMT_EXTERNAL -DSPDLOG_SHARED_LIB -DUFCX_SIGNATURE=\"b0ecf558a831e95a689410a968d4dcb1188416f7\" -Ddolfinx_EXPORTS .... -isystem  /...../spack_store/linux-sapphirerapids/superlu-dist-9.2.1-qwce4vb3xuhwgu4xjfdsfoxfn7utikcu/include -O3 -DNDEBUG -std=c++20 -fPIC -pthread -fopenmp -MD -MT dolfinx/CMakeFiles/dolfinx.dir/la/superlu_dist.cpp.o -MF CMakeFiles/dolfinx.dir/la/superlu_dist.cpp.o.d -o CMakeFiles/dolfinx.dir/la/superlu_dist.cpp.o -c /...../patch_fenics0.11/cpp/dolfinx/la/superlu_dist.cpp
  In file included from /.../spack_store/linux-sapphirerapids/superlu-dist-9.2.1-qwce4vb3xuhwgu4xjfdsfoxfn7utikcu/include/superlu_defs.h:55,
                   from /..../spack_store/linux-sapphirerapids/superlu-dist-9.2.1-qwce4vb3xuhwgu4xjfdsfoxfn7utikcu/include/superlu_ddefs.h:37,
                   from /.../patch_fenics0.11/cpp/dolfinx/la/superlu_dist.cpp:13:
> /...../softs/compilers/gcc/15.3.0/Installed/lib/gcc/x86_64-redhat-linux/15/include/omp.h:448:1: error: template with C linkage
    448 | template<typename __T, omp_allocator_handle_t __Handle>
        | ^~~~~~~~
> /...../patch_fenics0.11/cpp/dolfinx/la/superlu_dist.cpp:11:1: note: 'extern "C"' linkage started here
     11 | extern "C"
        | ^~~~~~~~~~
> /..../softs/compilers/gcc/15.3.0/Installed/lib/gcc/x86_64-redhat-linux/15/include/omp.h:475:1: error: template with C linkage
....

and here the context of line 448 in omp.h

#if __cplusplus >= 201103L

/* std::__throw_bad_alloc and std::__throw_bad_array_new_length.  */
#include <bits/functexcept.h>

namespace omp
{
namespace allocator
{

namespace __detail
{

template<typename __T, omp_allocator_handle_t __Handle> //ligne 448
struct __allocator_templ
{
  using value_type = __T;

I hope it meets your needs. Thanks!

4

Thanks for the full report. I’ve made a PR at: DOLFINx does not compile with GCC 15 when SuperLU_DIST is enabled and OpenMP is on (`-fopenmp`) by jorgensd · Pull Request #4708 · FEniCS/dolfinx · GitHub

For now you could apply the patch yourself locally and sandbox spack, or downgrade to a gcc pre 15 until it is merged and patched onto the 0.11 release.