SLEPC solver running in Parallel

1

Hi,

I am trying to solve Helmholtz equation in 2D and 3D. I can successfully calculate eigenvalues and eigenfunctions in serial running.

However when it comes to work in parallel, problems are taking longer than serial one. I have following settings for SLEPC;

def eps_solver(A, C, target, nev, two_sided=False):

    E = SLEPc.EPS().create(MPI.COMM_WORLD)

    C = - C
    E.setOperators(A, C)
    
    st = E.getST()
    st.setType('sinvert')

    E.setTarget(target) 
    E.setWhichEigenpairs(SLEPc.EPS.Which.TARGET_MAGNITUDE)  

    E.setTwoSided(two_sided)
    
    E.setDimensions(nev, PETSc.DECIDE)
    E.setTolerances(1e-8)
    E.setFromOptions()

    E.solve()

They are working excellent for serial running. However I need to solve the problem for larger and finer complex geometries. For this reason, the speed up in calculation becomes essential.

I am running Mirok’s example in here. It works perfectly for multiple cores. But my problem is taking so long when I run it in parallel.

I am wondering is current slepc4py is not supporting running in parallel?

Any suggestions or help would be much appreciated.

2

If you’re not getting expected parallel speed-up, that can sometimes be a result of BLAS configuration. There are a couple of BLAS questions to consider

  1. The default BLAS implementation is generic, not tuned to any processor and therefore relatively slow. Better performance would be available from specialized implementations such as OpenBLAS (this applies to LAPACK also),
  2. For BLAS parallelization you want a BLAS configuration that uses threads (pthreads or openmp) cf. Multicore and Ubuntu - #8 by dparsons . This particularly applies when running as a single-process job (see next point).
  3. DOLFIN (and PETSc/SLEPc) are MPI software. MPI is a different parallelization to threading and it’s hard to make the two techniques cooperate effectively at the same time. If you’re running as an MPI job, then you might get better performance blocking BLAS threads by setting the environment variable OMP_NUM_THREADS=1 , cf. Multicore and Ubuntu - #7 by dparsons
1 Like
3

Thanks for your reply.

First I have opened bashrc file by;

gedit ~/.bashrc

In the bashrc file, I have added enviromental variable to the bottom of the file;

export OMP_NUM_THREADS=1

and applied changes by

source ~/.bashrc

solved my problem.

4

I wouldn’t suggest setting it universally in .bashrc. Other applications might want to use full threading. But good to hear it helps SLEPc fire on all engines.