Segmentation fault in generating a FunctionSpace from a large mesh

1

Hello,

My code is getting stuck at the point of generating a FunctionSpace, it hangs at calling FunctionSpace for half an hour and then just error out with segmentation fault. I have attached my code as followed. I allocated 754Gb of memory for this job, I am not sure if this was a memory limit issue. Does anyone know how to fix this?

from dolfin import *
parameters["form_compiler"]["optimize"]     = True
parameters["form_compiler"]["cpp_optimize"] = True
parameters["form_compiler"]["representation"] = "uflacs"

mesh = BoxMesh(Point(0.0,0.0,0.0),Point(5.0e-2,5.0e-2,5.0e-2),100,100,100)

a  = FiniteElement("CG",  mesh.ufl_cell(), 4) 

b = VectorElement("Lagrange",  mesh.ufl_cell(), 2) 

c = MixedElement([a, b])

d = FunctionSpace(mesh, c)
2

Your function space will have about \sim80 million degrees of freedom, and you probably run out of memory.
How many processes (cores) are you running on?

Have you checked the memory usage for a smaller problem (like 50x50x50 and 75x75x75)?

1 Like
3

Right, looks like a 22x22x22 problem already fills up all the memory I have on this computing node, what would you recommend to do if I still need to push to a finer mesh?

4

Using memory_profiler (memory-profiler · PyPI) I ran the following code on my laptop (in serial one process using python3 -m memory_profiler 3D_prob.py

from dolfin import *
parameters["form_compiler"]["optimize"]     = True
parameters["form_compiler"]["cpp_optimize"] = True
parameters["form_compiler"]["representation"] = "uflacs"

@profile
def create_space(N):
    mesh = BoxMesh(Point(0.0,0.0,0.0),Point(5.0e-2,5.0e-2,5.0e-2), N, N, N)

    a  = FiniteElement("CG",  mesh.ufl_cell(), 4) 

    b = VectorElement("Lagrange",  mesh.ufl_cell(), 2) 

    c = MixedElement([a, b])

    d = FunctionSpace(mesh, c)


if __name__ == "__main__":
    create_space(22)

and got the following results:

Calling FFC just-in-time (JIT) compiler, this may take some time.
Calling FFC just-in-time (JIT) compiler, this may take some time.
Calling FFC just-in-time (JIT) compiler, this may take some time.
Calling FFC just-in-time (JIT) compiler, this may take some time.
Filename: 3D_prob.py

Line #    Mem usage    Increment  Occurences   Line Contents
============================================================
     6   97.082 MiB   97.082 MiB           1   @profile
     7                                         def create_space(N):
     8   99.758 MiB    2.676 MiB           1       mesh = BoxMesh(Point(0.0,0.0,0.0),Point(5.0e-2,5.0e-2,5.0e-2), N, N, N)
     9                                         
    10   99.980 MiB    0.223 MiB           1       a  = FiniteElement("CG",  mesh.ufl_cell(), 4) 
    11                                         
    12   99.980 MiB    0.000 MiB           1       b = VectorElement("Lagrange",  mesh.ufl_cell(), 2) 
    13                                         
    14   99.980 MiB    0.000 MiB           1       c = MixedElement([a, b])
    15                                         
    16  994.715 MiB  894.734 MiB           1       d = FunctionSpace(mesh, c)

I.e. the memory usage should be about 1 GB.
This indicates that something else might be wrong with your installation.
What system are you using, and how have you installed dolfin?

1 Like
5

Interesting, I will get memory_profiler later and see what it shows. I am running the codes on Linux CentOS, fenics is installed with anaconda.

6

My memory_profiler is nearly identical as yours.

Line #    Mem usage    Increment  Occurences   Line Contents
============================================================
     6   80.660 MiB   80.660 MiB           1   @profile
     7                                         def create_space(N):
     8   82.371 MiB    1.711 MiB           1       mesh = BoxMesh(Point(0.0,0.0,0.0),Point(5.0e-2,5.0e-2,5.0e-2), N, N, N)
     9                                         
    10   82.637 MiB    0.266 MiB           1       a  = FiniteElement("CG",  mesh.ufl_cell(), 4) 
    11                                         
    12   82.637 MiB    0.000 MiB           1       b = VectorElement("Lagrange",  mesh.ufl_cell(), 2) 
    13                                         
    14   82.637 MiB    0.000 MiB           1       c = MixedElement([a, b])
    15                                         
    16  974.070 MiB  891.434 MiB           1       d = FunctionSpace(mesh, c)
7

But you get a segfault? Could you post the error message that you are getting, and what script you are executing?

8

Definitely something wrong with FEniCS here (or there’s more to it than the 1GB in the memory profiler). Your test devoured all my swap (20G) and nearly crashed the system, I only barely killed it in time. I’m running latest library versions on Debian experimental.

There’s been a lot of changes in dolfinx. Try the equivalent test in dolfinx.

9

Did you run with N=22 or N=100?
N=22 should give 1 GB usage, N=100 i would guess would do over 300 GB

10

I just copied and pasted, so yeah, N=100. I’ll try N=22

…completes in 15 sec, no errors. Didn’t have time to check memory usage.