# Solving 3D Stokes problem is killed **URL:** \ **Category:** dolfinx\ **Tags:** io, mpi\ **Created:** [November 15, 2025, 2:55am UTC](https://fenicsproject.discourse.group/t/solving-3d-stokes-problem-is-killed/18370 "2025-11-15T02:55:00Z")\ **Posts on this page:** 16\ **Page:** 1
**Author:** ![ZhangQi](https://avatars.discourse-cdn.com/v4/letter/z/da6949/32.png) [@ZhangQi](https://fenicsproject.discourse.group/u/ZhangQi)\ **Post date:** [November 15, 2025, 2:55am UTC](https://fenicsproject.discourse.group/t/solving-3d-stokes-problem-is-killed/18370/1 "2025-11-15T02:55:00Z")
My dear, thank you very much for your previous assistance. The issue regarding the brain mesh has been resolved. However, the simulation failed due to the excessively dense mesh. To facilitate a better understanding of my program framework, I am providing the code supplied by [Solving 3D Stokes problem is slower on dolfinx than dolfin](https://fenicsproject.discourse.group/t/solving-3d-stokes-problem-is-slower-on-dolfinx-than-dolfin/15154), which follows the same program logic. Moreover, I have fixed a little bit of pressure. When executing with “mpirun -n 24”, the following error message occurs. I look forward to your reply. ```auto import os os.environ['OMP_NUM_THREADS'] = '1' from mpi4py import MPI comm = MPI.COMM_WORLD import numpy as np import time import ufl, basix from dolfinx import fem, io, mesh from ufl import grad, div, nabla_grad, dx, inner, dot from dolfinx.fem.petsc import LinearProblem # build mesh msh = mesh.create_box(MPI.COMM_WORLD, [np.array([0, 0, 0]), np.array([2, 1, 0.3])], [60, 60, 60], cell_type=mesh.CellType.tetrahedron) # define element Pv = basix.ufl.element("CG", msh.basix_cell(), 2, shape=(msh.geometry.dim,)) P1 = basix.ufl.element("CG", msh.basix_cell(), 1) TH = basix.ufl.mixed_element([Pv, P1]) # create functionspace W = fem.functionspace(msh, TH) V = fem.functionspace(msh, Pv) # define functions dU = ufl.TrialFunction(W) U = fem.Function(W) velocity = fem.Function(V) f = fem.Function(V) # set boundary conditions tdim = msh.topology.dim fdim = tdim - 1 msh.topology.create_connectivity(fdim, tdim) def LEFT(xx): return np.isclose(xx[0], 0) def RIGHT(xx): return np.isclose(xx[0], 2) def corner(x): return np.logical_and(np.isclose(x[2], 0.0), np.isclose(x[1], 0.0), np.isclose(x[0], 0.0)) boundary_surface = mesh.exterior_facet_indices(msh.topology) left_surface = mesh.locate_entities_boundary(msh, fdim, LEFT) right_surface = mesh.locate_entities_boundary(msh, fdim, RIGHT) side = np.concatenate((left_surface, right_surface)) other_surface = np.setdiff1d(boundary_surface, side) W_v_sub, _ = W.sub(0).collapse() Q0 = W.sub(1) V_in = fem.Function(W_v_sub) V_noslip = fem.Function(W_v_sub) V_noslip.x.array[:] = 0 V_in.sub(0).interpolate(lambda x: 1 * 4 * x[1] * (1 - x[1]) / (1) **2 * 4 * x[2] * (0.3 - x[2]) / (0.3)** 2) bcu = fem.dirichletbc(V_in, fem.locate_dofs_topological([W.sub(0), W.sub(0).collapse()[0]], fdim, left_surface), W.sub(0)) bcu1 = fem.dirichletbc(V_in, fem.locate_dofs_topological([W.sub(0), W.sub(0).collapse()[0]], fdim, right_surface), W.sub(0)) bcu2 = fem.dirichletbc(V_noslip, fem.locate_dofs_topological([W.sub(0), W.sub(0).collapse()[0]], fdim, other_surface), W.sub(0)) # fix pressure in one corner corner_vertex = mesh.locate_entities_boundary(msh, 0, corner) dofs = fem.locate_dofs_topological(Q0, 0, corner_vertex) bc3 = fem.dirichletbc(fem.Constant(msh, 0.0), dofs, Q0) bcs = [bcu, bcu1, bcu2] # define the linear problem h = 2 * ufl.Circumradius(msh) v, q = ufl.TestFunctions(W) u, p = ufl.split(U) du, dp = ufl.split(dU) F_a = ( -inner(dp, div(v)) * dx + inner(2 * ufl.sym(nabla_grad(du)), ufl.sym(nabla_grad(v))) * dx + inner(div(du), q) * dx + 0.01 * h * h * inner(grad(dp), grad(q)) * dx ) F_L = ( inner(f, v) * dx ) problem_U = LinearProblem(F_a, F_L, bcs=bcs, petsc_options={"ksp_type": "preonly", "pc_type": "lu", "pc_factor_mat_solver_type": "mumps"}) # solve ta = time.time() Uh = problem_U.solve() tb = time.time() if MPI.COMM_WORLD.rank == 0: print("Time: ", tb - ta) ``` Error ```auto =================================================================================== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = PID 8415 RUNNING AT LAPTOP-BBJKVL4R = EXIT CODE: 9 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =================================================================================== YOUR APPLICATION TERMINATED WITH THE EXIT STRING: Killed (signal 9) This typically refers to a problem with your application. Please see the FAQ page for debugging suggestions ``` **Version information:** DOLFINx version: 0.9.0 ---
**Author:** ![thingway](https://avatars.discourse-cdn.com/v4/letter/t/a5b964/32.png) [@thingway](https://fenicsproject.discourse.group/u/thingway)\ **Post date:** [November 15, 2025, 9:12am UTC](https://fenicsproject.discourse.group/t/solving-3d-stokes-problem-is-killed/18370/2 "2025-11-15T09:12:06Z")
I can run it with the same set as you , except h = 6. And memory access out of range when i run h = 60, a solver is needed. Reducing the number of processes might work. ---
**Author:** ![ZhangQi](https://avatars.discourse-cdn.com/v4/letter/z/da6949/32.png) [@ZhangQi](https://fenicsproject.discourse.group/u/ZhangQi)\ **Post date:** [November 15, 2025, 9:48am UTC](https://fenicsproject.discourse.group/t/solving-3d-stokes-problem-is-killed/18370/3 "2025-11-15T09:48:16Z")
Yes, it shows out of memory. ---
**Author:** ![ZhangQi](https://avatars.discourse-cdn.com/v4/letter/z/da6949/32.png) [@ZhangQi](https://fenicsproject.discourse.group/u/ZhangQi)\ **Post date:** [November 16, 2025, 8:59am UTC](https://fenicsproject.discourse.group/t/solving-3d-stokes-problem-is-killed/18370/4 "2025-11-16T08:59:46Z")
Can someone help me? Thank you！ ---
**Author:** ![dokken](https://yyz2.discourse-cdn.com/free1/user_avatar/fenicsproject.discourse.group/dokken/32/1560_2.png) [@dokken](https://fenicsproject.discourse.group/u/dokken)\ **Post date:** [November 16, 2025, 1:02pm UTC](https://fenicsproject.discourse.group/t/solving-3d-stokes-problem-is-killed/18370/5 "2025-11-16T13:02:38Z")
A minor change is to avoid repeating the call to `W.sub(0).collapse()` multiple times, and just use the output from the first call. ```auto W_v_sub, _ = W.sub(0).collapse() bcu = fem.dirichletbc( V_in, fem.locate_dofs_topological([W.sub(0), W_v_sub], fdim, left_surface), W.sub(0), ) bcu1 = fem.dirichletbc( V_in, fem.locate_dofs_topological([W.sub(0), W_v_sub], fdim, right_surface), W.sub(0), ) bcu2 = fem.dirichletbc( V_noslip, fem.locate_dofs_topological([W.sub(0), W_v_sub], fdim, other_surface), W.sub(0), ) ``` As I currently do not have a laptop with more than 16gb ram, I cannot run the code with the suggested resolution. How much memory do you have available ? ---
**Author:** ![Stein](https://yyz2.discourse-cdn.com/free1/user_avatar/fenicsproject.discourse.group/stein/32/8751_2.png) [@Stein](https://fenicsproject.discourse.group/u/Stein)\ **Post date:** [November 16, 2025, 1:13pm UTC](https://fenicsproject.discourse.group/t/solving-3d-stokes-problem-is-killed/18370/6 "2025-11-16T13:13:14Z")
Your system of equations is close to 5.5Mil dofs. At those numbers, direct solvers will get very memory intensive, which is probably the problem you’re running into. One option is to expand on the RAM your compute system has. But more appropriately, it is probably time you start to invest in getting an iterative solver to run. Those are way less memort intensive, and in general the way to go for solving big problems. In a nutshell, you’ll want to use GMRES with appropriate preconditioning. The preconditioning is really the crux, and also the main difficulty. On the forum you’ll find multiple threads that could be of use. An old thread of mine comes to mind: [Precondition appears not to happen - #5 by Stein](https://fenicsproject.discourse.group/t/precondition-appears-not-to-happen/15969/5) ---
**Author:** ![dokken](https://yyz2.discourse-cdn.com/free1/user_avatar/fenicsproject.discourse.group/dokken/32/1560_2.png) [@dokken](https://fenicsproject.discourse.group/u/dokken)\ **Post date:** [November 16, 2025, 1:18pm UTC](https://fenicsproject.discourse.group/t/solving-3d-stokes-problem-is-killed/18370/7 "2025-11-16T13:18:09Z")
When it comes to stokes equation, there are plenty of examples at > **[Stokes equations using Taylor-Hood elements — DOLFINx 0.10.0.post0 documentation](https://docs.fenicsproject.org/dolfinx/v0.10.0.post0/python/demos/demo_stokes.html)** ---
**Author:** ![Stein](https://yyz2.discourse-cdn.com/free1/user_avatar/fenicsproject.discourse.group/stein/32/8751_2.png) [@Stein](https://fenicsproject.discourse.group/u/Stein)\ **Post date:** [November 16, 2025, 1:32pm UTC](https://fenicsproject.discourse.group/t/solving-3d-stokes-problem-is-killed/18370/8 "2025-11-16T13:32:22Z")
Ah, right! Mistook the title for Navier-Stokes. But the iterative solvers and preconditioner a in that demo all act on block-structured matrices. For the mixed element, one has to build the IndexSet the way described in the thread I linked earlier. ---
**Author:** ![ZhangQi](https://avatars.discourse-cdn.com/v4/letter/z/da6949/32.png) [@ZhangQi](https://fenicsproject.discourse.group/u/ZhangQi)\ **Post date:** [November 16, 2025, 1:34pm UTC](https://fenicsproject.discourse.group/t/solving-3d-stokes-problem-is-killed/18370/9 "2025-11-16T13:34:20Z")
Thank you, my computer free memory in about 7 GB, I know this is far from enough, I have applied for a larger memory device, thank you for your reply. ---
**Author:** ![Stein](https://yyz2.discourse-cdn.com/free1/user_avatar/fenicsproject.discourse.group/stein/32/8751_2.png) [@Stein](https://fenicsproject.discourse.group/u/Stein)\ **Post date:** [November 16, 2025, 1:37pm UTC](https://fenicsproject.discourse.group/t/solving-3d-stokes-problem-is-killed/18370/10 "2025-11-16T13:37:11Z")
You’re using ‘mpirun -n 24’ on a system with 7GB of memory? ---
**Author:** ![ZhangQi](https://avatars.discourse-cdn.com/v4/letter/z/da6949/32.png) [@ZhangQi](https://fenicsproject.discourse.group/u/ZhangQi)\ **Post date:** [November 16, 2025, 1:38pm UTC](https://fenicsproject.discourse.group/t/solving-3d-stokes-problem-is-killed/18370/11 "2025-11-16T13:38:54Z")
Yes, I need to solve algebraic equations in terms of preconditioning iterations. This is a solver that I wrote ```auto ksp = PETSc.KSP().create(mesh.comm) ksp.setOperators(A) # 设置求解器选项 ksp.setType("gmres") ksp.setTolerances(rtol=1.0e-10, atol=1.0e-12, max_it=1000) pc = ksp.getPC() pc.setType("bjacobi") # 通过选项设置更详细的参数 当前残差 ≤ rtol × 初始残差 opts = PETSc.Options() opts["ksp_type"] = "gmres" opts["pc_type"] = "bjacobi" opts["sub_pc_type"] = "ilu" opts["sub_pc_factor_levels"] = 0 opts["ksp_rtol"] = 1.0e-10 opts["ksp_atol"] = 1.0e-12 opts["ksp_max_it"] = 1000 opts["ksp_monitor"] = None opts["ksp_error_if_not_converged"] = 1 ksp.setFromOptions() # Compute the solution wh = dolfinx.fem.Function(W) ksp.solve(b, wh.x.petsc_vec) ``` I will study your topic content next, thank you very much for your guidance, sincere thanks. ---
**Author:** ![ZhangQi](https://avatars.discourse-cdn.com/v4/letter/z/da6949/32.png) [@ZhangQi](https://fenicsproject.discourse.group/u/ZhangQi)\ **Post date:** [November 16, 2025, 1:40pm UTC](https://fenicsproject.discourse.group/t/solving-3d-stokes-problem-is-killed/18370/12 "2025-11-16T13:40:53Z")
Thank you for your help. I’m learning it. ---
**Author:** ![ZhangQi](https://avatars.discourse-cdn.com/v4/letter/z/da6949/32.png) [@ZhangQi](https://fenicsproject.discourse.group/u/ZhangQi)\ **Post date:** [November 16, 2025, 1:43pm UTC](https://fenicsproject.discourse.group/t/solving-3d-stokes-problem-is-killed/18370/13 "2025-11-16T13:43:43Z")
Yes, because I’m a beginner, I just took the test one by one from 2 to 24. ---
**Author:** ![ZhangQi](https://avatars.discourse-cdn.com/v4/letter/z/da6949/32.png) [@ZhangQi](https://fenicsproject.discourse.group/u/ZhangQi)\ **Post date:** [November 16, 2025, 1:53pm UTC](https://fenicsproject.discourse.group/t/solving-3d-stokes-problem-is-killed/18370/14 "2025-11-16T13:53:20Z")
This is the number of kernels and threads supported on my computer. ```auto NumberOfCores NumberOfLogicalProcessors 10 12 ``` ---
**Author:** ![dokken](https://yyz2.discourse-cdn.com/free1/user_avatar/fenicsproject.discourse.group/dokken/32/1560_2.png) [@dokken](https://fenicsproject.discourse.group/u/dokken)\ **Post date:** [November 16, 2025, 2:04pm UTC](https://fenicsproject.discourse.group/t/solving-3d-stokes-problem-is-killed/18370/15 "2025-11-16T14:04:12Z")
With only 7GB RAM, there is no way of using a direct solver to run the above code, as Stein already said (if it has over 5 million dofs). Additionally, using 24 processes for 5 million dofs is likely overkill, see for instance > [@Parallel slower runtime](https://fenicsproject.discourse.group/t/parallel-slower-runtime/8547/2): > > I cannot reproduce your problem: root@82df035a5e65:~/shared# python3 mwe.py MESH CREATED 0 , time : 0.08017910199987455 root@82df035a5e65:~/shared# mpirun -n 2 python3 mwe.py MESH CREATED 0 , time : 0.0637212950000503 MESH CREATED 1 , time : 0.06375895700011824 root@82df035a5e65:~/shared# mpirun -n 4 python3 mwe.py MESH CREATED 0 , time : 0.06122965600025054 MESH CREATED 1 , time : 0.06122667699992235 MESH CREATED 2 , time : 0.061187144000086846 MESH CREATED 3 , time : 0.0612255729997… ---
**Author:** ![ZhangQi](https://avatars.discourse-cdn.com/v4/letter/z/da6949/32.png) [@ZhangQi](https://fenicsproject.discourse.group/u/ZhangQi)\ **Post date:** [November 16, 2025, 2:09pm UTC](https://fenicsproject.discourse.group/t/solving-3d-stokes-problem-is-killed/18370/16 "2025-11-16T14:09:58Z")
Thank you for your guidance. I have applied with the research team to use a larger memory device.