# Parallel multiple tasks in class-based program

**URL:** <https://fenicsproject.discourse.group/t/parallel-multiple-tasks-in-class-based-program/427>\
**Category:** Uncategorized\
**Created:** [March 23, 2019, 8:48pm UTC](https://fenicsproject.discourse.group/t/parallel-multiple-tasks-in-class-based-program/427 "2019-03-23T20:48:26Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![111](https://yyz2.discourse-cdn.com/free1/user_avatar/fenicsproject.discourse.group/111/32/104_2.png) [@111](https://fenicsproject.discourse.group/u/111)\
**Post date:** [March 23, 2019, 8:48pm UTC](https://fenicsproject.discourse.group/t/parallel-multiple-tasks-in-class-based-program/427/1 "2019-03-23T20:48:26Z")

</div>

I have few different equations, that i need to solve. Every equation takes approximately 1 minute to solve, but N equations take N minutes to solve if I will solve their sequential. It is very long for me and I want to solve their parallel. I use python multiprocessing module, but it throws me a error:

> TypeError: can’t pickle dolfin.cpp.generation.UnitSquareMesh objects

I write my code in class - based style (I think, it’s the source of problem). On the example of poisson equation, it looks something like this:

```
class PoissonSolver:
    class Boundary(SubDomain):
        def inside(self, x, on_boundary):
            return on_boundary

    def run_solver(self):
        self.__define_mesh()
        self.__define_bc()
        self.__define_functions()
        self.__run_processes()

    def __define_mesh(self):
        self.mesh = UnitSquareMesh(30, 30)
        self.V = FunctionSpace(self.mesh, "Lagrange", 1)

    def __define_bc(self):
        self.u_D = Expression('1 + x[0] * x[0] + 2 * x[1] * x[1]', degree=2)
        self.bc = DirichletBC(self.V, self.u_D, self.Boundary())

    def __define_functions(self):
        self.u = TrialFunction(self.V)
        self.v = TestFunction(self.V)
        self.a = dot(grad(self.u), grad(self.v)) * dx
        self.f = [Constant(-i) for i in range(10)]
        self.L = [fun * self.v * dx for fun in self.f]

    def __run_processes(self):
        pool = multiprocessing.Pool(processes=4)
        pool.map(self.solve_eq, range(10))

    def solve_eq(self, i):
        u = Function(self.V)
        solve(self.a == self.L[i], u, self.bc)
        File(self.mesh.mpi_comm(), 'foo%g.xml' % i) << u

if __name__ == " __main__":
    solver = PoissonSolver()
    solver.run_solver()

```

I know that the multiprocessing module use pickle to share data between processes, but how can I get around it and solve my problem? Thank you for answers.  
Ubuntu, dolfin-version: 2018.1.0.

---

<div class="post-metadata">

**Author:** ![MiroK](https://yyz2.discourse-cdn.com/free1/user_avatar/fenicsproject.discourse.group/mirok/32/33_2.png) [@MiroK](https://fenicsproject.discourse.group/u/MiroK)\
**Post date:** [March 24, 2019, 8:48am UTC](https://fenicsproject.discourse.group/t/parallel-multiple-tasks-in-class-based-program/427/2 "2019-03-24T08:48:04Z")

</div>

Hi, consider

```python
from dolfin import *
import multiprocessing

def poisson_solver(i):
    mesh = UnitSquareMesh(MPI.comm_self, 30, 30)
    V = FunctionSpace(mesh, "Lagrange", 1)

    u_D = Expression('1 + x[0] * x[0] + 2 * x[1] * x[1]', degree=2)
    bc = DirichletBC(V, u_D, 'on_boundary')

    u, v = TrialFunction(V), TestFunction(V)
    a = dot(grad(u), grad(v)) * dx
    f = Constant(-i)
    L = f * v * dx

    u = Function(V)
    solve(a == L, u, bc)
    File(mesh.mpi_comm(), 'foo%g.xml' % i) << u

# ----------------------------------------------------------------------

if __name__ == " __main__":
    poisson_solver(0) # JIT compile everything first

    pool = multiprocessing.Pool(processes=4)
    pool.map(poisson_solver, range(10))
    pool.close()

```

---

<div class="post-metadata">

**Author:** ![111](https://yyz2.discourse-cdn.com/free1/user_avatar/fenicsproject.discourse.group/111/32/104_2.png) [@111](https://fenicsproject.discourse.group/u/111)\
**Post date:** [March 24, 2019, 9:30am UTC](https://fenicsproject.discourse.group/t/parallel-multiple-tasks-in-class-based-program/427/3 "2019-03-24T09:30:10Z")

</div>

Hi, thank you for answer! As I understand it in this example we don’t share dolfin objects between processes, as result it works correctly without exceptions. Is there any way to create the same objects (mesh, function space etc.) in advance and run in separate processes only different parts (constant creation and solve execution)? Or it’s impossible because dolfin objects can’t be pickled and we can’t pass them on to the process?

---

<div class="post-metadata">

**Author:** ![urbainvaes](https://yyz2.discourse-cdn.com/free1/user_avatar/fenicsproject.discourse.group/urbainvaes/32/509_2.png) [@urbainvaes](https://fenicsproject.discourse.group/u/urbainvaes)\
**Post date:** [September 16, 2019, 4:03pm UTC](https://fenicsproject.discourse.group/t/parallel-multiple-tasks-in-class-based-program/427/4 "2019-09-16T16:03:36Z")

</div>

I’m also trying to do exactly this.

---

<div class="post-metadata">

**Author:** ![robert](https://avatars.discourse-cdn.com/v4/letter/r/e95f7d/32.png) [@robert](https://fenicsproject.discourse.group/u/robert)\
**Post date:** [December 9, 2019, 8:54pm UTC](https://fenicsproject.discourse.group/t/parallel-multiple-tasks-in-class-based-program/427/5 "2019-12-09T20:54:44Z")

</div>

I also have the same question.

---

<div class="post-metadata">

**Author:** ![Andre\_Gesing](https://yyz2.discourse-cdn.com/free1/user_avatar/fenicsproject.discourse.group/andre_gesing/32/285_2.png) [@Andre\_Gesing](https://fenicsproject.discourse.group/u/Andre_Gesing)\
**Post date:** [October 14, 2020, 2:19pm UTC](https://fenicsproject.discourse.group/t/parallel-multiple-tasks-in-class-based-program/427/6 "2020-10-14T14:19:48Z")

</div>

Similar question. Anyone had any progress with it?

---

<div class="post-metadata">

**Author:** ![111](https://yyz2.discourse-cdn.com/free1/user_avatar/fenicsproject.discourse.group/111/32/104_2.png) [@111](https://fenicsproject.discourse.group/u/111)\
**Post date:** [October 14, 2020, 4:29pm UTC](https://fenicsproject.discourse.group/t/parallel-multiple-tasks-in-class-based-program/427/7 "2020-10-14T16:29:08Z")

</div>

Hi guys @Andre_Gesing, @robert, @urbainvaes.  
The only way I’ve come up with is to save objects to files on disk and read it in each process.  
In this way I avoided pickle serialisation and deserialisation.

---

<div class="post-metadata">

**Author:** ![nate](https://yyz2.discourse-cdn.com/free1/user_avatar/fenicsproject.discourse.group/nate/32/17_2.png) [@nate](https://fenicsproject.discourse.group/u/nate)\
**Post date:** [October 14, 2020, 7:13pm UTC](https://fenicsproject.discourse.group/t/parallel-multiple-tasks-in-class-based-program/427/8 "2020-10-14T19:13:46Z")

</div>

Patrick Farrell’s [defcon](https://bitbucket.org/pefarrell/defcon) project could perhaps provide a guide? `defcon` splits the MPI communicator so multiple `DefconWorker`s handle “parallel serialised” computation for the purpose of deflated continuation algorithms.
