Ticket #214 (closed defect: fixed)

Opened 3 years ago

Last modified 3 years ago

nested solvers throw PicklingError with mpi Pool map

Reported by: mmckerns Owned by: mmckerns
Priority: major Milestone: mystic-0.2a2
Component: pathos Keywords:
Cc:

Description (last modified by mmckerns) (diff)

nested solvers throw cPickle.PicklingError with mpi Pool map

Change History

comment:1 Changed 3 years ago by mmckerns

  • Description modified (diff)

comment:2 Changed 3 years ago by mmckerns

looks like from the traceback: 

  File "/Library/Frameworks/Python.framework/.../pyina.../scripts/ezpool.py", line 50, in <module>
    res = parallel_map(func, *args, **kwds) #XXX: called on ALL nodes ?
  File "/Library/Frameworks/Python.framework/.../pyina.../pyina/mpi_pool.py", line 123, in parallel_map
    comm.send(result, master, tag) #XXX: or write to results then merge?
  File "Comm.pyx", line 753, in mpi4py.MPI.Comm.send (src/mpi4py.MPI.c:53848)
  File "pickled.pxi", line 122, in mpi4py.MPI.PyMPI_send (src/mpi4py.MPI.c:20409)
  File "pickled.pxi", line 39, in mpi4py.MPI._p_Pickle.dump (src/mpi4py.MPI.c:19503)
cPickle.PicklingError: Can't pickle <type 'function'>: attribute lookup __builtin__.function failed

it'd be fixable by setting mpi4py's load and dump functions to use dill's as detailed on  the mpi4py mailing list: 

from mpi4py import MPI
MPI._p_pickle.dumps = dill.dumps
MPI._p_pickle.loads = dill.loads

comment:3 Changed 3 years ago by mmckerns

  • Status changed from new to assigned

comment:4 Changed 3 years ago by mmckerns

  • Status changed from assigned to closed
  • Resolution set to fixed

works, if upgrade to mpi4py-1.3.

See  pathos ticket #148.

Closed.

Note: See TracTickets for help on using tickets.