无法使用 python 的多处理 Pool.apply_async() 腌制 <type 'instancemethod'>

Posted 2023-02-16

【中文标题】无法使用 python 的多处理 Pool.apply_async() 腌制 <type \'instancemethod\'>

【英文标题】：Can't pickle <type 'instancemethod'> using python's multiprocessing Pool.apply_async()无法使用 python 的多处理 Pool.apply_async() 腌制 <type 'instancemethod'>

【发布时间】：2014-09-29 04:12:05

【问题描述】：

我想运行这样的东西：

from multiprocessing  import Pool
import time
import random

class Controler(object):
    def __init__(self):
        nProcess = 10
        pages = 10
        self.__result = []
        self.manageWork(nProcess,pages)

def BarcodeSearcher(x):
    return x*x

def resultCollector(self,result):
    self.__result.append(result)

def manageWork(self,nProcess,pages):
    pool = Pool(processes=nProcess)
    for pag in range(pages):
        pool.apply_async(self.BarcodeSearcher, args = (pag, ), callback = self.resultCollector)

    print self.__result

if __name__ == '__main__':
    Controler()

但代码导致错误：

   Exception in thread Thread-1:
Traceback (most recent call last):
  File "C:\Python26\lib\threading.py", line 522, in __bootstrap_inner
    self.run()
  File "C:\Python26\lib\threading.py", line 477, in run
    self.__target(*self.__args, **self.__kwargs)
  File "C:\python26\lib\multiprocessing\pool.py", line 225, in _handle_tasks
    put(task)
PicklingError: Can't pickle <type 'instancemethod'>: attribute lookup __builtin__.instancemethod failed

我看过帖子（post1，post2）来解决我的问题。我在第二篇文章中寻找类似 Mike McKerns 的解决方案，但没有使用悲情。

【参考方案1】：

正如 Alex Martelli 在您提供的第一个链接中所建议的那样，使用 copy_reg 有效：

import copy_reg
import types
import multiprocessing


def _pickle_method(m):
    if m.im_self is None:
        return getattr, (m.im_class, m.im_func.func_name)
    else:
        return getattr, (m.im_self, m.im_func.func_name)

copy_reg.pickle(types.MethodType, _pickle_method)


class Controler(object):
    def __init__(self):
        nProcess = 10
        pages = 10
        self.__result = []
        self.manageWork(nProcess, pages)

    def BarcodeSearcher(self, x):
        return x*x

    def resultCollector(self, result):
        self.__result.append(result)

    def manageWork(self, nProcess, pages):
        pool = multiprocessing.Pool(processes=nProcess)
        for pag in range(pages):
            pool.apply_async(self.BarcodeSearcher, args=(pag,),
                             callback=self.resultCollector)
        pool.close()
        pool.join()

        print(self.__result)

if __name__ == '__main__':
    Controler()

【讨论】：

收到错误TypeError: expected string or Unicode object, NoneType found