python 进程池

Posted 2022-06-30 厚积_薄发

multiprocessing.dummy replicates the API of multiprocessing but is no more than a wrapper around the threading module.

// 实际使用的是线程池而不是进程池

如果把

from multiprocessing.dummy import Pool
改成
from multiprocessing import Pool

则实际是进程池。

jobs = []
jobs.append(Job(a,b,c));
自己定义 Job 的类，jobs 是 job 实例的list.
Class Job():
	def __init__(self, a, b, c):
		...
	def do():
		...
	def after():
		...

from multiprocessing.dummy import Pool
pool = Pool(40)
        pool.map(merge_in_each_process, tuple(jobs))
        
def doJob(job):
    job.do()
    job.after()

此时，可以实现40个并发，执行 job 的内容 job.do() 和 job.after().

可参考： https://docs.python.org/2.7/library/multiprocessing.html#module-multiprocessing.dummy