Python如何使用多处理来返回不同的函数

Posted 2023-02-16

【中文标题】Python如何使用多处理来返回不同的函数

【英文标题】：Python how to use multiprocessing to different functions with return

【发布时间】：2021-11-27 23:23:26

【问题描述】：

我有几个 3 个小功能：

def getnpx(mt, age, interest):
    val = 1
    initval = 1
    for i in range(age, 55):
        val = val * mt[i]
        intval = val / (1 + interest) ** (i + 1 - age)
        initval = initval + intval
        
    return initval

def getnpx2(mt, age, interest):
    val = mt[age]
    initval = 1
    for i in range(age + 2, 55):
        val *= mt[i - 1]
        if mt[age]==0:
            intval =0
        else:
            intval = val / (1 + interest) ** (i - age - 1) / mt[age]
        initval = initval + intval
    return initval

def getnpxtocert(mt, age, maxvalue):
    val = mt[age]
    for i in range(age + 1, min(maxvalue, 7)):
        val = val * mt[i]
    return val

还有一个调用所有小函数的大函数：

def calcannfactprelim(pval, age, intrate, certper):
    npx = getnpx(pval, age + int(certper), intrate)
    npx2 = getnpx2(pval, age + int(certper), intrate)

    if certper == 0:
        index = 1
        index2 = pval[age + int(certper)]

    else:
        index = getnpxtocert(pval, age,
                             age + int(certper)) 
        index2 = getnpxtocert(pval, age,age + int(certper) + 1)

    return index*npx+index2*npx2

这些是要使用的变量：

pval = np.array([0.000291,0.00027,0.000257,0.000294,0.000325,0.00035,0.000371,0.000388,0.000402,0.000414,0.000425,0.000437,0.011016,0.012251,0.013657,0.015233,0.016979,0.018891,0.020967,0.023209,0.025644,0.028304,0.03122,0.034425,0.037948,0.041812,0.046037,0.050643,0.055651,0.06108,0.066948,0.073275,0.080076,0.08737,0.095169,0.103455,0.112208,0.121402,0.131017,0.14103,0.151422,0.162179,0.173279,0.184706,0.196946,0.210484,0.225806,0.243398,0.263745,0.287334,0.314649,0.346177,0.382403,0.423813,0.470893])
age=3
intrate=0.04
certper=1

常规功能测试结果：

start=time.time()
print(calcannfactprelim(pval, age, intrate, certper))
print(time.time()-start)

输出是：

0.0002941874880982305 #result
0.0                   #time

为了让这个函数更快，我使用python多重处理来并行运行它。

import multiprocessing

def calcannfactprelim_v(pval, age, intrate, certper):
    
    p1 = multiprocessing.Process(target=getnpx, args=(pval, age, intrate, certper,))
    p2 = multiprocessing.Process(target=getnpx2, args=(pval, age, intrate, certper,))

    # starting process 1
    p1.start()
    # starting process 2
    p2.start()

    # wait until process 1 is finished
    p1.join()
    # wait until process 2 is finished
    p2.join()

    # both processes finished
    if certper == 0:
        index = 1
        index2 = pval[age + int(certper)]

    else:
        index = getnpxtocert(pval, age,
                             age + int(certper)) 
        index2 = getnpxtocert(pval, age,age + int(certper) + 1)


    return index*npx+index2*npx2

但是我不知道这种情况如何返回值，有哪位朋友可以帮忙完成代码并做个测试？

【问题讨论】：

使用multiprocessing.Process()，您需要为进程创建一个队列，以便将值返回给主进程。或者，您可以使用多进程池，这将为您做同样的事情。有一些简单的例子here

【参考方案1】：

在父进程中创建一个对象，比如说，

class ResultHolder():
     def __init__():
          result_a = None
          ………
          result_n = None

将此类作为参数传递给要在子进程中运行的函数，并让函数将结果写入，例如：

instance_of_result_holder.result_a = the_function_output

我相信您也可以通过传递字典或列表来做到这一点。长话短说，对象以类似于传递引用的方式传递，这意味着您可以将它们用作父进程和子进程的两个独立世界（上下文）之间的中介（即子进程可以改变定义的对象在父级中，您将后者作为参数传递给相应的函数）。

池和队列是实现这一目标的另一种方式。

【参考方案2】：

您需要一些渠道来返回数据。这是在多处理池中为您完成的。

def calcannfactprelim_v(pval, age, intrate, certper):
    with multiprocessing.pool(2) as pool:
        a1 = pool.apply_async(getnpx, (pval, age, intrate, certper,))
        a2 = pool.apply_async(getnpx2, (pval, age, intrate, certper,))
        result1 = a1.get()
        result2 = a2.get()

    # both processes finished
    if certper == 0:
        index = 1
        index2 = pval[age + int(certper)]

    else:
        index = getnpxtocert(pval, age,
                             age + int(certper)) 
        index2 = getnpxtocert(pval, age,age + int(certper) + 1)
    return index*npx+index2*npx2

您甚至可以创建一个 1 池并在主程序中执行其中一个调用。假设一个返回一个大值，另一个返回一个小值。在你的 main 中做大的，这样从进程中将值返回给父进程的开销会更少。