进程池中的共享内存

Posted 2023-02-16

【中文标题】进程池中的共享内存

【英文标题】：Shared memory in pool of processes

【发布时间】：2021-01-17 00:18:39

【问题描述】：

我正在尝试在 4 个进程之间共享内存，但它不适合我，我想就我的问题寻求帮助。

看一下流程的逻辑：

from multiprocessing import Pool, Value
from time import sleep

def work(process):
    if process == 'A':
        sleep(secs=2)     # Processing task A
        a = True          # Marking a shared byte of completed task
    if process == 'B':
        sleep(secs=1)     # Starting processing task B
        while a is False: # Waiting until task A will complete
            pass
        sleep(secs=5)     # Continuing processing task B
        b = True          # Marking a shared byte of completed task
    if process == 'C':
        sleep(secs=7)     # Processing task C
    if process == 'D':
        sleep(secs=1)     # Starting processing task D
        while b is False: # Waiting until task B will complete
            pass
        sleep(secs=4)     # Continuing processing task D

def main():
    a = Value('b', False)
    b = Value('b', False)

    with Pool(processes=4) as pool:
        pool.map(func=work, iterable=('A', 'B', 'C', 'D'))

if __name__ == '__main__':
    main()

我无法共享值 a 和 b，我该怎么做？

非常感谢！

【参考方案1】：

首先，函数sleep 不 采用关键字secs= 参数。其次，您需要将值 a 和 b 作为参数传递给您的函数 work。第三，您需要设置和测试您的Value 实例的value 属性。第四，由于您将这些参数传递给池进程，您应该从SyncManager 实例中获取Value 实例，如下所示。

from multiprocessing import Pool, Manager
from time import sleep
from functools import partial

def work(a, b, process):
    if process == 'A':
        sleep(2)     # Processing task A
        print('A is marking a True')
        a.value = True          # Marking a shared byte of completed task
    elif process == 'B':
        sleep(1)     # Starting processing task B
        while a.value is False: # Waiting until task A will complete
            pass
        sleep(5)     # Continuing processing task B
        print('B is marking b True')
        b.value = True          # Marking a shared byte of completed task
    elif process == 'C':
        sleep(7)     # Processing task C
    elif process == 'D':
        sleep(1)     # Starting processing task D
        while b.value is False: # Waiting until task B will complete
            pass
        print('D has now found b to be True')
        sleep(4)     # Continuing processing task D

def main():
    manager = Manager()
    a = manager.Value('b', False)
    b = manager.Value('b', False)

    with Pool(processes=4) as pool:
        pool.map(func=partial(work, a, b), iterable=('A', 'B', 'C', 'D'))

if __name__ == '__main__':
    main()

打印：

A is marking a True
B is marking b True
D has now found b to be True

【参考方案2】：

您可以使用队列（multiprocessing.Queue()）来回传递消息。

查看此链接https://pymotw.com/2/multiprocessing/communication.html