进程间数据传递：Queue，Pipe 进程间数据共享：Manager

Posted 2021-01-08 staff

1.使用multiprocessing模块的Queue实现数据传递

‘‘‘
进程间通讯：
    Queue，用法跟线程里的Queue一样,put,get
    线程queue没有做序列化，进程queue做序列化了
    父进程的queue怎么传给子进程的？父进程的queue克隆了一份到子进程
    按理说两个queue没关系了。子进程向queue里放东西，queue序列化到一个中间地方
    父进程取就从中间地方反序列化，他们只是实现的数据的传递
‘‘‘
from multiprocessing import Queue,Process

def f(q):
    q.put("alex")

if __name__ == ‘__main__‘:
    q = Queue()
    p = Process(target=f, args=(q,))
    p.start()
    p.join()
    print(q.get())  # alex

 

2.使用multiprocessing模块的Pipe实现数据传递

‘‘‘
进程间通讯：
管道，用法类似于socket
一次send对应一次recv，如果对应次数不对会出现阻塞
‘‘‘
from multiprocessing import Process
from multiprocessing import Pipe

def f(conn):
    conn.send([42,None,‘hello‘])
    print(conn.recv())
    conn.close()

if __name__ == ‘__main__‘:
    parent_conn, child_conn = Pipe()
    p = Process(target=f, args=(child_conn,))
    p.start()
    print(parent_conn.recv())
    parent_conn.send(‘hello world‘)
    p.join()
‘‘‘
[42, None, ‘hello‘]
hello world
‘‘‘

 

3.使用multiprocessing模块的Manager实现数据共享

‘‘‘
两个进程怎么同时修改一份数据？
即数据的共享，需要用到manager
其实是copy了几份，再合并。
manager支持的类型有：
list, dict, Namespace, Lock, RLock, Semaphore, BoundedSemaphore, Condition, Event, Barrier, Queue, Value and Array.
manager是自动加锁的
‘‘‘
from multiprocessing import Manager,Process
import os
def f(d, l):
    d[os.getpid()] = os.getpid()
    l.append(os.getpid())
    print(l)

if __name__ == ‘__main__‘:
    with Manager() as manager:
        d = manager.dict() # 生成一个字典，可在多个进程间共享和传递
        l = manager.list(range(5)) # 列表...

        p_list = []
        for i in range(10):
            p = Process(target=f, args=(d, l))
            p.start()
            p_list.append(p)
        for res in p_list:
            res.join()
        print(d)
        print(l)
‘‘‘
[0, 1, 2, 3, 4, 3088]
[0, 1, 2, 3, 4, 3088, 7092]
[0, 1, 2, 3, 4, 3088, 7092, 5256]
[0, 1, 2, 3, 4, 3088, 7092, 5256, 6460]
[0, 1, 2, 3, 4, 3088, 7092, 5256, 6460, 6696]
[0, 1, 2, 3, 4, 3088, 7092, 5256, 6460, 6696, 6796]
[0, 1, 2, 3, 4, 3088, 7092, 5256, 6460, 6696, 6796, 6640]
[0, 1, 2, 3, 4, 3088, 7092, 5256, 6460, 6696, 6796, 6640, 6296]
[0, 1, 2, 3, 4, 3088, 7092, 5256, 6460, 6696, 6796, 6640, 6296, 5308]
[0, 1, 2, 3, 4, 3088, 7092, 5256, 6460, 6696, 6796, 6640, 6296, 5308, 6704]
{3088: 3088, 7092: 7092, 5256: 5256, 6460: 6460, 6696: 6696, 6796: 6796, 6640: 6640, 6296: 6296, 5308: 5308, 6704: 6704}
[0, 1, 2, 3, 4, 3088, 7092, 5256, 6460, 6696, 6796, 6640, 6296, 5308, 6704]
‘‘‘