Python ThreadPool 映射中的多个数组

Posted 2023-03-15

【中文标题】Python ThreadPool 映射中的多个数组

【英文标题】：Multiple arrays in Python ThreadPool map

【发布时间】：2021-11-07 05:49:59

【问题描述】：

我正在编写一个 python 程序，为工人提供工作。我遇到了multiprocess.pool.ThreadPool 的问题，它只允许 1 次迭代。我希望这两名工人拥有相同（或几乎）的工作量。代码如下：

from multiprocessing.pool import ThreadPool

def f(worker,work):
    print(f"worker | work")

workers = ["worker1","worker2"]
work = [1,2,3,4,5,6,7,8,9,10]

if __name__ == "__main__":
    with ThreadPool(len(workers)) as p:
        p.map(f,zip(workers,work))

预期输出（不需要工单）：

worker1 | 1
worker2 | 2
worker1 | 4
worker2 | 3
...

任何帮助将不胜感激。谢谢。

【参考方案1】：

我想你想要ThreadPool.starmap:

from multiprocessing.pool import ThreadPool
from itertools import cycle

def f(worker,work):
    print(f"worker | work")

workers = ["worker1","worker2"]
work = [1,2,3,4,5,6,7,8,9,10]

if __name__ == "__main__":
    with ThreadPool(len(workers)) as p:
        p.starmap(f,zip(cycle(workers),work))

输出：

worker1 | 1
worker2 | 2
worker1 | 3
worker1 | 5
worker2 | 4
worker2 | 6
worker1 | 7
worker1 | 9
worker2 | 10
worker2 | 8

---

注意，压缩worker时还需要使用cycle()，否则只会使用work列表中的前两个元素。