multiprocessing：maxtasksperchild和chunksize冲突？

Question

我正在使用multiprocessing中的Python 3.7模块。我的代码未按预期工作（请参阅此问题here）。有人建议将maxtasksperchild设置为1。然后，在阅读文档时，我认为最好也将chunksize设置为1。这是相关的代码部分：

# Parallel Entropy Calculation
# ============================
node_combinations = [(i, j) for i in g.nodes for j in g.nodes]
pool = Pool(maxtaskperchild=1)
start = datetime.datetime.now()
logging.info("Start time: %s", start)
print("Start time: ", start)
results = pool.starmap(g._log_probability_path_ij, node_combinations, chunksize=1)
end = datetime.datetime.now()
print("End time: ", end)
print("Run time: ", end - start)
logging.info("End time: %s", end)
logging.info("Total run time: %s", start)
pool.close()
pool.join()

这事与愿违。仅设置maxtasksperchild或仅设置chunksize即可在预期的时间内完成工作（对于用于测试代码的较小数据集）。设置都无法完成，几秒钟后什么也没真正运行（我用htop检查了内核是否在工作）。

问题

maxtasksperchild和chunksize设置在一起时是否冲突？
他们是否做同样的事情？ maxtasksperchild级别的Pool()和chunksize方法级别的Pool？

============================================== ========

编辑

我了解从所提供的代码摘录中可能无法进行调试，请在下面找到完整的代码。模块graph和graphfile只是我编写的GitHub中的小程序库。如果您希望运行代码，则可以使用上述GitHub存储库中data/目录中的任何文件。使用F2可以更好地进行简短测试，但是F1和F3会在HPC中造成麻烦。

import graphfile
import graph
from multiprocessing.pool import Pool
import datetime
import logging


def remove_i_and_f(edges):
    new_edges = dict()
    for k,v in edges.items():
        if 'i' in k:
            continue
        elif 'f' in k:
            key = (k[0],k[0])
            new_edges[key] = v
        else:
            new_edges[k] = v
    return new_edges



if __name__ == "__main__":
    import sys

    # Read data
    # =========
    graph_to_study = sys.argv[1]
    full_path = "/ComplexNetworkEntropy/"
    file = graphfile.GraphFile(full_path + "data/" + graph_to_study + ".txt")
    edges = file.read_edges_from_file()

    # logging
    # =======
    d = datetime.date.today().strftime("%Y_%m_%d")
    log_filename = full_path + "results/" + d + "_probabilities_log_" + graph_to_study + ".log"
    logging.basicConfig(filename=log_filename, level=logging.INFO, format='%(asctime)s === %(message)s')
    logging.info("Graph to study: %s", graph_to_study)
    logging.info("Date: %s", d)

    # Process data
    # ==============
    edges = remove_i_and_f(edges)
    g = graph.Graph(edges)

    # Parallel Entropy Calculation
    # ============================
    node_combinations = [(i, j) for i in g.nodes for j in g.nodes]
    pool = Pool(maxtasksperchild=1)
    start = datetime.datetime.now()
    logging.info("Start time: %s", start)
    print("Start time: ", start)
    results = pool.starmap(g._log_probability_path_ij, node_combinations, chunksize=1)
    end = datetime.datetime.now()
    print("End time: ", end)
    print("Run time: ", end - start)
    logging.info("End time: %s", end)
    logging.info("Total run time: %s", start)
    pool.close()
    pool.join()