python 使用多处理模块通过多个进程执行功能的玩具示例。

Posted

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了python 使用多处理模块通过多个进程执行功能的玩具示例。相关的知识,希望对你有一定的参考价值。

# Demo how multiprocessing works when using classes.

"""
It's convenient to wrap complex functionality in an class, but the 
multiprocessing module executes functions. Thus, we create a function 
that employs a convenience class.

Note: The multiprocessing module makes use of pickling. Objects in a 
module must be visible at the top level of the module to be pickled. 
In other words, we cannot write functions within functions, etc.

https://docs.python.org/3/library/pickle.html#pickle-picklable
"""

import multiprocessing as mp


class LengthComputer(object):
    """
    Example class wrapping functionality.
    """
    def __init__(self, s):
        """
        Create instance: 's' is a string.
        """
        object.__init__(self)
        self.s = s

    def writeLengthToFile(self):
        """
        Write length of 'self.s' to a file named <self.s>.txt.
        """
        length = len(self.s)
        with open(self.s + '.txt', 'w') as f:
            # Convert to string or we fail to write out the value!
            f.write(str(length))
            f.flush()


def outputLengthOfString(s):
    """
    Create a file named 's'.txt, the contents being the length of 's'.
    """
    comp = LengthComputer(s)
    comp.writeLengthToFile()


if __name__ == '__main__':
    """
    Take some sample strings, and for each create a file named after the 
    string, with the string's length as the contents of the file. Use multiple 
    processes to do so.
    """
    strings = 'one two three four'.split()
    pool = mp.Pool()
    for s in strings:
        # We can omit the 'callback' keyword if we're not using a callback function.
        pool.apply_async(outputLengthOfString, args=(s,))
    pool.close()
    pool.join()

以上是关于python 使用多处理模块通过多个进程执行功能的玩具示例。的主要内容,如果未能解决你的问题,请参考以下文章

python_多线程多进程

如何使用 Python 多处理和 memory_profiler 分析多个子进程?

Python多进程multiprocessing模块介绍

多进程编程

Python 多进程教程

python多进程