python基础===多进程

Posted 2020-11-03 botoo

进程线程的区别在进程，线程，协程的区别

linux或者unix有fork()函数，但是不支持win系统。

multiprocessing

multiprocessing模块是跨平台版本的多进程模块。支持win系统,用法如下：

from multiprocessing import Process
import os

# 子进程要执行的代码
def run_proc(name):
    print(\'Run child process %s (%s)...\' % (name, os.getpid()))

if __name__==\'__main__\':
    print(\'Parent process %s.\' % os.getpid())
    p = Process(target=run_proc, args=(\'test\',))
    print(\'Child process will start.\')
    p.start()
    p.join()
    print(\'Child process end.\')


>>>
Parent process 9860.
Child process will start.
Run child process test (9764)...
Child process end.

*该py文件是通过cmd窗口执行，不然执行不了多进程

创建子进程时，只需要传入一个执行函数和函数的参数，创建一个Process实例，用start()方法启动，这样创建进程比fork()还要简单。

join()方法可以等待子进程结束后再继续往下运行，通常用于进程间的同步。

 

Pool

如果要启动大量的子进程，可以用进程池的方式批量创建子进程：

from multiprocessing import Pool
import os, time, random


def long_time_task(name):
    print("Run task %s (%s)...." % (name, os.getpid()))
    start = time.time()
    time.sleep(random.random()*3)
    end = time.time()
    print("Task %s runs %0.2f seconds." %(name, (end - start)))


if __name__ == "__main__":
    print("Parent process %s." %os.getpid())
    p = Pool(4)                         #允许运行多少个进程
    for i in range(5):
        p.apply_async(long_time_task, args = (i, ))
    print("waiting for all subprocesses done....")
    p.close()
    p.join()
    print("All  subprocesses done.")

>>>
Parent process 10852.
waiting for all subprocesses done....
Run task 0 (9620)....
Run task 1 (10180)....
Run task 2 (8116)....
Task 2 runs 0.03 seconds.
Run task 3 (8116)....
Run task 4 (8744)....
Task 4 runs 0.42 seconds.
Task 0 runs 0.64 seconds.
Task 1 runs 1.15 seconds.
Task 3 runs 2.90 seconds.
All  subprocesses done.

 

 

 

 

 

 

参考资料：

https://www.liaoxuefeng.com/wiki/0014316089557264a6b348958f449949df42a6d3a2e542c000/001431927781401bb47ccf187b24c3b955157bb12c5882d000