python基础===一行 Python 代码实现并行（转）

Posted 2021-02-09 botoo

原文：https://medium.com/building-things-on-the-internet/40e9b2b36148

译文：https://segmentfault.com/a/1190000000414339 

 

#from multiprocessing import Pool
from multiprocessing.dummy import Pool as ThreadPool
import requests

urls = [
    ‘http://www.python.org‘, 
    ‘http://www.python.org/about/‘,
    ‘http://www.onlamp.com/pub/a/python/2003/04/17/metaclasses.html‘,
    ‘http://www.python.org/doc/‘,
    ‘http://www.python.org/download/‘,
    ‘http://www.python.org/getit/‘,
    ‘http://www.python.org/community/‘,
    ‘https://wiki.python.org/moin/‘,
    ‘http://planet.python.org/‘,
    ‘https://wiki.python.org/moin/LocalUserGroups‘,
    ‘http://www.python.org/psf/‘,
    ‘http://docs.python.org/devguide/‘,
    ‘http://www.python.org/community/awards/‘,
    ‘http://www.python.org/community/fuck/‘
    # etc.. 
    ]

pool = ThreadPool(4)   # Sets the pool size to 4


def code(x):
    r = requests.get(x)
    _code = r.status_code
    print(f"{x}     状态码是:{_code}")
results = pool.map(code, urls)


pool.close()
pool.join()

 

在生产环境中，我们可以为 CPU 密集型任务和 IO 密集型任务分别选择多进程和多线程库来进一步提高执行速度——这也是解决死锁问题的良方。此外，由于 map 函数并不支持手动线程管理，反而使得相关的 debug 工作也变得异常简单。