Python之路第十七篇:Python并发编程|协程

Posted 2022-09-24 hackerer

一、协程

协程，又叫微线程，纤程。英文名Coroutine。协程本质上就是一个线程

优点1：协程极高的执行效率。因为子程序切换不是线程切换，而是由程序自身控制，因此，没有线程切换的开销，和多线程比，线程数量越多，协程的性能优势就越来越明显。(简单来说没有切换的消耗)

优点2：不需要多线程的锁机制，因为只有一个线程，也不存在同时写变量冲突，在协程中控制共享资源不加锁，只需要判断状态就好，所以执行效率比多线程高很多。(没有锁的概念)

因为协程是一个线程执行，那怎么利用多核CPU呢？最简单的方法是多进程+协程，既充分利用多核，又充分发挥协程的高效率，可获得极高的性能。

二、yield的简单实现

#_*_ coding:utf-8 _*_
# Author:Simon
# Datetime:2019/9/2 21:07
# Software:PyCharm

import time
import queue

def consumer(name):
    print("--->ready to eat baozi...")
    while True:
        new_baozi = yield
        print("[%s] is eating baozi %s" % (name,new_baozi))
        #time.sleep(1)

def producer():

    r = con.__next__()
    r = con2.__next__()
    n = 0
    while 1:
        time.sleep(1)
        print("\033[32;1m[producer]\033[0m is making baozi %s and %s" %(n,n+1) )
        con.send(n)
        con2.send(n+1)

        n +=2

if __name__ == ‘__main__‘:
    con = consumer("c1")
    con2 = consumer("c2")
    p = producer()

三、Greenlet模块

greenlet是一个用C实现的协程模块，相比与python自带的yield，它可以使你在任意函数之间随意切换，而不需把这个函数先声明为generator

#_*_ coding:utf-8 _*_
# Author:Simon
# Datetime:2019/9/2 21:29
# Software:PyCharm
from greenlet import greenlet

def test1():
    print(12)
    gr2.switch()
    print(34)
    gr2.switch()
def test2():
    print(56)
    gr1.switch()
    print(78)

gr1 = greenlet(test1)
gr2 = greenlet(test2)
gr1.switch()

四、Gevent模块

本质就是封装了greenlet模块，它能检测I/O并且遇到I/O自动切换到另外一个任务执行；可以帮我们提升效率

#_*_ coding:utf-8 _*_
# Author:Simon
# Datetime:2019/9/2 21:39
# Software:PyCharm

import gevent

import requests,time

start=time.time()

def f(url):
    print(‘GET: %s‘ % url)
    resp =requests.get(url)
    data = resp.text
    print(‘%d bytes received from %s.‘ % (len(data), url))

gevent.joinall([

        gevent.spawn(f, ‘https://www.jd.org/‘),
        gevent.spawn(f, ‘https://www.xiaohua.com/‘),
        gevent.spawn(f, ‘https://www.baidu.com/‘),
        gevent.spawn(f, ‘https://www.taobao.com/‘),

])

# f(‘https://www.jd.org/‘)
# f(‘https://www.xiaohua.com/‘)
# f(‘https://baidu.com/‘)
# f(‘https://www.taobao.com/‘)
# 
# print("cost time:",time.time()-start)

gevent异步提交任务

from gevent import monkey;monkey.patch_all()
import gevent
import time

def eat(name):
    print(‘%s eat 1‘ % name)
    time.sleep(3)
    print(‘%s eat 2‘ % name)

def play(name):
    print(‘%s play 1‘ % name)
    time.sleep(4)
    print(‘%s play 2‘ % name)

g1=gevent.spawn(eat,‘simon‘)
g2=gevent.spawn(play,‘zhuzhu‘)

# time.sleep(5)

# g1.join()
# g2.join()

gevent.joinall([g1,g2]) #相当于上边两行代码

打印：
simon eat 1
zhuzhu play 1
simon eat 2
zhuzhu play 2

基于gevent模块实现并发的套接字通信

单线程、多任务的I/O操作。

#基于gevent实现
from gevent import monkey,spawn;monkey.patch_all()
from socket import *

def communicate(conn):
    while True:
        try:
            data=conn.recv(1024)
            if not data:break
            conn.send(data.upper())
        except ConnectionResetError:
            break
    conn.close()

def server(ip,port):
    server = socket(AF_INET, SOCK_STREAM)
    server.bind((ip,port))
    server.listen(5)

    while True:
        conn, addr = server.accept()
        spawn(communicate,conn) #造一个协程对象，提交完这个对象它不会执行
    server.close()

if __name__ == ‘__main__‘:
    g=spawn(server,‘127.0.0.1‘,8090)
    g.join()

##客户端

from socket import *
from threading import Thread,currentThread

def client():
    client=socket(AF_INET,SOCK_STREAM)
    client.connect((‘127.0.0.1‘,8090))

    while True:
        client.send((‘%s hello‘ %currentThread().getName()).encode(‘utf-8‘))
        data=client.recv(1024)
        print(data.decode(‘utf-8‘))
    client.close()
if __name__ == ‘__main__‘:
    for i in range(500):
        t=Thread(target=client)
        t.start()