浅析Python多线程

Posted 2020-08-14

　　今天看了几篇博客，主要讲解线程的实例以及如何避免线程间的竞争，觉得感觉对自己很有用，所以在此先写先来以备以后自己查阅.

　　实例一：我们将要请求三个不同的url

1.单线程：

 1 import time
 2 from urllib.request import urlopen
 3 
 4 
 5 def get_responses():
 6      urls = [
 7         ‘http://www.baidu.com‘,
 8                 ‘http://www.taobao.com‘,
 9                 ‘http://www.alibaba.com‘,
10     ]
11     start = time.time()
12     for url in urls:
13         print(url)
14         resp = urlopen(url)
15         print(resp.getcode())   #得到状态码
16     print("spent time:%s" % (time.time()-start))
17 
18 get_responses()

解释：
url顺序的被请求
除非cpu从一个url获得了回应，否则不会去请求下一个url
网络请求会花费较长的时间，所以cpu在等待网络请求的返回时间内一直处于闲置状态。

输出为：
http://www.baidu.com
200
http://www.taobao.com
200
http://www.alibaba.com
200
spent time:1.1927924156188965

2.多线程：

from urllib.request import urlopen
import time
from threading import Thread


class GetUrlThread(Thread):
    def __init__(self, url):
        self.url = url
        super(GetUrlThread, self).__init__()
    def run(self):
        resp = urlopen(self.url)
        print(self.url, resp.getcode())


def get_responses():
    urls = [
        ‘http://www.baidu.com‘,
        ‘http://www.taobao.com‘,
        ‘http://www.alibaba.com‘,
    ]
    start = time.time()
    threads = []
    for url in urls:
        t = GetUrlThread(url)
        threads.append(t)
        t.start()
    for t in threads:
        t.join()
    print("spent time:%s" % (time.time()-start))

get_responses()

解释：
意识到了程序在执行时间上的提升
我们写了一个多线程程序来减少cpu的等待时间，当我们在等待一个线程内的网络请求返回时，这时cpu可以切换到其他线程去进行其他线程内的网络请求。
我们期望一个线程处理一个url，所以实例化线程类的时候我们传了一个url。
线程运行意味着执行类里的run()方法。
无论如何我们想每个线程必须执行run()。
为每个url创建一个线程并且调用start()方法，这告诉了cpu可以执行线程中的run()方法了。
我们希望所有的线程执行完毕的时候再计算花费的时间，所以调用了join()方法。
join()可以通知主线程等待这个线程结束后，才可以执行下一条指令。
每个线程我们都调用了join()方法，所以我们是在所有线程执行完毕后计算的运行时间。
关于线程：
cpu可能不会在调用start()后马上执行run()方法。
你不能确定run()在不同线程建间的执行顺序。
对于单独的一个线程，可以保证run()方法里的语句是按照顺序执行的。
这就是因为线程内的url会首先被请求，然后打印出返回的结果。

输出为：
http://www.baidu.com 200
http://www.alibaba.com 200
http://www.taobao.com 200
spent time:0.6294200420379639

实例二：全局变量的线程安全问题（race condition）

1.BUG版：

from threading import Thread
import time

#define a global variable
some_var = 0

class IncrementThread(Thread):
    def run(self):
        # we want to read a global variable
        # and then increment it
        global some_var
        read_var = some_var
        print("some_var in %s is %d" % (self.name, read_var))
        time.sleep(0.1)
        some_var = read_var + 1
        print("some_var in %s is %d" % (self.name, some_var))


def use_increment_thread():
    threads = []
    for i in range(50):
        t = IncrementThread()
        threads.append(t)
        t.start()
    for t in threads:
        t.join()
    print("After 50 modifications, some_var should have become 50")
    print("After 50 modifications, some_var is %d" % some_var)

use_increment_thread()

解释： 

有一个全局变量，所有的线程都想修改它。
所有的线程应该在这个全局变量上加 1 。
有50个线程，最后这个数值应该变成50，但是它却没有。
为什么没有达到50？
在some_var是15的时候，线程t1读取了some_var，这个时刻cpu将控制权给了另一个线程t2。
t2线程读到的some_var也是15
t1和t2都把some_var加到16
当时我们期望的是t1 t2两个线程使some_var + 2变成17
在这里就有了资源竞争。
相同的情况也可能发生在其它的线程间，所以出现了最后的结果小于50的情况。
输出为：
some_var in Thread-1 is 0
some_var in Thread-2 is 0
some_var in Thread-3 is 0
some_var in Thread-4 is 0
some_var in Thread-5 is 0
some_var in Thread-6 is 0
some_var in Thread-7 is 0
some_var in Thread-8 is 0
some_var in Thread-9 is 0
some_var in Thread-10 is 0
some_var in Thread-11 is 0
some_var in Thread-12 is 0
some_var in Thread-13 is 0
some_var in Thread-14 is 0
some_var in Thread-15 is 0
some_var in Thread-16 is 0
some_var in Thread-17 is 0
some_var in Thread-18 is 0
some_var in Thread-19 is 0
some_var in Thread-20 is 0
some_var in Thread-21 is 0
some_var in Thread-22 is 0
some_var in Thread-23 is 0
some_var in Thread-24 is 0
some_var in Thread-25 is 0
some_var in Thread-26 is 0
some_var in Thread-27 is 0
some_var in Thread-28 is 0
some_var in Thread-29 is 0
some_var in Thread-30 is 0
some_var in Thread-31 is 0
some_var in Thread-32 is 0
some_var in Thread-33 is 0
some_var in Thread-34 is 0
some_var in Thread-35 is 0
some_var in Thread-36 is 0
some_var in Thread-37 is 0
some_var in Thread-38 is 0
some_var in Thread-39 is 0
some_var in Thread-40 is 0
some_var in Thread-41 is 0
some_var in Thread-42 is 0
some_var in Thread-43 is 0
some_var in Thread-44 is 0
some_var in Thread-45 is 0
some_var in Thread-46 is 0
some_var in Thread-47 is 0
some_var in Thread-48 is 0
some_var in Thread-49 is 0
some_var in Thread-50 is 0
some_var in Thread-6 is 1
some_var in Thread-5 is 1
some_var in Thread-2 is 1
some_var in Thread-4 is 1
some_var in Thread-1 is 1
some_var in Thread-3 is 1
some_var in Thread-12 is 1
some_var in Thread-13 is 1
some_var in Thread-11 is 1
some_var in Thread-10 is 1
some_var in Thread-9 is 1
some_var in Thread-7 is 1
some_var in Thread-8 is 1
some_var in Thread-21 is 1
some_var in Thread-20 is 1
some_var in Thread-19 is 1
some_var in Thread-18 is 1
some_var in Thread-17 is 1
some_var in Thread-15 is 1
some_var in Thread-14 is 1
some_var in Thread-16 is 1
some_var in Thread-26 is 1
some_var in Thread-25 is 1
some_var in Thread-24 is 1
some_var in Thread-22 is 1
some_var in Thread-23 is 1
some_var in Thread-31 is 1
some_var in Thread-29 is 1
some_var in Thread-28 is 1
some_var in Thread-27 is 1
some_var in Thread-30 is 1
some_var in Thread-38 is 1
some_var in Thread-37 is 1
some_var in Thread-36 is 1
some_var in Thread-35 is 1
some_var in Thread-32 is 1
some_var in Thread-33 is 1
some_var in Thread-34 is 1
some_var in Thread-44 is 1
some_var in Thread-43 is 1
some_var in Thread-42 is 1
some_var in Thread-41 is 1
some_var in Thread-40 is 1
some_var in Thread-39 is 1
some_var in Thread-50 is 1
some_var in Thread-49 is 1
some_var in Thread-48 is 1
some_var in Thread-47 is 1
some_var in Thread-45 is 1
some_var in Thread-46 is 1
After 50 modifications, some_var should have become 50
After 50 modifications, some_var is 1

解决竞争带锁版：

 1 from threading import Lock, Thread
 2 import time
 3 lock = Lock()
 4 some_var = 0
 5 
 6 class IncrementThread(Thread):
 7     def run(self):
 8         #we want to read a global variable
 9         #and then increment it
10         global some_var
11         lock.acquire()
12         read_value = some_var
13         print("some_var in %s is %d" % (self.name, read_value))
14         time.sleep(0.1)
15         some_var = read_value + 1
16         print("some_var in %s after increment is %d" % (self.name, some_var))
17         lock.release()
18 
19 def use_increment_thread():
20     threads = []
21     for i in range(50):
22         t = IncrementThread()
23         threads.append(t)
24         t.start()
25     for t in threads:
26         t.join()
27     print("After 50 modifications, some_var should have become 50")
28     print("After 50 modifications, some_var is %d" % (some_var,))
29 
30 use_increment_thread()

解释： 

Lock 用来防止竞争条件
如果在执行一些操作之前，线程t1获得了锁。其他的线程在t1释放Lock之前，不会执行相同的操作
我们想要确定的是一旦线程t1已经读取了some_var，直到t1完成了修改some_var，其他的线程才可以读取some_var
这样读取和修改some_var成了逻辑上的原子操作。
输出为：
some_var in Thread-1 is 0
some_var in Thread-1 after increment is 1
some_var in Thread-2 is 1
some_var in Thread-2 after increment is 2
some_var in Thread-3 is 2
some_var in Thread-3 after increment is 3
some_var in Thread-4 is 3
some_var in Thread-4 after increment is 4
some_var in Thread-5 is 4
some_var in Thread-5 after increment is 5
some_var in Thread-6 is 5
some_var in Thread-6 after increment is 6
some_var in Thread-7 is 6
some_var in Thread-7 after increment is 7
some_var in Thread-8 is 7
some_var in Thread-8 after increment is 8
some_var in Thread-9 is 8
some_var in Thread-9 after increment is 9
some_var in Thread-10 is 9
some_var in Thread-10 after increment is 10
some_var in Thread-11 is 10
some_var in Thread-11 after increment is 11
some_var in Thread-12 is 11
some_var in Thread-12 after increment is 12
some_var in Thread-13 is 12
some_var in Thread-13 after increment is 13
some_var in Thread-14 is 13
some_var in Thread-14 after increment is 14
some_var in Thread-15 is 14
some_var in Thread-15 after increment is 15
some_var in Thread-16 is 15
some_var in Thread-16 after increment is 16
some_var in Thread-17 is 16
some_var in Thread-17 after increment is 17
some_var in Thread-18 is 17
some_var in Thread-18 after increment is 18
some_var in Thread-19 is 18
some_var in Thread-19 after increment is 19
some_var in Thread-20 is 19
some_var in Thread-20 after increment is 20
some_var in Thread-21 is 20
some_var in Thread-21 after increment is 21
some_var in Thread-22 is 21
some_var in Thread-22 after increment is 22
some_var in Thread-23 is 22
some_var in Thread-23 after increment is 23
some_var in Thread-24 is 23
some_var in Thread-24 after increment is 24
some_var in Thread-25 is 24
some_var in Thread-25 after increment is 25
some_var in Thread-26 is 25
some_var in Thread-26 after increment is 26
some_var in Thread-27 is 26
some_var in Thread-27 after increment is 27
some_var in Thread-28 is 27
some_var in Thread-28 after increment is 28
some_var in Thread-29 is 28
some_var in Thread-29 after increment is 29
some_var in Thread-30 is 29
some_var in Thread-30 after increment is 30
some_var in Thread-31 is 30
some_var in Thread-31 after increment is 31
some_var in Thread-32 is 31
some_var in Thread-32 after increment is 32
some_var in Thread-33 is 32
some_var in Thread-33 after increment is 33
some_var in Thread-34 is 33
some_var in Thread-34 after increment is 34
some_var in Thread-35 is 34
some_var in Thread-35 after increment is 35
some_var in Thread-36 is 35
some_var in Thread-36 after increment is 36
some_var in Thread-37 is 36
some_var in Thread-37 after increment is 37
some_var in Thread-38 is 37
some_var in Thread-38 after increment is 38
some_var in Thread-39 is 38
some_var in Thread-39 after increment is 39
some_var in Thread-40 is 39
some_var in Thread-40 after increment is 40
some_var in Thread-41 is 40
some_var in Thread-41 after increment is 41
some_var in Thread-42 is 41
some_var in Thread-42 after increment is 42
some_var in Thread-43 is 42
some_var in Thread-43 after increment is 43
some_var in Thread-44 is 43
some_var in Thread-44 after increment is 44
some_var in Thread-45 is 44
some_var in Thread-45 after increment is 45
some_var in Thread-46 is 45
some_var in Thread-46 after increment is 46
some_var in Thread-47 is 46
some_var in Thread-47 after increment is 47
some_var in Thread-48 is 47
some_var in Thread-48 after increment is 48
some_var in Thread-49 is 48
some_var in Thread-49 after increment is 49
some_var in Thread-50 is 49
some_var in Thread-50 after increment is 50
After 50 modifications, some_var should have become 50
After 50 modifications, some_var is 50

实例三：多线程环境下的原子操作

BUG版本：

 1 from threading import Thread
 2 import time
 3 
 4 class CreateListThread(Thread):
 5     def run(self):
 6         self.entries = []
 7         for i in range(10):
 8             # time.sleep(0.1)
 9             self.entries.append(i)
10         for each in self.entries:
11             print(each, end = " ")
12             time.sleep(0.1)
13 
14 def use_create_list_thread():
15     for i in range(3):
16         t = CreateListThread()
17         t.start()
18 
19 use_create_list_thread()

 

 

解释：
当一个线程正在打印的时候，cpu切换到了另一个线程，所以产生了不正确的结果。我们需要确保print self.entries是个逻辑上的原子操作，以防打印时被其他线程打断。
因为打印的速度太快，我在此有意放大了这个时间，加了一个time.sleep(0.1)
输出为：
0 0 0 1 1 1 2 2 2 3 3 3 4 4 4 5 5 5 6 6 6 7 7 7 8 8 8 9 9 9

2.加锁保证操作的原子性

 1 from threading import Thread, Lock
 2 import time
 3 
 4 lock = Lock()
 5 
 6 
 7 class CreateListThread(Thread):
 8     def run(self):
 9         self.entries = []
10         for i in range(10):
11             time.sleep(0.1)
12             self.entries.append(i)
13         lock.acquire()
14         for each in self.entries:
15             print(each, end = " ")
16             time.sleep(0.1)
17         lock.release()
18 
19 
20 def use_create_list_thread():
21     for i in range(3):
22         t = CreateListThread()
23         t.start()
24 
25 use_create_list_thread()

输出为：
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9