python 生产者消费者线程模型

Posted 2020-06-29

python 多线程生产者消费者模型：

一个生产者多个消费者

The Queue module implements multi-producer, multi-consumer queues. It is especially useful in threaded programming when information must be exchanged safely between multiple threads. The Queue class in this module implements all the required locking semantics. It depends on the availability of thread support in Python; see the threading module.

Queue模块实现了多生产者、多消费者的队列，它在多线程编程中多线程的数据安全交换尤其有用，在模块中Queue类实现了所需的锁原语。在python中依赖于可用的多线程支持。

在本例中原本是使用条件变量来同步生产者消费者之间的队列信息，结果出问题了，查看文档后说Since the Queue has a Condition and that condition has its Lock we don‘t need to bother about Condition and Lock，翻译过来就是：因为队列有自己的条件对象，该对象有它自己的锁，我们不必担心条件对象和锁之间的处理。下面是处理代码

import logging

from threading import Thread, Condition, Lock

from Queue import Queue

from time import sleep

logger = logging.getLogger(‘mglottery.log‘)

sleep_time = 5

MAX_LENGTH = 16

THREAD_COUNT = 4

queue = Queue(MAX_LENGTH)

class ProducerThread(Thread):

    def __init__(self, name="producer"):

        print name

        super(ProducerThread, self).__init__(name=name)

    def add_task(self, start, end, poison=False):

        for index in xrange(start, end):

            goods = "can %d" % index if poison is False else None

            queue.put(goods)

            print ‘product one thing‘

    def run(self):

        global queue

        while 1:

            self.add_task(0, 100)

            #send poison pill

            self.add_task(0, THREAD_COUNT, True)

            break

class ConsumerThread(Thread):

    def __init__(self, name):

        print name

        super(ConsumerThread, self).__init__(name=name)

    def run(self):

        global queue

        times = 0

        while True:

            goods = queue.get()

            if goods is None:

                print "consumer receive a poison pill, thread will exit"

                break

            else:

                print ‘thread %s get job, queue size:%d ,goods:%s‘ % (self.getName(), queue.qsize(), goods)

if __name__=="__main__":

    producer = ProducerThread(name="Can producer")

    consumers = [ConsumerThread("Can consumer %d"%i) for i in range(THREAD_COUNT)]

    producer.start()

    for consumer in consumers:

        consumer.start()

    for consumer in consumers:

        consumer.join()

    producer.join()

本文出自 “彬临城下” 博客，请务必保留此出处http://cbotz.blog.51cto.com/4039348/1755495