利用BlockingCollection实现生产者和消费者队列,实现写文本

Posted 梦里的畅泳

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了利用BlockingCollection实现生产者和消费者队列,实现写文本相关的知识,希望对你有一定的参考价值。

  最近开发几个小项目,需要把结果写到txt文件里面,并且按照时间进行分文件,由于对于效率要求较高,所以采用 生产者和消费者 模型来进行写出文本,线程中只需要添加队列就立即返回,而不需要等待写文件的时间

public class WriteItem : IDisposable
    {
        public string Filename { get; }
        public Encoding Encode { get; }
        public bool Append { get; }
        public bool TimeName { get; }

        private StreamWriter _writer;

        private readonly BlockingCollection<string> _blocking = new BlockingCollection<string>();

        public void Write(string msg)
        {
            _blocking.Add(msg);
        }

        public void WriteLine(string msg)
        {
            Write(msg + Environment.NewLine);
        }

        public WriteItem(string filename, Encoding encode, bool append = true, bool timeName = false)
        {
            Filename = filename;
            Encode = encode;
            Append = append;
            TimeName = timeName;

            if (timeName && string.IsNullOrEmpty(Path.GetExtension(filename)))
            {
                this.Filename = Path.Combine(this.Filename, DateTime.Now.ToString("yyyy-MM-dd") + ".txt");
            }
            var dir = Path.GetDirectoryName(this.Filename);
            Directory.CreateDirectory(dir ?? throw new InvalidOperationException());
            _writer = new StreamWriter(this.Filename, this.Append, this.Encode) { AutoFlush = true };
            Task.Factory.StartNew(() =>
            {
                foreach (var s in _blocking.GetConsumingEnumerable())
                {
                    if (TimeName)
                    {
                        var fileNameWithoutExtension = Path.GetFileNameWithoutExtension(filename);
                        var nowDay = DateTime.Now.ToString("yyyy-MM-dd").ToString();
                        if (fileNameWithoutExtension != nowDay)
                        {
                            _writer.Dispose();
                            _writer = new StreamWriter(this.Filename, this.Append, this.Encode) { AutoFlush = true };
                        }
                    }
                    _writer.Write(s);
                }

            }, TaskCreationOptions.LongRunning);
        }

        public void Dispose()
        {
            _writer?.Dispose();
            _blocking?.Dispose();
        }
    }

  然后再写了个字典来维护:

 public class FileWriteQueue
    {
        private static readonly Dictionary<string, WriteItem> Dictionary = new Dictionary<string, WriteItem>();

        public static void AddOrUpdate(string key, WriteItem item)
        {
            if (Dictionary.ContainsKey(key))
            {
                Dictionary[key].Dispose();
                Dictionary[key] = item;
            }
            else
            {
                Dictionary.Add(key, item);
            }
        }

        public static WriteItem Get(string key)
        {
            return Dictionary[key];
        }
    }

  在实际使用添加WirteItem,设置好输出目录就行了:

            FileWriteQueue.AddOrUpdate("success",new WriteItem(Path.Combine("结果","成功"),Encoding.Default,true,true));
            FileWriteQueue.AddOrUpdate("error", new WriteItem(Path.Combine("结果", "失败"), Encoding.Default, true, true));
            for (int i = 0; i < 1000; i++)
            {
                FileWriteQueue.Get("success").WriteLine(i.ToString());
            }

 

以上是关于利用BlockingCollection实现生产者和消费者队列,实现写文本的主要内容,如果未能解决你的问题,请参考以下文章

csharp .NET 4 BlockingCollection的生产者/消费者实现

如何加快大块 BlockingCollection 的实现

异步简析之BlockingCollection实现生产消费模式

使用blockingcollection和tasks .net 4 TPL的经典生产者消费者模式

BlockingCollection 中的多个消费者是不是同时处理?

BlockingCollection实现单体程序内队列