.NET - 阻塞主线程,直到有任何可用线程
Posted
技术标签:
【中文标题】.NET - 阻塞主线程,直到有任何可用线程【英文标题】:.NET - Block main thread until there are any available threads 【发布时间】:2013-04-19 01:44:27 【问题描述】:我有一个进程,我的主线程正在读取文件并将其拆分为多个部分。然后这些部分需要进一步处理。我想利用任何可用的线程,以便下游处理使用尽可能多的 CPU(或尽可能多的内核)。我不想从主线程创建过多的积压,所以我需要主线程等待添加到队列中,直到有另一个可用线程。
我看到很多类似VB.NET 4.0: Looking to execute multiple threads, but wait until all threads are completed before resuming 的文章,但它们都在等待所有线程完成,而我只需要任何线程都可用
这是我可以使用Task Parallel Library 解决的问题,还是我应该手动创建线程并监控线程池?
Using Reader As New StreamReader(FileName)
Do
CurrentBlockSize = Reader.ReadBlock(CurrentBuffer, 0, BufferSize)
RunningBuffer &= New String(CurrentBuffer)
If RunningBuffer.Contains(RowDelimiter) Then
LineParts = RunningBuffer.Split(RowDelimiter)
For I As Integer = 0 To LineParts.Count - 1
If I < LineParts.Count - 1 Then
'Make synchronous call that blocks until'
'another thread is available to process the line'
AddLineToTheProcessingQueue(CurrentLine)
Else
RunningBuffer = LineParts(I)
End If
Next
End If
Loop While CurrentBlockSize = BufferSize
End Using
【问题讨论】:
这几乎肯定是白费力气,这种代码几乎总是受磁盘限制。一个简单的检查:重启你的机器并运行单线程版本。如果一个内核上的 CPU 负载不超过 50%,那么添加更多线程并不能使其更快。 @HansPassant,下游工作将涉及处理大量数据并将其通过 TCP/IP 发送到数据库,这可能会比磁盘读取速度慢。这就是我想要多线程的部分 然后拉,不要推数据。就像大多数程序读取文件一样。 【参考方案1】:将此代码粘贴到新的控制台应用程序中。
Imports System.Threading
Module Module1
' I just picked 6 randomly, not sure what is a good strategy for picking this number
' also, not sure what is the difference between a Worker Thread and a Completion thread
Const MaxWorkerThreads As Integer = 6
Const MaxCompletionPortThreads As Integer = 6
Sub Main()
ThreadPool.SetMaxThreads(MaxWorkerThreads, MaxCompletionPortThreads)
Dim availableWorkerThreads As Integer
Dim availableCompletionPortThreads As Integer
For i As Integer = 0 To 100
' GetAvailableThreads returns results via output parameters
ThreadPool.GetAvailableThreads(availableWorkerThreads, availableCompletionPortThreads)
Dim tries As Integer = 0
Do While (availableWorkerThreads = 0)
' this loop does not execute if there are available threads
' you may want to add a fail-safe to check "tries" in case the child threads get stuck
tries += 1
Console.WriteLine(String.Format("waiting to start item 0, attempt 1, available threads: 2, 3", i, tries, availableWorkerThreads, availableCompletionPortThreads))
' failure to call Sleep will make your program unresponsive
Thread.Sleep(1000)
' call GetAvailableThreads again for the next test at the top of the loop
ThreadPool.GetAvailableThreads(availableWorkerThreads, availableCompletionPortThreads)
Loop
' this is how you pass parameters to a thread created through QueueUserWorkItem
Dim parameters As Object() = i
ThreadPool.QueueUserWorkItem(AddressOf DoWork, parameters)
' According to MSDN, you must Sleep after calling QueueUserWorkItem, or else the current thread will just exit
Thread.Sleep(500)
Next
End Sub
Sub DoWork(parameters As Object())
Dim itemNumber = parameters(0)
Dim sleepLength = itemNumber * 1000
Console.WriteLine(String.Format("Item: 0 - sleeping for 1 miliseconds.", itemNumber, sleepLength))
Thread.Sleep(sleepLength)
Console.WriteLine(String.Format("Item: 0 - done sleeping.", itemNumber))
End Sub
End Module
【讨论】:
【参考方案2】:我不确定您究竟为什么要这样做,但您可以通过使用BlockingCollection
或设置了BoundedCapacity
的数据流块来实现非常相似的效果。
例如,如果您将容量设置为 1 并且您的消费者此时正忙,那么您将无法将第二个项目添加到队列中,直到其中一个消费者完成其当前工作并将该项目从队列。这两个版本都为您提供了一种等待,直到您可以将另一个项目添加到队列中。
【讨论】:
是的,我昨晚开始阅读有关 TPL-Data Flow 和 Bounded Capacity 的内容,非常有希望。我会在一些测试后报告。你有任何有限容量的数据流块的示例代码吗? @TomHalladay 你可以在 Mono 的 TDF 版本中查看 this set of tests。以上是关于.NET - 阻塞主线程,直到有任何可用线程的主要内容,如果未能解决你的问题,请参考以下文章