为啥模拟的 Parallel 函数在第一次运行时执行缓慢但随后加速?
Posted
技术标签:
【中文标题】为啥模拟的 Parallel 函数在第一次运行时执行缓慢但随后加速?【英文标题】:Why impersonated Parallel function performs slow on the first run but speeds up subsequently?为什么模拟的 Parallel 函数在第一次运行时执行缓慢但随后加速? 【发布时间】:2019-02-27 13:38:15 【问题描述】:所以我有一些代码可以将文件复制到网络上的 5 台远程 PC。我有一个模拟需要登录的方法的类
''' <summary>
''' Returns the names of the files including their path that match the regex pattern found in this directory.
''' </summary>
''' <param name="searchOption">
''' The parameter option SearchOption.AllDirectories indicates that the search should include the current directory and any subdirectories
''' The parameter option SearchOption.TopDirectoryOnly indicates that the search should only include the current directory.
''' </param>
Public Function GetFileContent(filePath As String, searchPattern As String, Optional searchOption As SearchOption = Nothing) As String() Implements IEPMADirectoryAccess.GetFileContent
If searchOption = Nothing Then
searchOption = SearchOption.TopDirectoryOnly
End If
LogonUser(UserName, Domain, Password, LogonType.LOGON32_LOGON_NEW_CREDENTIALS, 0, UserToken)
Using WindowsIdentity.Impersonate(UserToken)
Return Directory.GetFiles(filePath, searchPattern, searchOption)
End Using
End Function
''' <summary>
''' Get the name of the Ward from the directory of the current file path
''' </summary>
Public Function GetWardName(filePath As String) As String Implements IEPMADirectoryAccess.GetWardName
LogonUser(UserName, Domain, Password, LogonType.LOGON32_LOGON_NEW_CREDENTIALS, 0, UserToken)
Using WindowsIdentity.Impersonate(UserToken)
Return New DirectoryInfo(Path.GetDirectoryName(filePath)).Name
End Using
End Function
''' <summary>
''' Creates a directory for the given path
''' </summary>
Public Sub CreateDirectory(directoryPath As String) Implements IEPMADirectoryAccess.CreateDirectory
LogonUser(UserName, Domain, Password, LogonType.LOGON32_LOGON_NEW_CREDENTIALS, 0, UserToken)
Using WindowsIdentity.Impersonate(UserToken)
Directory.CreateDirectory(directoryPath)
End Using
End Sub
''' <summary>
''' Deletes a file from the specified path
''' </summary>
Public Sub DeleteFile(filePath As String) Implements IEPMADirectoryAccess.DeleteFile
LogonUser(UserName, Domain, Password, LogonType.LOGON32_LOGON_NEW_CREDENTIALS, 0, UserToken)
Using WindowsIdentity.Impersonate(UserToken)
File.Delete(filePath)
End Using
End Sub
''' <summary>
''' Copies a file from the source path to the destination path
''' </summary>
Public Sub CopyFile(sourceFilePath As String, destFilePath As String, overwrite As Boolean) Implements IEPMADirectoryAccess.CopyFile
LogonUser(UserName, Domain, Password, LogonType.LOGON32_LOGON_NEW_CREDENTIALS, 0, UserToken)
Using WindowsIdentity.Impersonate(UserToken)
File.Copy(sourceFilePath, destFilePath, overwrite)
End Using
End Sub
并行循环中的部分代码如下所示:
'If PC is switched on
If IsOnline(device.PCName) Then
'For each PC of the ward, get the corresponding folder path
targetPath = "\\" + device.PCName + "\c$Report P\"
'Requires access to the network share
If Not Directory.Exists(targetPath) Then
DirectoryAccess.CreateDirectory(targetPath)
End If
'Requires access to the network share
For Each fileName As String In DirectoryAccess.GetFileContent(targetPath, "*.pdf")
'Purge (Delete) previous versions of this file in the destination folder
Dim fileConst As String
fileConst = arr(1)
If fileName.Contains(fileConst) Then
'Requires access to the network share
DirectoryAccess.DeleteFile(fileName)
End If
Next
DirectoryAccess.CopyFile(f, Path.Combine(targetPath, Path.GetFileName(f), True)
End If
在主模块中,我正在计时使用 普通 for 循环与并行 for 循环相比,但 for 循环中的代码内容相同:
enter image description here
最初,并行复制为 18 秒,但第二次运行时变为 2.82 秒。
enter image description here
第三次运行是 1.81 秒。
我检查了用于模拟的 using 块已正确处理并在退出 using 块后恢复为我的原始凭据。我不知道为什么它在第一次运行时很慢,但后来加速了?会不会是一开始四台电脑都是不活动的,所以复制的过程会激活它们,但需要更长的时间?
我有一个 IsOnline 函数来检查 PC 是否通电,如果没有,我就跳过它们以确保它不会减慢进程。
enter image description here
【问题讨论】:
【参考方案1】:事实证明它很慢,因为时间是在任何时候在不同数量的电脑上测量的。一些电脑在测量期间进入睡眠状态,因此影响了处理时间
【讨论】:
以上是关于为啥模拟的 Parallel 函数在第一次运行时执行缓慢但随后加速?的主要内容,如果未能解决你的问题,请参考以下文章
为啥在使用 joblib.Parallel 时保护主循环很重要?
GNU Parallel 和 Bash 函数:如何运行手册中的简单示例
为啥覆盖 Parallel.foreach 循环的 .NET 单元测试依赖于硬件?