Powershell - 如何处理非时间戳日志文件条目，包括空行

Posted 2023-04-18

技术标签:

【中文标题】Powershell - 如何处理非时间戳日志文件条目，包括空行【英文标题】：Powershell - how to handle non timestamped log file entries, including blank lines 【发布时间】：2014-09-23 13:51:27 【问题描述】：

我问了一个关于如何根据开始和结束时间戳从日志文件中提取数据的问题，最近我得到了一个很好且快速的答案，有关详细信息，请参阅此问题/帖子。

Powershell - How to select a block of text from a text based log file that has a time stamp in the log file entry

但是，我现在发现我正在尝试实现的目标存在问题，因为我正在处理/查询的日志文件的每一行都没有时间戳条目，实际上有些行是空白的，而且还有跨多行拆分的 XML 条目。

此处显示了一个简短的经过处理的提取物，以尝试显示我正在尝试处理的日志文件数据的概念：

1600 00:06:45 CMD1: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:06:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:06:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:06:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:06:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:06:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:06:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:06:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:06:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:06:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
OUTBOUND XML STRING RESULT OF TRANSFORMATION:

<?xml version="1.0"?>

<TgwGiMessage version="1.0">

    <ShortDataTransmitRequest>

        <SourceITSI SSI="XXXXXXX" EXT="0" CPTI="0" SNA="0"/>

        <DestinationAddress>

            <DestinationITSI SSI="XXXXXXX" EXT="0" CPTI="0" SNA="0"/>

        </DestinationAddress>

        <DeliveryConfirmation>1</DeliveryConfirmation>

        <MessageText>XML Text Message</MessageText>

    </ShortDataTransmitRequest>

</TgwGiMessage>


16e8 00:06:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
2138 00:06:45 01-BASICDT::HS: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:06:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
1e0c 00:06:45 IOM WRITE: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:06:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:06:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:06:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
APD OUTBOUND XML STRING RESULT OF TRANSFORMATION:

上一个问题/帖子为我提供了能够识别和输出日志文件数据所需的解决方案，只要日志文件条目具有时间戳。

如果日志文件条目没有时间戳，我会在脚本尝试处理这些行时收到错误报告：

Index was outside the bounds of the array.
At line:9 char:5
+ if ($parts[1] -ge $StartTime -and $parts[1] -le $EndTime) 
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OperationStopped: (:) [], IndexOutOfRangeException
    + FullyQualifiedErrorId : System.IndexOutOfRangeException

这很明显，因为空白行不能被分割，而那些没有时间戳的，即使它们可以分割也不符合我作为时间戳测试的要求。

有人可以建议我如何阅读日志文件并在两个时间戳之间输出日志文件中的所有条目。

我不是程序员，只是一个管理员黑客，所以我正在努力研究如何解决这个问题。

我非常乐意尝试任何方法，但如果有任何意见可以让我了解如何最好地尝试和解决这个问题，我们将不胜感激。

更新 - PSGuy，在您上次更新之后，我尝试确定为什么在将您的脚本应用于我的实时日志文件时我没有得到所需的结果。

我创建了一个稍长的日志文件示例来运行，ExampleLogfile.log 如下所示，这提供了几行我试图提取日志的确切内容的行，包括 XML 行。

1600 00:06:45 CMD1: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:06:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:06:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:06:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:07:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:07:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:07:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:07:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:08:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:08:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
OUTBOUND XML STRING RESULT OF TRANSFORMATION:

<?xml version="1.0"?>

<TgwGiMessage version="1.0">

    <ShortDataTransmitRequest>

        <SourceITSI SSI="XXXXXXX" EXT="0" CPTI="0" SNA="0"/>

        <DestinationAddress>

            <DestinationITSI SSI="XXXXXXX" EXT="0" CPTI="0" SNA="0"/>

        </DestinationAddress>

        <DeliveryConfirmation>1</DeliveryConfirmation>

        <MessageText>XML Text Message</MessageText>

    </ShortDataTransmitRequest>

</TgwGiMessage>


16e8 00:08:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
2138 00:08:45 01-BASICDT::HS: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:09:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
1e0c 00:09:45 IOM WRITE: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:09:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:09:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:10:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
APD OUTBOUND XML STRING RESULT OF TRANSFORMATION:
1600 00:10:45 CMD1: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:10:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:10:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:11:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:11:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:11:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:11:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:12:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:12:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:12:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
OUTBOUND XML STRING RESULT OF TRANSFORMATION:

<?xml version="1.0"?>

<TgwGiMessage version="1.0">

    <ShortDataTransmitRequest>

        <SourceITSI SSI="XXXXXXX" EXT="0" CPTI="0" SNA="0"/>

        <DestinationAddress>

            <DestinationITSI SSI="XXXXXXX" EXT="0" CPTI="0" SNA="0"/>

        </DestinationAddress>

        <DeliveryConfirmation>1</DeliveryConfirmation>

        <MessageText>XML Text Message</MessageText>

    </ShortDataTransmitRequest>

</TgwGiMessage>


16e8 00:12:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
2138 00:13:45 01-BASICDT::HS: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:13:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
1e0c 00:13:45 IOM WRITE: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:13:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:14:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:14:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
APD OUTBOUND XML STRING RESULT OF TRANSFORMATION:
1600 00:14:45 CMD1: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:14:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:15:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:15:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:15:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:15:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:16:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:16:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:16:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:16:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
OUTBOUND XML STRING RESULT OF TRANSFORMATION:

<?xml version="1.0"?>

<TgwGiMessage version="1.0">

    <ShortDataTransmitRequest>

        <SourceITSI SSI="XXXXXXX" EXT="0" CPTI="0" SNA="0"/>

        <DestinationAddress>

            <DestinationITSI SSI="XXXXXXX" EXT="0" CPTI="0" SNA="0"/>

        </DestinationAddress>

        <DeliveryConfirmation>1</DeliveryConfirmation>

        <MessageText>XML Text Message</MessageText>

    </ShortDataTransmitRequest>

</TgwGiMessage>


16e8 00:17:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
2138 00:17:45 01-BASICDT::HS: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:17:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
1e0c 00:17:45 IOM WRITE: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:18:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:18:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:18:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
APD OUTBOUND XML STRING RESULT OF TRANSFORMATION:

然后我将您的代码编辑如下，从 00:08:45 提取一段 5 分钟的日志

$file = Get-Content "c:\temp\ExampleLogfile.log"

# create your text pattern for regex matches here
$myPattern = "\d1,2\:\d1,2\:\d1,2"

# what time boundaries do you want to check?
$tempTime = "00:08:45"
$lowerBound = [DateTime]::Parse($tempTime)
$upperBound = $lowerBound.AddMinutes(5)
    # tempTime can use any System.DateTime static method once
    # you have a date time, and should ideally be an input parameter
    # lowerBound should also be an input parameter

# this could also be a for, do / while, or for each
# loop through the file until we reach the end
for ($i = 0; $i -le $file.GetUpperBound(""); $i++)

    # this will loop until we reach the last line
    if ($file[$i] -match $myPattern)
    
        $time = $file[$i].Split(' ')[1]
        if ([DateTime]::Parse($time) -ge $lowerBound -and [DateTime]::Parse($time) -le $upperBound)
        
            $file[$i] # this will print to a console
                # using the Tee-Object cmdlet will output it to the console
                # and a file, which could be useful for developers and you

产生以下输出：

16e8 00:08:45 CL: Standard log file entry: Standard log file entry: Standard log file entry:
16e8 00:08:45 CL: Standard log file entry: Standard log file entry: Standard log file entry:
16e8 00:08:45 CL: Standard log file entry: Standard log file entry: Standard log file entry:
2138 00:08:45 01-BASICDT::HS: Standard log file entry: Standard log file entry: Standard log file entry:
16e8 00:09:45 CL: Standard log file entry: Standard log file entry: Standard log file entry:
1e0c 00:09:45 IOM WRITE: Standard log file entry: Standard log file entry: Standard log file entry:
16e8 00:09:45 CL: Standard log file entry: Standard log file entry: Standard log file entry:
16e8 00:09:45 CL: Standard log file entry: Standard log file entry: Standard log file entry:
16e8 00:10:45 CL: Standard log file entry: Standard log file entry: Standard log file entry:
1600 00:10:45 CMD1: Standard log file entry: Standard log file entry: Standard log file entry:
16e8 00:10:45 CL: Standard log file entry: Standard log file entry: Standard log file entry:
16e8 00:10:45 CL: Standard log file entry: Standard log file entry: Standard log file entry:
16e8 00:11:45 CL: Standard log file entry: Standard log file entry: Standard log file entry:
16e8 00:11:45 CL: Standard log file entry: Standard log file entry: Standard log file entry:
16e8 00:11:45 CL: Standard log file entry: Standard log file entry: Standard log file entry:
16e8 00:11:45 CL: Standard log file entry: Standard log file entry: Standard log file entry:
16e8 00:12:45 CL: Standard log file entry: Standard log file entry: Standard log file entry:
16e8 00:12:45 CL: Standard log file entry: Standard log file entry: Standard log file entry:
16e8 00:12:45 CL: Standard log file entry: Standard log file entry: Standard log file entry:
16e8 00:12:45 CL: Standard log file entry: Standard log file entry: Standard log file entry:
2138 00:13:45 01-BASICDT::HS: Standard log file entry: Standard log file entry: Standard log file entry:
16e8 00:13:45 CL: Standard log file entry: Standard log file entry: Standard log file entry:
1e0c 00:13:45 IOM WRITE: Standard log file entry: Standard log file entry: Standard log file entry:
16e8 00:13:45 CL: Standard log file entry: Standard log file entry: Standard log file entry:

所以它给了我从开始时间 ($tempTime = "00:08:45") 到分钟数 ($upperBound = $lowerBound.AddMinutes(5)) 的所有带时间戳的行，但它没有输出XML 或其他非时间戳行。使用上面的 ExampleLogfile.log 时，我试图实现以下输出：

16e8 00:08:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:08:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
OUTBOUND XML STRING RESULT OF TRANSFORMATION:

<?xml version="1.0"?>

<TgwGiMessage version="1.0">

    <ShortDataTransmitRequest>

        <SourceITSI SSI="XXXXXXX" EXT="0" CPTI="0" SNA="0"/>

        <DestinationAddress>

            <DestinationITSI SSI="XXXXXXX" EXT="0" CPTI="0" SNA="0"/>

        </DestinationAddress>

        <DeliveryConfirmation>1</DeliveryConfirmation>

        <MessageText>XML Text Message</MessageText>

    </ShortDataTransmitRequest>

</TgwGiMessage>


16e8 00:08:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
2138 00:08:45 01-BASICDT::HS: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:09:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
1e0c 00:09:45 IOM WRITE: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:09:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:09:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:10:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
APD OUTBOUND XML STRING RESULT OF TRANSFORMATION:
1600 00:10:45 CMD1: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:10:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:10:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:11:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:11:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:11:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:11:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:12:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:12:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:12:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
OUTBOUND XML STRING RESULT OF TRANSFORMATION:

<?xml version="1.0"?>

<TgwGiMessage version="1.0">

    <ShortDataTransmitRequest>

        <SourceITSI SSI="XXXXXXX" EXT="0" CPTI="0" SNA="0"/>

        <DestinationAddress>

            <DestinationITSI SSI="XXXXXXX" EXT="0" CPTI="0" SNA="0"/>

        </DestinationAddress>

        <DeliveryConfirmation>1</DeliveryConfirmation>

        <MessageText>XML Text Message</MessageText>

    </ShortDataTransmitRequest>

</TgwGiMessage>


16e8 00:12:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
2138 00:13:45 01-BASICDT::HS: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:13:45 CL: Standard log file entry: Standard log file entry: Standard log file entry: 
1e0c 00:13:45 IOM WRITE: Standard log file entry: Standard log file entry: Standard log file entry: 
16e8 00:13:45 CL: Standard log file entry: Standard log file entry: Standard log file entry:

我在这里做错了吗？这听起来像一个简单的要求，但我正在努力解决这个问题。

【问题讨论】：

【参考方案1】：

我会尝试这样的：

$file = Get-Content MyFile.log

# create your text pattern for regex matches here
$myPattern = "\d1,2\:\d1,2\:\d1,2"

# what time boundaries do you want to check?
$tempTime = "00:06:40"
$lowerBound = [DateTime]::Parse($lowerBound)
$upperBound = $lowerBound.AddMinutes(30)
    # tempTime can use any System.DateTime static method once
    # you have a date time, and should ideally be an input parameter
    # lowerBound should also be an input parameter

# this could also be a for, do / while, or for each
# loop through the file until we reach the end
for ($i = 0; $i -le $file.GetUpperBound(""); $i++)

    # this will loop until we reach the last line
    if ($file[$i] -match $myPattern)
    
        $time = $file[$i].Split(' ')[1]
        if ([DateTime]::Parse($time) -ge $lowerBound -and [DateTime]::Parse($time) -le $upperBound)
        
            $file[$i] # this will print to a console
                # using the Tee-Object cmdlet will output it to the console
                # and a file, which could be useful for developers and you

我还为您将其添加到了 pastebin。脚本在这里的作用很简单，但我会解释更多

获取文件内容（默认为字符串数组）创建与文件时间格式匹配的模式声明您想要的时间（tempTime，这将是您的输入参数）解析时间，获取上界时间（30分钟、60分钟等）遍历文件的每一行，如果它与正则表达式匹配，则在每个空间上拆分，获取数组索引 1（它将位于的那个），并将其存储在时间变量中确定时间（仍为字符串格式）是否大于或等于您的时间下限且小于或等于您的时间上限打印匹配的行返回

如果您在此处添加 System.String[] 类型的变量（使用 $myVariable = @() 行执行此操作），则可以执行 $myVariable += $file[$I]，然后使用此处封装此脚本的函数末尾的 return 关键字。

这是否有助于提供全面的答案？对于其他格式，此代码肯定存在问题，但因为我们明确地将其用于您拥有的日志格式，所以它应该可以正常工作。

它在 pastebin 上：http://pastebin.com/p1k7XPpD

另外，很抱歉我昨晚没有机会回答。我遇到了电脑问题，在试图解决它时睡着了。

更新：-------------------------------- 另一件事：我看到您在最后一个线程中使用了字符串本身，并且只是进行了简单的字符串比较。如果您只想走那条路，那将可行。创建两个 DateTime 对象不是必需的。要将上面的所有代码加载到函数中，您需要两个输入参数：一个是 System.String 类型，另一个是 System.UInt32 类型（Int32 和 UInt32 之间的区别在于 UInt32 不允许负数） .然后你的函数会被这样调用：

Get-LogExtract -StartTime "00:06:40" -AddMinutes 30

您只需在函数中添加对 AddMinutes 的引用。另一种方法是修改我在这里给你的内容，并使用 DateTime.Parse 方法简单地解析它们。或者，您可以将其保留为字符串比较；我正在使用 DateTime，但如果您知道它有效，那么使用字符串不会有任何问题。我针对您的示例输出运行了这个，效果很好。

【讨论】：

我刚刚看到问题是重读代码。在让它工作后，我更改了一些变量名称，因为它们以前有些模棱两可。如果将第 8 行 ($lowerBound = [DateTime]::Parse($lowerBound) ) 更改为：$lowerBound = [DateTime]::Parse($tempTime)，这将修复脚本。很抱歉！【参考方案2】：

看到上面的错误，我更新了代码......不确定你是否看到了评论，所以又来了：

$file = Get-Content MyFile.log

# create your text pattern for regex matches here
$myPattern = "\d1,2\:\d1,2\:\d1,2"

# what time boundaries do you want to check?
$tempTime = "00:06:40"
$lowerBound = [DateTime]::Parse($tempTime)
$upperBound = $lowerBound.AddMinutes(30)
    # tempTime can use any System.DateTime static method once
    # you have a date time, and should ideally be an input parameter
    # lowerBound should also be an input parameter

# this could also be a for, do / while, or for each
# loop through the file until we reach the end
for ($i = 0; $i -le $file.GetUpperBound(""); $i++)

    # this will loop until we reach the last line
    if ($file[$i] -match $myPattern)
    
        $time = $file[$i].Split(' ')[1]
        if ([DateTime]::Parse($time) -ge $lowerBound -and [DateTime]::Parse($time) -le $upperBound)
        
            $file[$i] # this will print to a console
                # using the Tee-Object cmdlet will output it to the console
                # and a file, which could be useful for developers and you

【讨论】：

PSGuy - 感谢您的更新。我看过你的帖子，但没有机会回复。我还尝试了您在尝试对脚本进行故障排除时建议的更改，假设它是基本的，但即使使用您在此处提供的代码，也只更改日志文件名的变量、开始时间 ($tempTime) 和$upperBound = $lowerBound.AddMinutes(2)，脚本仍然从00:00:00开始输出日志文件内容。感谢您继续尝试并协助解决这一问题，Andy PSGuy - 我更新了我的原始问题，提供了一些进一步的细节以显示当前的进展。我创建了一个简短的测试文件 ExampleLogfile.log 来尝试使脚本正常工作，但我现在似乎要倒退了……有关详细信息，请参阅更新。安迪

以上是关于Powershell - 如何处理非时间戳日志文件条目，包括空行的主要内容，如果未能解决你的问题，请参考以下文章

spark如何处理非数值的聚合最大值？ [复制]

Vue 组件 - 如何处理非反应性数据？

如何处理非规范化数据以传递给平面列表？

SQL Server 如何处理非聚集索引中的包含列？

C# HttpWebRequest.GetResponse - 如何处理非异常与 webexception 响应的 StatusCode 使用情况？

Latex 下如何处理非首行的缩进对齐？