Unix Shell 脚本仅从文件内容中获取需要的行，并在下一行考虑某些条件

Posted 2023-03-15

技术标签:

【中文标题】Unix Shell 脚本仅从文件内容中获取需要的行，并在下一行考虑某些条件【英文标题】：Unix Shell Script get only needed lines from a file content with certain criteria on the next line to be consider 【发布时间】：2013-09-25 07:47:39 【问题描述】：

帮助.......仍然无法正常工作

Related Topic

我有一个文件包含字符串：

Logs.txt

http://mediafire.com/?a6o37kfc7m68dri

部分内容：

BIF in setInputTag> set input tag value, name/value:ACCOUNT_NO/1746056 
BIF in setInputTag> set input tag value, name/value:bill_ref_no/197546058 
BIF in setInputTag> set input tag value, name/value:bill_ref_resets/1 
BIF in setInputTag> set input tag value, name/value:DISPUTE_AMOUNT/0 
BIF in setInputTag> set input tag value, name/value:NOD_FLAG/0 
BIF in setInputTag> set input tag value, name/value:OUTSTANDING_BALANCE/510312
BIF in setInputTag> set input tag value, name/value:TOTAL_DUE/510312 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(BILLZIPCODE) into ACCT map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(BILLEDACCOUNTNUMBER) into ACCT map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(BILLNUMBER) into ACCT map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(BILLNUMBERRESETS) into ACCT map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(OWNINGACCOUNTNUMBER) into SI inherited_map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(SUBSCRBNUMBER) into SI inherited_map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(SUBSCRBNUMBERRESET) into SI inherited_map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(DEFAULTEXTERNALID) into SI inherited_map 
BIF in InvoiceRecord> INFO: Output data to SI map,successful. SI=10593131,0
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(OWNINGACCOUNTNUMBER) into SI inherited_map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(SUBSCRBNUMBER) into SI inherited_map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(SUBSCRBNUMBERRESET) into SI inherited_map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(DEFAULTEXTERNALID) into SI inherited_map 
BIF in InvoiceRecord> INFO: Output data to SI map,successful. SI=26203008,0
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(OWNINGACCOUNTNUMBER) into SI inherited_map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(SUBSCRBNUMBER) into SI inherited_map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(SUBSCRBNUMBERRESET) into SI inherited_map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(DEFAULTEXTERNALID) into SI inherited_map 
BIF in InvoiceRecord> INFO: Output data to SI map,successful. SI=57292633,0
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(OWNINGACCOUNTNUMBER) into SI inherited_map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(SUBSCRBNUMBER) into SI inherited_map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(SUBSCRBNUMBERRESET) into SI inherited_map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(DEFAULTEXTERNALID) into SI inherited_map 
BIF in InvoiceRecord> INFO: Output data to SI map,successful. SI=914416,0
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(OWNINGACCOUNTNUMBER) into SI inherited_map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(SUBSCRBNUMBER) into SI inherited_map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(SUBSCRBNUMBERRESET) into SI inherited_map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(DEFAULTEXTERNALID) into SI inherited_map 
BIF in InvoiceRecord> INFO: Output data to SI map,successful. SI=914415,0
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(OWNINGACCOUNTNUMBER) into SI inherited_map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(SUBSCRBNUMBER) into SI inherited_map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(SUBSCRBNUMBERRESET) into SI inherited_map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(DEFAULTEXTERNALID) into SI inherited_map 
BIF in InvoiceRecord> INFO: Output data to SI map,successful. SI=57292561,0
BIF in InvoiceRecord> INFO: Output data to SI map,successful. SI=914415,0
BIF in InvoiceRecord> INFO: Output data to SI map,successful. SI=914416,0
BIF in InvoiceRecord> INFO: Output data to SI map,successful. SI=10593131,0
BIF in InvoiceRecord> INFO: Output data to SI map,successful. SI=26203008,0
BIF in InvoiceRecord> INFO: Output data to SI map,successful. SI=57292561,0
BIF in InvoiceRecord> INFO: Output data to SI map,successful. SI=57292633,0
BIF in InvoiceRecord> INFO: Output data to SI map,successful. SI=914415,0
BIF in InvoiceRecord> INFO: Output data to SI map,successful. SI=57292561,0
BIF in InvoiceRecord> INFO: Output data to SI map,successful. SI=57292561,0
BIF in setInputTag> set input tag value, name/value:ACCOUNT_NO/1746056 
BIF in setInputTag> set input tag value, name/value:bill_ref_no/202085235 
BIF in setInputTag> set input tag value, name/value:bill_ref_resets/1 
BIF in setInputTag> set input tag value, name/value:DISPUTE_AMOUNT/0 
BIF in setInputTag> set input tag value, name/value:NOD_FLAG/0 
BIF in setInputTag> set input tag value, name/value:OUTSTANDING_BALANCE/962908
BIF in setInputTag> set input tag value, name/value:TOTAL_DUE/962908 
BIF in run> INFO: updateParameterValues finished
BIF in getFile> INFO: Feed file(/arbordata/feedfile/PS-M26-1104-20130726-20130911-201711-1-bif01-1.DAT) for current invoice
BIF in insertNewFile> m_pcurFileNameAttr->file_seq_no:(1)  
BIF in insertNewFile> INFO:Output fileHeader(000020130911201711
) to feedfile,successful 
BIF in setInputTag> set input tag value, name/value:ACCOUNT_NO/1746056 
BIF in setInputTag> set input tag value, name/value:bill_ref_no/202085235 
BIF in setInputTag> set input tag value, name/value:bill_ref_resets/1 
BIF in setInputTag> set input tag value, name/value:DISPUTE_AMOUNT/0 
BIF in setInputTag> set input tag value, name/value:NOD_FLAG/1 
BIF in setInputTag> set input tag value, name/value:OUTSTANDING_BALANCE/962908
BIF in setInputTag> set input tag value, name/value:TOTAL_DUE/962908 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(BILLZIPCODE) into ACCT map 
BIF in addInheritedByField> INFO: Succeed to put the inherited value of field(BILLEDACCOUNTNUMBER) into ACCT map 
BIF in addInheritedByField> INFO: Succeed to put the inhelibarbor_db/database_gen.c(4670): arb_db_err - Entry
   errtype           = 2
   exec_user_handler = 1
   errcode           = 1422
   errtext           = Error in execution of OCI function unused:
retcode = -1 (OCI_ERROR)
ORA-01422: exact fetch returns more than requested number of rows
ORA-06512: at "ARBOR.PROJ_CSA_PROC2", line 9

我的场景是我想创建一个 .sh 脚本，该脚本将从文件中获取所需的所有字符串并将其格式化，如下所示，最后将其输出到一个新文件：

编辑

1729030|192840714|1|1422|Error in execution of OCI function unused:
1746056|202085235|1|1422|Error in execution of OCI function unused:
809434|201632617|1|1422|Error in execution of OCI function unused:
6993375|202888775|1|1422|Error in execution of OCI function unused:
7453294|199851665|1|1422|Error in execution of OCI function unused:

地点：

第 1 列：BIF in setInputTag> set input tag value, name/value:ACCOUNT_NO/1729030 第 2 列：BIF in setInputTag> set input tag value, name/value:bill_ref_no/192840714 第 3 列：BIF in setInputTag> set input tag value, name/value:bill_ref_resets/1 第 4 列：errcode =1422 Column5：errtext =未使用的OCI函数执行错误：

编辑

我终于感觉离我的目标更近了

执行此代码后：

egrep -e 'ACCOUNT_NO|bill_ref_no|bill_ref_resets|errcode|errtext' logs.txt | awk 'c-->0;$0~sif(b)for(c=b+1;c>1;c--)print r[(NR-c+1)%b];print;c=abr[NR%b]=$0' b=4 a=0 s='errtext'

（同上：分行） - 运行时将其放入 1 行：

egrep -e 'ACCOUNT_NO|bill_ref_no|bill_ref_resets|errcode|errtext' logs.txt
|awk 'c-->0;$0~sif(b)for(c=b+1;c>1;c--)print r[(NR-c+1)%b];print;c=abr[NR%b]=$0'
b=4 a=0 s='errtext'

编辑

将这行代码添加到我之前的代码之后：

| sed -e 's/=/:/' -e 's/^BIF.*://' -e 's=/=:=' -e 's/^ *//' -e 's/ *: /:/' | sed -e 's/ACCOUNT_NO://g' -e 's/bill_ref_no:/|/g' -e 's/bill_ref_resets:/|/g' -e 's/errcode:/|/g' -e 's/errtext:/|/g'

这是我现在的输出：

1729030
|192840714
|1
|1422
|Error in execution of OCI function unused:
1746056
|202085235
|1
|1422
|Error in execution of OCI function unused:
809434
|201632617
|1
|1422
|Error in execution of OCI function unused:
6993375
|202888775
|1
|1422
|Error in execution of OCI function unused:
7453294
|199851665
|1
|1422
|Error in execution of OCI function unused:

现在的最后一个问题是如何修剪换行符以在下面输出

它尝试添加此代码，但它不起作用：

| sed -e 's/\n//g;'

如何修剪那些新行？

想要的输出是：

1729030|192840714|1|1422|Error in execution of OCI function unused:
1746056|202085235|1|1422|Error in execution of OCI function unused:
809434|201632617|1|1422|Error in execution of OCI function unused:
6993375|202888775|1|1422|Error in execution of OCI function unused:
7453294|199851665|1|1422|Error in execution of OCI function unused:

我现在的完整代码：

egrep -e 'ACCOUNT_NO|bill_ref_no|bill_ref_resets|errcode|errtext' logs.txt | awk 'c-->0;$0~sif(b)for(c=b+1;c>1;c--)print r[(NR-c+1)%b];print;c=abr[NR%b]=$0' b=4 a=0 s='errtext' | sed -e 's/=/:/' -e 's/^BIF.*://' -e 's=/=:=' -e 's/^ *//' -e 's/ *: /:/' | sed -e 's/ACCOUNT_NO://g' -e 's/bill_ref_no:/|/g' -e 's/bill_ref_resets:/|/g' -e 's/errcode:/|/g' -e 's/errtext:/|/g' | sed -e 's/\n//g;'

【问题讨论】：

sed -n -e '/value:ACCOUNT_NO/N;N;s/..\n$.$/|\1/g;p;' logs.txt> OUPUT_logs.out 这是我们迄今为止尝试过的......但这不是新场景。我是 shell 脚本的新手...请帮帮我.. @user2775189 这些值是否只显示在一行中？我的意思是如果所有这些值都可以在源文件的一行中找到。能不能展示一部分源文件有一套完整的数据？部分内容请参考已编辑问题谢谢嗨@konsolebox 请参考编辑后的帖子...谢谢 【参考方案1】：

请试试这个，

egrep -e 'ACCOUNT_NO|bill_ref_no|bill_ref_resets|errcode|(Error in execution)'
logs.txt       
| 
grep -B4 'Error in execution' 
|
sed -e 's/ACCOUNT_NO\/\(.*\)/\1|/' -e 's/.*bill_ref_no\/\(.*\)/\1|/' -e 's/.*bill_ref_resets\/\(.*\)/\1|/' -e 's/.*errcode.*= \(.*\)/\1|/' 
-e 's/.*errtext.*= \(.*\)/\1/' -e '/--/d' 
| tr -d '\n' | sed  -e 's/:/:\n/g' -e 's/ |/|/g'
> out_put_file.txt

我使用 bash 在 Ubuntu 12.10 上工作。这太长了，所以我把它分成几行，当你运行它时，你需要把它们放在一行中。 *** 也有一些格式问题。我需要编辑它以显示完整的命令行。

这是我的输出，

1729030|192840714|1|1422|Error in execution of OCI function unused:
1746056|202085235|1|1422|Error in execution of OCI function unused:
809434|201632617|1|1422|Error in execution of OCI function unused:
6993375|202888775|1|1422|Error in execution of OCI function unused:
7453294|199851665|1|1422|Error in execution of OCI function unused:

根据给定的 logs.txt，您的原始输出的第二行似乎是错误的。

【讨论】：

运行uname -a或运行echo $HOSTNAME。 egrep: can't open logs.txt grep: illegal option -- B grep: illegal option -- 4 usage: grep [-E|-F] [-c|-l|-q] [-bhinsvwx] -e pattern_list... [-f pattern_file...] [file...] usage: grep [-E|-F] [-c|-l|-q] [-bhinsvwx] [-e pattern_list...] -f pattern_file... [file...] usage: grep [-E|-F] [-c|-l|-q] [-bhinsvwx] pattern [file...] uname -a B.11.23 U ia64 1103159669 无限制用户许可第一个错误意味着您的logs.txt 不在当前文件夹中。对于第二个错误，只需将grep 替换为egrep。 egrep: illegal option -- B egrep: illegal option -- 4 usage: egrep [-c|-l|-q] [-bhinsvwx] -e pattern_list... [-f pattern_file...] [file...]` usage: egrep [-c|-l|-q] [-bhinsvwx] [-e pattern_list...] -f pattern_file... [file...] usage: egrep [-c|-l|-q] [-bhinsvwx] pattern [file...]【参考方案2】：

只需一个 awk 命令即可：

#!/usr/bin/awk -f


    if (/ACCOUNT_NO/) 
        sub(/.*\//, "")
        account_no = $NF
        step = 2
     else if (/bill_ref_no/) 
        if (step++ == 2) 
            sub(/.*\//, "")
            bill_ref_no = $NF
         else 
            step = 1
        
     else if (/bill_ref_resets/) 
        if (step++ == 3) 
            sub(/.*\//, "")
            bill_ref_resets = $NF
         else 
            step = 1
        
     else if (/errcode/) 
        if (step++ == 4) 
            sub(/.* = /, "")
            errcode = $0
         else 
            step = 1
        
     else if (/errtext/) 
        if (step == 5) 
            sub(/.* = /, "")
            print account_no "|" bill_ref_no "|" bill_ref_resets "|" errcode "|" $0
        
        step = 1

运行方式：

awk -f script.awk log.txt

基于上述输入的输出：

1746056|202085235|1|1422|Error in execution of OCI function unused:

【讨论】：

syntax error The source line is 3.The error context is<<< >>>awk: QuittingThe source line is 3. @user2775189 我再次测试了脚本，它运行良好。也许这取决于你的awk？你确定你抄得好？ @konslebox 是否存在关于 awk 的兼容性问题？因为再次尝试它，它再次返回一个与我之前的评论相同的错误......怎么了？ HP-UX 是服务器 @user2775189 我也用其他 awk 变体对其进行了测试，对我来说效果很好。

以上是关于Unix Shell 脚本仅从文件内容中获取需要的行，并在下一行考虑某些条件的主要内容，如果未能解决你的问题，请参考以下文章

仅从 Bash 脚本中的路径获取文件名 [重复]

Unix shell 脚本和 PLSQL

用于在远程系统上归档文件的 Unix shell 脚本

在UNIX shell编程里如何比较两个文件哪个修改的时间晚些!急谢谢!

在 Unix/Linux 中判断两个文件是不是具有相同内容的最快方法？

Unix shell script 找出脚本文件所在的目录？