2018-08-18

Posted 2020-12-31 2kp2

9.1 正则介绍_grep上

• 正则就是一串有规律的字符串
• 掌握好正则对于编写shell脚本有很大帮助
• 各种编程语言中都有正则，原理是一样的
• 本章将要学习grep/egrep、sed、awk

grep -参数 ‘关键词‘ [文件名]

-c 查看行数

[[email protected] ~]# grep -c ‘nologin‘ /etc/passwd
15

-n 查看行号

[[email protected] ~]# grep -n ‘nologin‘ /etc/passwd
2:bin:x:1:1:bin:/bin:/sbin/nologin
3:daemon:x:2:2:daemon:/sbin:/sbin/nologin
4:adm:x:3:4:adm:/var/adm:/sbin/nologin
5:lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
9:mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
10:operator:x:11:0:operator:/root:/sbin/nologin
11:games:x:12:100:games:/usr/games:/sbin/nologin
12:ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin
13:nobody:x:99:99:Nobody:/:/sbin/nologin
14:systemd-network:x:192:192:systemd Network Management:/:/sbin/nologin
15:dbus:x:81:81:System message bus:/:/sbin/nologin
16:polkitd:x:999:998:User for polkitd:/:/sbin/nologin
17:sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
18:postfix:x:89:89::/var/spool/postfix:/sbin/nologin
19:chrony:x:998:996::/var/lib/chrony:/sbin/nologin

-i 不区分大小写

-v 反转查找

[[email protected] ~]# grep -v ‘nologin‘ /etc/passwd
root:x:0:0:root:/root:/bin/bash
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
aming:x:1000:1000::/home/aming:/bin/bash

-r 当指定要查找的是目录而非文件时，必须使用这项参数，否则grep命令将回报信息并停止动作

[[email protected] ~]# grep -n ‘nologin‘ /etc/
grep: /etc/: 是一个目录

-A 后面跟数字，过滤出符合要求的行以及下面n行

[[email protected] ~]# grep -A2 ‘root‘ /etc/passwd
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
--
operator:x:11:0:operator:/root:/sbin/nologin
games:x:12:100:games:/usr/games:/sbin/nologin
ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin

-B 同上，过滤出符合要求的行以及上面n行

[[email protected] ~]# grep -B2 ‘root‘ /etc/passwd
root:x:0:0:root:/root:/bin/bash
--
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
operator:x:11:0:operator:/root:/sbin/nologin

-C 同上，同时过滤出符合要求的行以及上下各n行

[[email protected] ~]# grep -C2 ‘root‘ /etc/passwd
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
--
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
operator:x:11:0:operator:/root:/sbin/nologin
games:x:12:100:games:/usr/games:/sbin/nologin
ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin

9.2 grep中

使用通配符进行搜索

[[email protected] ~]# grep [0-9] /etc/inittab
# multi-user.target: analogous to runlevel 3
# graphical.target: analogous to runlevel 5

^# 以#开头的行

[[email protected] ~]# grep -v ‘^#‘ /etc/fstab

/dev/mapper/centos-root /                       xfs     defaults        0 0
UUID=b2dc408e-ccc7-4fc1-8929-2a70d47cd5a0 /boot                   xfs     defaults        0 0
/dev/mapper/centos-swap swap                    swap    defaults        0 0
[[email protected] ~]# grep ‘^#‘ /etc/fstab
#
# /etc/fstab
# Created by anaconda on Fri Jul 27 05:10:42 2018
#
# Accessible filesystems, by reference, are maintained under ‘/dev/disk‘
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#

[^0-9] ^放入[]为非的意思，此处为非数字的行

9.3 grep下

r.o .为任意一个字符

[[email protected] ~]# grep ‘r.o‘ passwd 
root:x:0:0:root:/root:/bin/bash
r8o
r*o
operator:x:11:0:operator:/root:/sbin/nologin

o*o *左边的字符重复0~n次

[[email protected] ~]# grep ‘o*‘ passwd 
r8o
r*o
hooom
hom
hooooooooooom
[[email protected] ~]# grep ‘o*m‘ passwd 
hooom
hom
hooooooooooom

当出现{}()|+?时，需要用以下表达方式：

grep ‘o{3}‘ filename
grep -E ‘o{3}‘ filename
egrep ‘o{3}‘ filename

o{3} 大括号内表示前一个字符连续出现的次数

[[email protected] ~]# egrep ‘o{3}‘ passwd 
hooom
hooooooooooom
[[email protected] ~]# egrep ‘o{2,3}‘ passwd 
root:x:0:0:root:/root:/bin/bash
hooom
hooooooooooom
hoomo

o+o +左边的字符重复1~n次

root:x:0:0:root:/root:/bin/bash
hooom
hooooooooooom
hoomo

o?c ?左边的字符重复0~1次

[[email protected] ~]# egrep ‘o?c‘ passwd 
oc
c

root|nologin 或者

[[email protected] ~]# egrep ‘root|nologin‘ passwd 
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
operator:x:11:0:operator:/root:/sbin/nologin
games:x:12:100:games:/usr/games:/sbin/nologin
ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin
nobody:x:99:99:Nobody:/:/sbin/nologin
systemd-network:x:192:192:systemd Network Management:/:/sbin/nologin
dbus:x:81:81:System message bus:/:/sbin/nologin
polkitd:x:999:998:User for polkitd:/:/sbin/nologin
sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
postfix:x:89:89::/var/spool/postfix:/sbin/nologin
chrony:x:998:996::/var/lib/chrony:/sbin/nologin

9.4/9.5 sed

sed ‘/关键词/‘p filename

-n 显示关键词所在行

[[email protected] ~]# sed -n ‘/root/‘p passwd 
root:x:0:0:root:/root:/bin/bash
operator:x:11:0:operator:/root:/sbin/nologin

当出现{}()|+?时，需要加-r参数

[[email protected] ~]# sed -nr ‘/o{3}/‘p passwd 
hooom
hooooooooooom

打印第五行

[[email protected] ~]# sed -n ‘5‘p passwd 
hom

打印1~5行

[[email protected] ~]# sed -n ‘1,5‘p passwd 
root:x:0:0:root:/root:/bin/bash
r8o
r*o
hooom
hom

打印25~尾行

[[email protected] ~]# sed -n ‘25,$‘p passwd 
polkitd:x:999:998:User for polkitd:/:/sbin/nologin
sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
postfix:x:89:89::/var/spool/postfix:/sbin/nologin
chrony:x:998:996::/var/lib/chrony:/sbin/nologin
aming:x:1000:1000::/home/aming:/bin/bash

-e 匹配多个关键词

[[email protected] ~]# sed -n -e ‘1‘p -e ‘/bus/‘p passwd 
root:x:0:0:root:/root:/bin/bash
dbus:x:81:81:System message bus:/:/sbin/nologin

I 不区分大小写

sed -n ‘/bus/‘Ip passwd

d 删除匹配的行（原文件不变，加-i则原文件改变）

[[email protected] ~]# sed ‘1,25‘d passwd 
sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
postfix:x:89:89::/var/spool/postfix:/sbin/nologin
chrony:x:998:996::/var/lib/chrony:/sbin/nologin
aming:x:1000:1000::/home/aming:/bin/bash

sed ‘1,3s/ot/to/g‘ test.txts表示替换，g表示全局，1到3行把ot替换成to

sed ‘s/[0-9]//g‘ test.txt把所有数字替换成空（删除）

& 表示前面搜索的关键词

[[email protected] ~]# sed ‘s/^.*$/123:&/‘ passwd
123:root:x:0:0:root:/root:/bin/bash

123 表示前面第几个括号的内容

[[email protected] ~]# sed -r ‘s/([^:]+)(:.*:)([^:]+)/321/‘ passwd 
/bin/bash:x:0:0:root:/root:root

9.6/9.7 awk

以：为分隔符，打印第一段（不更改原文件）

[[email protected] ~]# awk -F ‘:‘ ‘{print $1}‘ passwd 
root
bin
daemon

打印全部

aming
[[email protected] ~]# awk ‘{print $0}‘ passwd 
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin

打印指定段

[[email protected] ~]# awk -F ‘:‘ ‘{print $1,$2,$4}‘ passwd 
root x 0
bin x 1
daemon x 2

指定打印的分隔符

[[email protected] ~]# awk -F ‘:‘ ‘{print $1"@"$2"@"$4}‘ passwd 
[email protected]@0
[email protected]@1
[email protected]@2
[[email protected] ~]# awk -F ‘:‘ ‘{OFS="@"} {print $1,$3,$7}‘ passwd
[email protected]@/bin/bash
[email protected]@/sbin/nologin
[email protected]@/sbin/nologin

在指定段搜索关键词

[[email protected] ~]# awk -F ‘:‘ ‘$1 ~ /oo/‘ passwd 
root:x:0:0:root:/root:/bin/bash
[[email protected] ~]# awk -F ‘:‘ ‘$1 ~ /o{2}/‘ passwd 
root:x:0:0:root:/root:/bin/bash
[[email protected] ~]# awk -F ‘:‘ ‘$1 ~ /o{2}/ {print $1}‘ passwd
root
[[email protected] ~]# awk -F ‘:‘ ‘$1 ~ /root|aming/ {print $6}‘ passwd
/root
/home/aming

数学计算

[[email protected] ~]# awk -F ‘:‘ ‘$3==0 {print $1}‘ passwd
root
[[email protected] ~]# awk -F ‘:‘ ‘$3>=1000 {print $1}‘ passwd
aming
[[email protected] ~]# awk -F ‘:‘ ‘$3<5 {print $1}‘ passwd
root
bin
daemon
adm
lp

两个字段比较

[[email protected] ~]# awk -F ‘:‘ ‘$3==$4 {print $0}‘ passwd
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
[[email protected] ~]# awk -F ‘:‘ ‘$3>$4 && $3<1000 {print $0}‘ passwd
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
operator:x:11:0:operator:/root:/sbin/nologin
polkitd:x:999:998:User for polkitd:/:/sbin/nologin
chrony:x:998:996::/var/lib/chrony:/sbin/nologin
[[email protected] ~]# awk -F ‘:‘ ‘$3<5 && $7 ~ /bash/  {print $0}‘ passwd
root:x:0:0:root:/root:/bin/bash
[[email protected] ~]# awk -F ‘:‘ ‘$3<5 || $7 ~ /bash/  {print $0}‘ passwd
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
aming:x:1000:1000::/home/aming:/bin/bash

NR行号

[[email protected] ~]# awk -F ‘:‘ ‘ {print NR,$1,$3,$7}‘ passwd
1 root 0 /bin/bash
2 bin 1 /sbin/nologin
3 daemon 2 /sbin/nologin
[[email protected] ~]# awk -F ‘:‘ ‘NR>20 {print $1,$3,$7}‘ passwd
ftp 14 /sbin/nologin
nobody 99 /sbin/nologin
systemd-network 192 /sbin/nologin
dbus 81 /sbin/nologin
polkitd 999 /sbin/nologin
sshd 74 /sbin/nologin
postfix 89 /sbin/nologin
chrony 998 /sbin/nologin
aming 1000 /bin/bash

NF段数

[[email protected] ~]# awk -F ‘:‘ ‘ {print NF,$1,$3,$7}‘ passwd
7 root 0 /bin/bash
7 bin 1 /sbin/nologin
7 daemon 2 /sbin/nologin
[[email protected] ~]# awk -F ‘:‘ ‘NF==1 {print $1}‘ passwd
r8o

赋值

[[email protected] ~]# awk -F ‘:‘ ‘{OFS=":"} $1="root"‘ passwd
root:x:0:0:root:/root:/bin/bash
root:x:1:1:bin:/bin:/sbin/nologin
root:x:2:2:daemon:/sbin:/sbin/nologin
[[email protected] ~]# awk -F ‘:‘ ‘{(tot=tot+$3)}; END {print tot}‘ /etc/passwd
3605

扩展

把一个目录下，过滤所有*.php文档中含有eval的行
grep -r --include="*.php" ‘eval‘ /data/