三剑客之awk

Posted 2021-02-06 yjiu1990

三剑客之awk

awk命令的执行过程

#满足
awk从文件中读取一行内容到内存中--》判断是否满足条件--满足---执行对应的命令---输出到屏幕
#不满足
awk从文件中读取一行内容到内存中--》判断是否满足条件--不满足---继续读取文件里的内容直到文件最后

awk命令的语法

awk  参数  ‘模式{动作}‘  文件
awk  参数  ‘条件（找谁）{干啥}‘  文件
~：表示包含的意思
gsub:表示替换，语法(gsbu(/目标/,"替换成什么",第几列))

akw命令之查询

#示例文件
[[email protected] files]# cat reg.txt 
Zhang Dandan    41117397   :250:100:175
Zhang Xiaoyu    390320151  :155:90:201
Meng  Feixue    80042789   :250:60:50
Wu    Waiwai    70271111   :250:80:75
Liu   Bingbing  41117483   :250:100:175
Wang  Xiaoai    3515064655 :50:95:135
Zi    Gege      1986787350 :250:168:200
Li    Youjiu    918391635  :175:75:300
Lao   Nanhai    918391635  :250:100:175
#显示Xiaoyu的姓氏和ID号码
[[email protected] files]# awk ‘/Xiaoyu/{print $1,$2}‘ reg.txt 
Zhang Xiaoyu
#显示所有以41开头的ID号码的人的全名和ID号码
[[email protected] files]# awk ‘$3~/^41/{print $1,$2,$3}‘ reg.txt 
Zhang Dandan 41117397
Liu Bingbing 41117483
#显示所有ID号码最后一位数字是1或5的人的全名
[[email protected] files]# awk ‘$3~/[15]$/{print $1,$2}‘ reg.txt 
Zhang Xiaoyu
Wu Waiwai
Wang Xiaoai
Li Youjiu
Lao Nanhai
[[email protected] files]# 

awk命令之替换

[[email protected] files]# cat reg.txt 
Zhang Dandan    41117397   :250:100:175
Zhang Xiaoyu    390320151  :155:90:201
Meng  Feixue    80042789   :250:60:50
Wu    Waiwai    70271111   :250:80:75
Liu   Bingbing  41117483   :250:100:175
Wang  Xiaoai    3515064655 :50:95:135
Zi    Gege      1986787350 :250:168:200
Li    Youjiu    918391635  :175:75:300
Lao   Nanhai    918391635  :250:100:175
#显示Xiaoyu的捐款.每个值时都有以$开头.如$520$200$135
[[email protected] files]# awk ‘gsub(/:/,"$",$4)‘ reg.txt 
Zhang Dandan 41117397 $250$100$175
Zhang Xiaoyu 390320151 $155$90$201
Meng Feixue 80042789 $250$60$50
Wu Waiwai 70271111 $250$80$75
Liu Bingbing 41117483 $250$100$175
Wang Xiaoai 3515064655 $50$95$135
Zi Gege 1986787350 $250$168$200
Li Youjiu 918391635 $175$75$300
Lao Nanhai 918391635 $250$100$175
[[email protected] files]# 

awk命令之BEGIN模式：主要用来测试与计算

[[email protected] ~]# awk ‘BEGIN{print 100 * 2.2}‘
220

awk命令之END模式：用来计算显示最终统计结果

[[email protected] ~]# #统计/etc/services文件里面的空行数量
[[email protected] ~]# awk ‘/^$/{i++}END{print i}‘ /etc/services 
16

awk命令之数组:用来统计与计算

处理以下文件内容,将域名取出并根据域名进行计数排序处理:(百度和sohu面试题)
[[email protected] ~]# cat url.txt 
http://www.etiantian.org/index.html
http://www.etiantian.org/1.html
http://post.etiantian.org/index.html
http://mp3.etiantian.org/index.html
http://www.etiantian.org/3.html
http://post.etiantian.org/2.html
#方法一：使用单个添加
[[email protected] ~]# awk -F ‘[/.]+‘ ‘{h[$2]++}END{print h["www"],h["mp3"]}‘ url.txt 
3 1
#方法二：使用for循环
[[email protected] ~]# awk -F ‘[/.]+‘ ‘{h[$2]++}END{for(pol in h)print pol "	" h[pol]}‘ url.txt 
www    3
mp3    1
post    2
[[email protected] ~]# 

secure系统日志分析练习

谁在破解你的密码（Failed password 每个ip地址出现的次数）

[[email protected] files]# awk ‘/Failed password/{h[$(NF-3)]++}END{for(pol in h)print pol"	" h[pol]}
‘ secure-20161219 218.65.30.126    17163
218.65.30.61    17163
125.16.71.175    4
169.46.38.74    9
183.136.238.78    30
218.2.0.16    10
91.223.133.33    2
222.186.50.206    3289
51.254.143.19    9
113.207.7.3    316
111.73.46.156    3206
123.31.34.141    39
187.115.73.70    9
182.100.67.119    17163
218.87.109.150    17163
218.87.109.151    17163
..........

分析系统的每个用户被破解的次数

[[email protected] files]# awk ‘/error: maximum/{h[$(NF-4)]++}END{for(pol in h)print pol "	" h[pol]}
‘ secure-20161219 218.65.30.25    11440
218.65.30.61    2860
182.100.67.119    2860
218.65.30.53    5720
116.55.245.57    53
218.87.109.150    2860
218.87.109.151    2860
59.63.166.84    395
218.65.30.122    2859
182.100.67.120    2860
112.85.42.103    3009
218.65.30.123    2860
218.65.30.124    2860
218.87.109.154    3533
113.207.7.3    52
112.85.42.124    2860
112.85.42.107    2860
218.65.30.126    2860
112.85.42.99    2859