awk数组

Posted jazzxs

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了awk数组相关的知识,希望对你有一定的参考价值。

awk数组

1.1 awk命令用法

awk  参数  ‘条件{动作}’

1.2 awk执行过程

 技术分享图片

 

1.3 条件

1, 正则表达式

2, 比较表达式    NR>2

3, 范围

4, 特殊条件 BEGIN    END

1.4 awk实践

1.4.1 awk简单实践

[[email protected] files]# cat red.log    环境准备

Zhang Dandan    41117397   :250:100:175

Zhang Xiaoyu    390320151  :155:90:201

Meng  Feixue    80042789   :250:60:50

Wu    Waiwai    70271111   :250:80:75

Liu   Bingbing  41117483   :250:100:175

Wang  Xiaoai    3515064655 :50:95:135

Zi    Gege      1986787350 :250:168:200

Li    Youjiu    918391635  :175:75:300

Lao   Nanhai    918391635  :250:100:175

1.  显示Xiaoyu的姓氏和id号码

[[email protected] files]# awk ‘$2~/Xiaoyu/{print $1,$3}‘ red.log

2.显示所有以41开头的ID号码的人的全名和ID号

[[email protected] files]# awk ‘$3~/^41/{print $1,$2,$3}‘ red.log

3.显示所有ID号码最后一位数字是1或5的人的全名

[[email protected] files]# awk ‘$3~/[15]$/{print $1,$2}‘ red.log

4.显示Xiaoyu的捐款.每个值时都有以$开头.如$520$200$135

[[email protected] files]# awk ‘$2~/Xiaoyu/{gsub(/:/,"$",$NF);print}‘ red.log

5.显示第一行到第五行

[[email protected] files]# awk ‘NR==1,NR==5‘ red.log

6显示3开头的行开始到5开头的行结束

[[email protected] files]# awk ‘/3/,/5/‘ num.txt

7.显示从{开始到}结束

[[email protected] files]# awk ‘/{/,/}/‘ num.txt

8,把从{开始到}结束内容里的oldboy替换为lidao

[[email protected] files]# awk ‘/{/,/}/{gsub(/oldboy/,”lidao”,);print}‘ num.txt

9,BEGIN{} 在awk读取文件内容之前显示

[[email protected] files]# awk ‘BEGIN{print "this is start"}   {print $0}‘  red.log

10,END{} 在awk读取文件内容之后执行

统计etc/services的空行数       

[[email protected] files]# awk ‘/^$/{i++}END{print i}‘ /etc/services           求次数

11,求总和

[[email protected] files]# awk ‘{i=i+$1}END{print i}‘ a.txt

1.5 awk数组

[[email protected] files]# sort -t "." -rnk4 ip.txt |uniq -c |sort -rn

      4 10.0.0.13

      2 10.0.0.15

      2 10.0.0.12

      1 10.0.0.88

      1 10.0.0.33

      1 10.0.0.24

      1 10.0.0.228

      1 10.0.0.17

      1 10.0.0.121

      1 10.0.0.111

      1 10.0.0.11

#-t 指定分隔符

#-r 逆序

#-n 按照数字进行排序

#-k 第几列

 

统计access.log中每个ip出现的次数(第1列)

[[email protected] files]# awk ‘{print $1}‘ access.log |sort |uniq -c |sort -rn|head

  12049 58.220.223.62

  10856 112.64.171.98

   1982 114.83.184.139

   1662 117.136.66.10

   1318 115.29.245.13

    961 223.104.5.197

    957 116.216.0.60

    939 180.111.48.14

    871 223.104.5.202

    869 223.104.4.139

 

[[email protected] files]# awk ‘{h[$1]++}END{for( police in h ) print police,h[police]}‘ ip.log

10.0.0.17 1

10.0.0.228 1

10.0.0.111 1

10.0.0.121 1

10.0.0.11 1

10.0.0.12 2

10.0.0.88 1

10.0.0.13 4

10.0.0.24 1

10.0.0.33 1

10.0.0.15 2

[[email protected] files]# #统计 分类出现次数

访问IP求和

[[email protected] files]# awk ‘{h[$1]++}END{for( police in h ) print police,h[police]}‘ access.log |sort -rnk2 |head

58.220.223.62 12049

112.64.171.98 10856

114.83.184.139 1982

117.136.66.10 1662

115.29.245.13 1318

223.104.5.197 961

116.216.0.60 957

180.111.48.14 939

223.104.5.202 871

223.104.4.139 869

1.1 考试题:处理以下文件内容,将域名取出并根据域名进行计数排序处理(百度和sohu面试题)

[[email protected] files]# cat oldboy.txt

http://www.etiantian.org/index.html

http://www.etiantian.org/1.html

http://post.etiantian.org/index.html

http://mp3.etiantian.org/index.html

http://www.etiantian.org/3.html

http://post.etiantian.org/2.html

 

[[email protected] files]# awk -F"/+" ‘{print $2}‘ oldboy.txt |sort|uniq -c

      1 mp3.etiantian.org

      2 post.etiantian.org

      3 www.etiantian.org

awk数组:

[[email protected] files]# awk -F"/+" ‘{h[$2]++}‘END‘{for(p in h) print p,h[p]}‘ oldboy.txt

mp3.etiantian.org 1

post.etiantian.org 2

www.etiantian.org 3

1.2 secure-20161219 查看破解最多10ip 显示出被破解次数最多10个用户

[[email protected] files]# head secure-20161219

Dec 11 03:49:23 localhost sshd[27086]: Did not receive identification string from 123.31.34.190 port 55390

Dec 11 03:49:24 localhost sshd[27087]: Invalid user support from 123.31.34.190 port 55493

Dec 11 03:49:24 localhost sshd[27087]: input_userauth_request: invalid user support [preauth]

Dec 11 03:49:25 localhost sshd[27087]: error: Could not get shadow information for NOUSER

Dec 11 03:49:25 localhost sshd[27087]: Failed password for invalid user support from 123.31.34.190 port 55493 ssh2

 

查看破解最多10个ip

[[email protected] files]# awk ‘/Failed password/{h[$(NF-3)]++}END{for(p in h) print p,h[p]}‘ secure-20161219|sort -nrk2|head

218.65.30.25 68652

218.65.30.53 34326

218.87.109.154 21201

112.85.42.103 18065

112.85.42.99 17164

218.87.109.151 17163

218.87.109.150 17163

218.65.30.61 17163

218.65.30.126 17163

218.65.30.124 17163

 

显示出被破解次数最多10个用户

[[email protected] files]# awk ‘/Failed password/{h[$(NF-5)]++}END{for(p in h) print p,h[p]}‘ secure-20161219|sort -nrk2|head

root 364610

admin 725

user 245

oracle 119

support 104

guest 79

test 70

ubnt 47

pi 41

webadmin 36

 

access每个ip地址访问了网站次数

[[email protected] files]# awk ‘{h[$1]++}END{for(p in h) print p,h[p]}‘ access.log|sort -nrk2|head

58.220.223.62 12049

112.64.171.98 10856

114.83.184.139 1982

117.136.66.10 1662

115.29.245.13 1318

223.104.5.197 961

116.216.0.60 957

180.111.48.14 939

223.104.5.202 871

223.104.4.139 869

1.3 计算两列求和

[[email protected] files]# cat 1.txt

a 2

b 3

c 4

a 5

a 5

d 6

e 7

b 3

f 5

[[email protected] files]# awk ‘{h[$1]=h[$1]+$2}END{for(p in h) print p,h[p]}‘ 1.txt

a 12

b 6

c 4

d 6

e 7

f 5

 

统计access.log中每个ip使用的流量总数 显示流量总数最多的前10

[[email protected] files]# awk ‘{h[$1]=h[$1]+$10}END{for(p in h)print p,h[p]}‘ access.log |sort -nrk2|head

114.83.184.139 31362956

117.136.66.10 22431302

116.216.30.47 21466000

223.104.5.197 21464856

116.216.0.60 19145329

114.141.164.180 17219553

114.111.166.22 17121524

223.104.5.202 16911512

116.228.21.187 15969887

112.64.171.98 15255013

以上是关于awk数组的主要内容,如果未能解决你的问题,请参考以下文章

awk数组

awk入门及awk数组相关实战

awk数组函数脚本

awk数组

awk详解 数组

awk=================awk 中的next ,exit 和 数组