Linux命令之awk数组使用范例

Posted

tags:

篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了Linux命令之awk数组使用范例相关的知识，希望对你有一定的参考价值。

目录

取ifconfig bond0的IP地址    1

命令如下：    2

统计apache日志单IP访问请求数排名    2

第一种方法    2

第二种方法    2

统计域名访问量    3

第一种方法：    3

第二种方法：    3

计算每个人的总工资和平均工资    4

命令如下：    4

对本地IP和远程IP去重并统计重复数    4

命令如下：    5

统计源IP，端口及目的IP同时去重    6

命令如下：    6

美化效果如下：    7

 

取ifconfig bond0的IP地址

 1 [[email protected] mnt]# ifconfig bond0
 2 
 3 bond0 Link encap:Ethernet HWaddr 00:0C:29:27:8F:AD
 4 
 5 inet addr:192.168.1.123 Bcast:192.168.1.255 Mask:255.255.255.0
 6 
 7 inet6 addr: fe80::20c:29ff:fe27:8fad/64 Scope:Link
 8 
 9 UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
10 
11 RX packets:27291 errors:0 dropped:0 overruns:0 frame:0
12 
13 TX packets:15128 errors:0 dropped:0 overruns:0 carrier:0
14 
15 collisions:0 txqueuelen:0
16 
17 RX bytes:38569717 (36.7 MiB) TX bytes:1032861 (1008.6 KiB)

命令如下：

1 [[email protected] mnt]# ifconfig bond0 |awk -F "[ :]+" ‘NR==2{print NR" "$4}‘
2 
3 2 192.168.1.123

 

统计apache日志单IP访问请求数排名

 1 [[email protected] mnt]# head -5 2.txt
 2 
 3 10.0.0.3 -- [21/Mar/2015-07:50:17+0800]*GET/HTTP/1.1*200 19 *-*
 4 
 5 10.0.0.3 -- [21/Mar/2015-07:50:17+0800]*GET/HTTP/1.1*200 19 *-*
 6 
 7 10.0.0.5 -- [21/Mar/2015-07:50:17+0800]*GET/HTTP/1.1*200 19 *-*
 8 
 9 10.0.0.3 -- [21/Mar/2015-07:50:17+0800]*GET/HTTP/1.1*200 19 *-*
10 
11 10.0.0.6 -- [21/Mar/2015-07:50:17+0800]*GET/HTTP/1.1*200 19 *-*

 

第一种方法

[[email protected] mnt]# awk ‘{print $1}‘ 2.txt |sort|uniq -c

35 10.0.0.3

5 10.0.0.4

10 10.0.0.5

10 10.0.0.6

 

第二种方法

[[email protected] mnt]# awk ‘{array[$1]++} END {for(key in array) print key,array[key]}‘ 2.txt

10.0.0.3 35

10.0.0.4 5

10.0.0.5 10

10.0.0.6 10

　　

统计域名访问量

[[email protected] mnt]# cat 1.txt

http://www.baidu.com/index.html

http://www.163.com/1.html

http://www.cnblogs.com/index.html

http://www.baidu.com/2.html

http://www.163.com/index.html

http://www.qq.com/index.html

http://www.baidu.com/3.html

http://www.163.com/2.html

http://www.baidu.com/2.html

 

第一种方法：

 1 [[email protected] mnt]# awk ‘{split($0,array,"/+");key=array[2];count[key]++}END{for(kk in count) print kk,count[kk]}‘ 1.txt
 2 
 3 www.qq.com 1
 4 
 5 www.cnblogs.com 1
 6 
 7 www.baidu.com 4
 8 
 9 www.163.com 3
10 
11

第二种方法：

1 [[email protected] mnt]# awk -F [/]+ ‘{array[$2]++} END {for(key in array) print key,array[key]}‘ 1.txt
2 
3 www.qq.com 1
4 
5 www.cnblogs.com 1
6 
7 www.baidu.com 4
8 
9 www.163.com 3

 

计算每个人的总工资和平均工资

 1 [[email protected] mnt]# cat 3.txt
 2 
 3 001 wodi 12k
 4 
 5 002 yingsui 15k
 6 
 7 003 jeacen 10k
 8 
 9 004 yideng 10k
10 
11 005 kuqi 8k
12 
13 006 xiaofen 6k
14 
15 007 wodi 11k
16 
17 008 yingsui 12k
18 
19 009 jeacen 4k
20 
21 010 kuqi 12k
22 
23 011 yideng 11k
24 
25 012 xiaofen 10k

 

命令如下：

[[email protected] mnt]# awk ‘{array[$2]+=$3;count[$2]++}END{for(key in array) print key,array[key]"k",array[key]/count[key]}‘ 3.txt

kuqi 20k 10

jeacen 14k 7

yingsui 27k 13.5

xiaofen 16k 8

wodi 23k 11.5

yideng 21k 10.5

　　

对本地IP和远程IP去重并统计重复数

 1 [[email protected] mnt]# cat 4.txt
 2 
 3 Proto Recv-Q Send-Q Local Addree Foreign Addree              State
 4 
 5 tcp            0         0 0.0.0.0:80         0.0.0.0:*     LISTEN
 6 
 7 tcp            0         0 115.29.49.213:80 117.136.27.254:13779     SYN_RECV
 8 
 9 tcp            0         0 115.29.49.213:80     113.97.117.157:1847     SYN_RECV
10 
11 tcp            0         0 115.29.49.213:80     117.136.40.20:19594     SYN_RECV
12 
13 tcp            0         0 115.29.49.213:80     117.136.40.20:19595     SYN_RECV
14 
15 tcp            0         0 115.29.49.213:80     121.236.219.69:45363     SYN_RECV
16 
17 tcp            0         0 0.0.0.0:21         0.0.0.0:*     LISTEN
18 
19 tcp            0         0 0.0.0.0:22     0.0.0.0:*         LISTEN
20 
21 unix         3 [] SYREAM CONNECTED 11183664 /TMP/mysql.SOCK
22 
23 unix         3 [] SYREAM CONNECTED 11183646 /TMP/MYSQL.SOCK
24 
25 unix         3 [] SYREAM CONNECTED 11183665 /TMP/MYSQL.SOCK
26 
27 unix         3 [] SYREAM CONNECTED 11183668 /TMP/MYSQL.SOCK
28 
29 unix         3 [] SYREAM CONNECTED 11183654 /TMP/MYSQL.SOCK
30 
31 unix         3 [] SYREAM CONNECTED 11183655 /TMP/MYSQL.SOCK
32 
33 unix         3 [] SYREAM CONNECTED 11183668 /TMP/MYSQL.SOCK
34 
35 unix         3 [] SYREAM CONNECTED 11183676 /TMP/MYSQL.SOCK
36 
37 unix         3 [] SYREAM CONNECTED 11183672 /TMP/MYSQL.SOCK

 

命令如下：

 1 [[email protected] mnt]# awk -F "[ :]+" ‘/^tcp/{array[$3" "$5]++}END{for(key in array) print key,array[key]}‘ 4.txt
 2 
 3 115.29.49.213 113.97.117.157 1
 4 
 5 115.29.49.213 117.136.27.254 1
 6 
 7 0.0.0.0 0.0.0.0 3
 8 
 9 115.29.49.213 117.136.40.20 2
10 
11 115.29.49.213 121.236.219.69 1

 

统计源IP，端口及目的IP同时去重

 1 [[email protected] mnt]# head -10 5.txt
 2 
 3 Dec 2 01:17:42 10.0.0.0 2009 RV016 RGFW-OUT:ACCEPT (TCP 10.0.0.131:1227->210.192.121.172:80 on ixp7) [0,0]
 4 
 5 Dec 2 01:17:42 10.0.0.0 2009 RV016 RGFW-OUT:ACCEPT (TCP 10.0.0.131:1227->210.192.121.172:80 on ixp7) [0,0]
 6 
 7 Dec 2 01:17:42 10.0.0.0 2009 RV016 RGFW-OUT:ACCEPT (TCP 10.0.0.131:1227->210.192.121.172:80 on ixp7) [0,0]
 8 
 9 Dec 2 01:17:42 10.0.0.0 2009 RV016 RGFW-OUT:ACCEPT (TCP 10.0.0.131:1227->210.192.121.172:80 on ixp7) [0,0]
10 
11 Dec 2 01:17:42 10.0.0.0 2009 RV016 RGFW-OUT:ACCEPT (TCP 10.0.0.131:1227->210.192.121.172:80 on ixp7) [0,0]
12 
13 Dec 2 01:17:42 10.0.0.0 2009 RV016 RGFW-OUT:ACCEPT (TCP 10.0.0.43:54963->203.81.19.92:80 on ppp6) [0,0]
14 
15 Dec 2 01:17:42 10.0.0.0 2009 RV016 RGFW-OUT:ACCEPT (UDP 10.0.0.19:1441->121.14.96.233:80 on ppp0) [0,0]
16 
17 Dec 2 01:17:42 10.0.0.0 2009 RV016 RGFW-OUT:ACCEPT (UDP 172.16.1.103:57318->211.147.6.3:80 on ppp2) [0,0]
18 
19 Dec 2 01:17:42 10.0.0.0 2009 RV016 RGFW-OUT:ACCEPT (TCP 172.16.1.203:4372->61.135.163.86:80 on ixp7) [0,0]
20 
21 Dec 2 01:17:42 10.0.0.0 2009 RV016 RGFW-OUT:ACCEPT (TCP 10.0.0.131:1227->210.192.121.172:80 on ixp7) [0,0]

 

命令如下：

 1 [[email protected] mnt]# vim 5.sh
 2 
 3 {
 4 
 5 split($9,array,":|->")
 6 
 7 sip=array[1]
 8 
 9 sport=array[2]
10 
11 mip=array[3]
12 
13 if (!((sip,sport,mip) in tree)){
14 
15 tree[sip,sport,mip] = 1
16 
17 }
18 
19 }
20 
21 END{
22 
23 for (key in tree)
24 
25 print key
26 
27 }

 

 1 [[email protected] mnt]# awk -f 5.sh 5.txt
 2 
 3 172.16.1.10357318211.147.6.3
 4 
 5 10.0.0.191441121.14.96.233
 6 
 7 172.16.1.203437261.135.163.86
 8 
 9 10.0.0.4354963203.81.19.92
10 
11 10.0.0.1311227210.192.121.172

 

美化效果如下：

[[email protected] mnt]# cat 5.sh
BEGIN{
	printf("%-16s %-6s %-16s\n","SIP","SPORT","MIP")
}
{
	split($9,array,":|->")
	sip=array[1]
	sport=array[2]
	mip=array[3]
	if (!((sip,sport,mip) in tree)){
	tree[sip,sport,mip] = 1
	}
}
END{
	for (key in tree){
		split(key,out,SUBSEP)
	printf("%-16s %-6s %-16s\n", out[1],out[2],out[3])
	}
}

　　

 1 [[email protected] mnt]# awk -f 5.sh 5.txt
 2 
 3 SIP SPORT MIP
 4 
 5 172.16.1.103 57318 211.147.6.3
 6 
 7 10.0.0.19 1441 121.14.96.233
 8 
 9 172.16.1.203 4372 61.135.163.86
10 
11 10.0.0.43 54963 203.81.19.92
12 
13 10.0.0.131 1227 210.192.121.172

 

以上是关于Linux命令之awk数组使用范例的主要内容，如果未能解决你的问题，请参考以下文章

打开高效文本编辑之门_Linux awk之关联数组

Linux系统命令三剑客之 awk

文本处理之awk高级进阶

awk命令使用范例

linux之awk命令

Linux运维三剑客awk必会知识--模式与操作内自变量语句块数组