记一次mogodb占用cpu高问题
Posted 巫谢
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了记一次mogodb占用cpu高问题相关的知识，希望对你有一定的参考价值。
公司服务器上安装了contly，是一个开源的node.js项目，用于统计手机app使用情况，后端数据储存使用的mongodb，使用的时候经常发现mongodb占用cpu非常高，打到了210%的爆表值
top - 13:42:39 up 308 days, 23:01, 2 users, load average: 2.84, 2.96, 2.93 Tasks: 209 total, 1 running, 208 sleeping, 0 stopped, 0 zombie %Cpu(s): 59.4 us, 2.7 sy, 0.0 ni, 36.9 id, 0.2 wa, 0.7 hi, 0.0 si, 0.0 st KiB Mem: 8173524 total, 6095460 used, 2078064 free, 133752 buffers KiB Swap: 1048572 total, 750536 used, 298036 free. 3480688 cached Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 16870 root 20 0 48.452g 2.643g 2.490g S 210.5 33.9 38607:41 mongod 28508 root 20 0 891316 110272 5976 S 5.0 1.3 10:07.08 nodejs 28589 root 20 0 893736 114044 5972 S 5.0 1.4 13:19.13 nodejs 28566 root 20 0 959004 113128 5976 S 4.0 1.4 11:16.49 nodejs 28670 root 20 0 1189932 383820 5980 S 4.0 4.7 26:00.97 nodejs 16367 www-data 20 0 86960 2952 996 S 1.0 0.0 237:47.74 nginx 16368 www-data 20 0 87028 3212 996 S 1.0 0.0 235:54.43 nginx 16370 www-data 20 0 86964 2952 996 S 1.0 0.0 245:46.74 nginx 16371 www-data 20 0 86960 2948 996 S 1.0 0.0 246:40.89 nginx 1154 root 20 0 23792 1764 1144 R 0.7 0.0 0:00.23 top 7 root 20 0 0 0 0 S 0.3 0.0 1053:48 rcu_sched 8 root 20 0 0 0 0 S 0.3 0.0 229:02.41 rcuos/0 6559 root 20 0 3957096 217820 5668 S 0.3 2.7 230:37.95 jsvc 1 root 20 0 37348 3704 976 S 0.0 0.0 0:19.37 init 2 root 20 0 0 0 0 S 0.0 0.0 0:02.51 kthreadd 3 root 20 0 0 0 0 S 0.0 0.0 68:42.78 ksoftirqd/0 5 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/0:0H 9 root 20 0 0 0 0 S 0.0 0.0 225:56.62 rcuos/1 10 root 20 0 0 0 0 S 0.0 0.0 217:26.86 rcuos/2 11 root 20 0 0 0 0 S 0.0 0.0 228:57.90 rcuos/3 12 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuos/4 13 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuos/5 14 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuos/6 15 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuos/7 16 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuos/8 17 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuos/9 18 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuos/10 19 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuos/11 20 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuos/12 21 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuos/13 22 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuos/14 23 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuos/15 24 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuos/16 25 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuos/17 26 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuos/18 27 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuos/19 28 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuos/20 29 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuos/21 30 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuos/22 31 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuos/23 32 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuos/24 33 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuos/25 34 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuos/26 35 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuos/27 36 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuos/28 37 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuos/29 38 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuos/30 39 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuos/31 40 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcu_bh 41 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuob/0 42 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuob/1 43 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuob/2 44 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuob/3 45 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuob/4 46 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuob/5 47 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuob/6 48 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuob/7 49 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuob/8 50 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuob/9 51 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuob/10 52 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuob/11
仔细查看是中断很多，命令是pid -w 1
root@localhost:~# pidstat -w 1 Linux 3.13.0-24-generic (localhost) 07/14/2017 _x86_64_ (4 CPU) 01:44:41 PM UID PID cswch/s nvcswch/s Command 01:44:42 PM 0 7 260.40 0.00 rcu_sched 01:44:42 PM 0 8 78.22 0.00 rcuos/0 01:44:42 PM 0 9 49.50 0.00 rcuos/1 01:44:42 PM 0 10 88.12 0.00 rcuos/2 01:44:42 PM 0 11 96.04 0.00 rcuos/3 01:44:42 PM 0 74 0.99 0.00 watchdog/0 01:44:42 PM 0 75 0.99 0.00 watchdog/1 01:44:42 PM 0 77 7.92 0.00 ksoftirqd/1 01:44:42 PM 0 80 0.99 0.00 watchdog/2 01:44:42 PM 0 82 39.60 0.00 ksoftirqd/2 01:44:42 PM 0 85 0.99 0.00 watchdog/3 01:44:42 PM 0 87 9.90 0.00 ksoftirqd/3 01:44:42 PM 0 1103 7.92 0.00 kworker/u64:2 01:44:42 PM 0 1213 0.99 4.95 pidstat 01:44:42 PM 0 1265 0.99 0.00 supervisord 01:44:42 PM 0 1279 9.90 0.00 vmtoolsd 01:44:42 PM 0 1373 0.99 0.00 fail2ban-server 01:44:42 PM 0 1954 18.81 0.00 xfsaild/sdb1 01:44:42 PM 0 2132 9.90 0.00 kworker/1:1H 01:44:42 PM 0 6800 0.99 0.00 kworker/0:1 01:44:42 PM 0 6807 0.99 0.00 kworker/2:2 01:44:42 PM 1000 14269 0.99 0.00 zabbix_agentd 01:44:42 PM 1000 14273 0.99 0.00 zabbix_agentd 01:44:42 PM 33 16367 154.46 0.00 nginx 01:44:42 PM 33 16368 178.22 0.00 nginx 01:44:42 PM 33 16370 166.34 0.00 nginx 01:44:42 PM 33 16371 168.32 0.00 nginx 01:44:42 PM 0 16870 100.00 0.00 mongod 01:44:42 PM 0 28498 0.99 0.00 supervisord 01:44:42 PM 0 28508 126.73 39.60 nodejs 01:44:42 PM 0 28566 141.58 46.53 nodejs 01:44:42 PM 0 28589 137.62 110.89 nodejs 以上是关于记一次mogodb占用cpu高问题的主要内容，如果未能解决你的问题，请参考以下文章记一次线上Java程序导致服务器CPU占用率过高的问题排除过程