Arthas - 线上突然CPU飙升怎么查

Posted 2021-09-16 大军

我们在线上经常发现CPU突然飙升的情况，我们也可以用Arthas进行查找飙升的代码在哪里。

实例代码

当我们访问/cpu的时候，我们通过开启线程池的方式，调用一个while(true)的方法。

@RestController
public class CpuController {
    @RequestMapping("/cpu")
    public String cpu() {
        fun1();
        return "CPU";
    }

    private void fun1() {
        fun2();
    }

    private void fun2() {
        fun3();
    }

    private void fun3() {
        int cnt = Runtime.getRuntime().availableProcessors() * 2;
        ExecutorService executorService = Executors.newFixedThreadPool(cnt);
        for (int i = 0; i < cnt; i++) {
            MyThread thread = new MyThread();
            executorService.submit(thread);
        }
    }
}

class MyThread extends Thread {
    @Override
    public void run() {
        while (true) {

        }
    }
}

查找

查找CPU、内存等情况，用dashboard命令。
在我们还没有访问地址的时候，我们可以看到CPU的使用率极低。

ID NAME                GROUP     PRIORI STATE  %CPU  DELTA_ TIME   INTER DAEMON 
-1 C2 CompilerThread0  -         -1     -      0.0   0.000  0:3.26 false true   
-1 C2 CompilerThread1  -         -1     -      0.0   0.000  0:3.00 false true   
29 DestroyJavaVM       main      5      RUNNAB 0.0   0.000  0:2.09 false false  
-1 C1 CompilerThread2  -         -1     -      0.0   0.000  0:1.38 false true   
-1 VM Thread           -         -1     -      0.0   0.000  0:0.09 false true   
-1 GC task thread#0 (P -         -1     -      0.0   0.000  0:0.07 false true   
-1 GC task thread#2 (P -         -1     -      0.0   0.000  0:0.07 false true   
-1 GC task thread#1 (P -         -1     -      0.0   0.000  0:0.06 false true   
42 arthas-NettyHttpTel system    5      RUNNAB 0.0   0.000  0:0.06 false true   
-1 GC task thread#3 (P -         -1     -      0.0   0.000  0:0.06 false true   
Memory           used  total max  usage GC                                      
heap             84M   264M       4.88%                     6                   
ps_eden_space    73M   175M  632M       gc.ps_scavenge.time 48                  
ps_survivor_spac 0K    10240 1024 0.00% (ms)                                    
e                      K     0K         gc.ps_marksweep.cou 2                   
ps_old_gen       10M   79M        0.83% nt                                      
nonheap          53M   63M   -1         gc.ps_marksweep.tim 88                  
code_cache       4M    12M   240M 2.07% e(ms)                                   
Runtime                                                                         
os.name                                 Linux                                   
os.version                              3.10.0-1160.6.1.el7.x86_64              
java.version                            1.8.0_275    

当我们访问http://192.168.0.101:8080/cpu后，查看面板内容如下，CPU已经到50%了。

ID NAME                GROUP     PRIORI STATE  %CPU  DELTA_ TIME   INTER DAEMON 
53 pool-1-thread-4     main      5      RUNNAB 50.74 2.538  0:18.4 false false  
49 pool-1-thread-2     main      5      RUNNAB 50.47 2.524  0:17.7 false false  
55 pool-1-thread-5     main      5      RUNNAB 50.41 2.521  0:17.8 false false  
47 pool-1-thread-1     main      5      RUNNAB 49.79 2.490  0:17.6 false false  
51 pool-1-thread-3     main      5      RUNNAB 49.61 2.481  0:17.7 false false  
61 pool-1-thread-8     main      5      RUNNAB 49.6  2.481  0:17.5 false false  
57 pool-1-thread-6     main      5      RUNNAB 49.39 2.470  0:17.3 false false  
59 pool-1-thread-7     main      5      RUNNAB 49.36 2.469  0:17.6 false false  
-1 C1 CompilerThread2  -         -1     -      0.11  0.005  0:1.76 false true   
45 Timer-for-arthas-da system    5      RUNNAB 0.04  0.002  0:0.02 false true   
Memory           used  total max  usage GC                                      
heap             115M  264M       6.67%                     6                   
ps_eden_space    105M  175M  632M       gc.ps_scavenge.time 48                  
ps_survivor_spac 0K    10240 1024 0.00% (ms)                                    
e                      K     0K         gc.ps_marksweep.cou 2                   
ps_old_gen       10M   79M        0.83% nt                                      
nonheap          57M   65M   -1         gc.ps_marksweep.tim 88                  
code_cache       7M    12M   240M 3.17% e(ms)                                   
Runtime                                                                         
os.name                                 Linux                                   
os.version                              3.10.0-1160.6.1.el7.x86_64              
java.version                            1.8.0_275                       

ID为53的，是Java级别的线程ID，我们用thread命令查看。

[arthas@16571]$ thread 53
"pool-1-thread-4" Id=53 RUNNABLE
    at com.dajun.arthas.controller.MyThread.run(CpuController.java:38)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

他这里的结果显示了CpuController.java:38这行代码，就是我们上面的while(true)代码，我们就可以通过找到的代码进行处理CPU飙升的故障。