hadoop3.1.3的Yarn常用命令yarn node查看节点状态

Posted 2022-06-15 爱上口袋的天空

1、yarn application查看任务

Yarn状态的查询，除了可以在hadoop21:8088页面查看外，还可以通过命令操作。常见的命令操作如下所示：

1.1、列出所有Application：

[kgf@hadoop21 hadoop-3.1.3]$ yarn application -list
2022-06-14 14:21:19,003 INFO client.RMProxy: Connecting to ResourceManager at hadoop21/192.168.56.21:8032
Total number of applications (application-types: [], states: [SUBMITTED, ACCEPTED, RUNNING] and tags: []):0
                Application-Id      Application-Name        Application-Type          User           Queue                   State             Final-State             Progress                        Tracking-URL
[kgf@hadoop21 hadoop-3.1.3]$

1.2、根据Application状态过滤：yarn application -list -appStates （所有状态：ALL、NEW、NEW_SAVING、SUBMITTED、ACCEPTED、RUNNING、FINISHED、FAILED、KILLED）

[kgf@hadoop21 hadoop-3.1.3]$ yarn application -list -appStates FINISHED
2022-06-14 14:21:54,965 INFO client.RMProxy: Connecting to ResourceManager at hadoop21/192.168.56.21:8032
Total number of applications (application-types: [], states: [FINISHED] and tags: []):1
                Application-Id      Application-Name        Application-Type          User           Queue                   State             Final-State             Progress                        Tracking-URL
application_1655215095011_0001            word count               MAPREDUCE           kgf         default                FINISHED               SUCCEEDED                 100% http://hadoop20:19888/jobhistory/job/job_1655215095011_0001
[kgf@hadoop21 hadoop-3.1.3]$

1.3、Kill掉Application

[kgf@hadoop21 hadoop-3.1.3]$ yarn application -kill application_1655215095011_0001
2022-06-14 14:22:48,333 INFO client.RMProxy: Connecting to ResourceManager at hadoop21/192.168.56.21:8032
Application application_1655215095011_0001 has already finished
[kgf@hadoop21 hadoop-3.1.3]$

2、yarn logs查看日志

2.1、查询Application日志：yarn logs -applicationId <ApplicationId>

[kgf@hadoop21 hadoop-3.1.3]$ yarn logs -applicationId application_1655215095011_0001

2.2、查询Container日志：yarn logs -applicationId <ApplicationId> -containerId <ContainerId>

[kgf@hadoop21 hadoop-3.1.3]$ yarn logs -applicationId application_1655215095011_0001 -containerId container_1655215095011_0001_01_000001

3、yarn applicationattempt查看尝试运行的任务

3.1、列出所有Application尝试的列表：yarn applicationattempt -list <ApplicationId>

[kgf@hadoop21 hadoop-3.1.3]$ yarn applicationattempt -list application_1655215095011_0001
2022-06-14 14:31:50,092 INFO client.RMProxy: Connecting to ResourceManager at hadoop21/192.168.56.21:8032
Total number of application attempts :1
         ApplicationAttempt-Id                 State                        AM-Container-Id                            Tracking-URL
appattempt_1655215095011_0001_000001                FINISHED    container_1655215095011_0001_01_000001  http://hadoop21:8088/proxy/application_1655215095011_0001/
[kgf@hadoop21 hadoop-3.1.3]$

3.2、打印ApplicationAttemp状态：yarn applicationattempt -status <ApplicationAttemptId>

[kgf@hadoop21 hadoop-3.1.3]$ yarn applicationattempt -status appattempt_1655215095011_0001_000001
2022-06-14 14:41:30,992 INFO client.RMProxy: Connecting to ResourceManager at hadoop21/192.168.56.21:8032
Application Attempt Report :
        ApplicationAttempt-Id : appattempt_1655215095011_0001_000001
        State : FINISHED
        AMContainer : container_1655215095011_0001_01_000001
        Tracking-URL : http://hadoop21:8088/proxy/application_1655215095011_0001/
        RPC Port : 44008
        AM Host : hadoop22
        Diagnostics :
[kgf@hadoop21 hadoop-3.1.3]$

4、yarn container查看容器

4.1、列出所有Container：yarn container -list <ApplicationAttemptId>

[kgf@hadoop21 hadoop-3.1.3]$ yarn container -list appattempt_1655215095011_0001_000001
2022-06-14 14:45:20,569 INFO client.RMProxy: Connecting to ResourceManager at hadoop21/192.168.56.21:8032
Total number of containers :0
                  Container-Id            Start Time             Finish Time                   State                    Host       Node Http Address                                LOG-URL
[kgf@hadoop21 hadoop-3.1.3]$

4.2、打印Container状态： yarn container -status <ContainerId>

[atguigu@hadoop102 hadoop-3.1.3]$ yarn container -status container_1612577921195_0001_01_000001
2021-02-06 10:29:58,554 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032
Container with id 'container_1612577921195_0001_01_000001' doesn't exist in RM or Timeline Server.

注：只有在任务跑的途中才能看到container的状态

5、yarn node查看节点状态 

5.1、列出所有节点：yarn node -list -all

[kgf@hadoop21 hadoop-3.1.3]$ yarn node -list -all
2022-06-14 14:47:23,197 INFO client.RMProxy: Connecting to ResourceManager at hadoop21/192.168.56.21:8032
Total Nodes:3
         Node-Id             Node-State Node-Http-Address       Number-of-Running-Containers
  hadoop21:37105                RUNNING     hadoop21:8042                                  0
  hadoop22:41546                RUNNING     hadoop22:8042                                  0
  hadoop20:43215                RUNNING     hadoop20:8042                                  0
[kgf@hadoop21 hadoop-3.1.3]$

6、yarn queue查看队列

打印队列信息：yarn queue -status <QueueName>

[kgf@hadoop21 hadoop-3.1.3]$ yarn queue -status default
2022-06-14 14:48:36,392 INFO client.RMProxy: Connecting to ResourceManager at hadoop21/192.168.56.21:8032
Queue Information :
Queue Name : default
        State : RUNNING
        Capacity : 100.0%
        Current Capacity : .0%
        Maximum Capacity : 100.0%
        Default Node Label expression : <DEFAULT_PARTITION>
        Accessible Node Labels : *
        Preemption : disabled
        Intra-queue Preemption : disabled
[kgf@hadoop21 hadoop-3.1.3]$