Command
You can use YARN commands to perform operations on YARN clusters, such as starting ResourceManager, submitting applications, killing applications, querying node status, and downloading container logs.
For details about the commands, see http://hadoop.apache.org/docs/r3.1.1/hadoop-yarn/hadoop-yarn-site/YarnCommands.html
Common Commands
YARN commands can be used by common users and administrators. Common users can run a few of YARN commands, such as jar and logs. Administrators have the permission to run most YARN commands.
Users can run the following command to query usage of YARN.
yarn --help
Usage: Go to any directory on the YARN client, execute source command to import the environment varables, and run the command. The command format is as follows.
yarn [--config confdir] COMMAND
The version 8.0.2.1 is used as an example. Replace it with the actual version number.
|
Command |
Description |
|---|---|
|
resourcemanager |
Runs a ResourceManager. Notice: Before running commands, for example, the following two commands, on the server as user omm (the client must have the execute permission of user omm), export environment variables first.
|
|
nodemanager |
Runs a NodeManager. Notice: Before running commands, for example, the following two commands, on the server as user omm (the client must have the execute permission of user omm), export environment variables first.
|
|
rmadmin |
Runs administrator's tool for dynamically updating information. |
|
version |
Displays the version. |
|
jar <jar> |
Runs a JAR file. |
|
logs |
Obtains container logs. |
|
classpath |
Displays the class path of the Hadoop JAR package and other library files. |
|
daemonlog |
Obtains or sets the service log level. |
|
CLASSNAME |
Runs a class named by CLASSNAME. |
|
top |
Runs the cluster usage monitor tool. |
|
-Dmapreduce.job.hdfs-servers |
If OBS is connected, but the server still uses HDFS, you need to explicitly use this parameter in the command line to specify the HDFS address. The format is hdfs://{NAMESERVICE}. Replace NAMESERVICE with the HDFS NameService name. If the current HDFS has multiple NameService instances, you need to specify all NameService instances and separate them with commas (,), for example, hdfs://nameservice1,hdfs://nameservice2. |
Superior Scheduler Command
Superior Scheduler Engine provides a CLI that display detail information of Superior Scheduler Engine. For executing superior command we need to use the "<HADOOP_HOME>/bin/superior" script.
Here is the format of "superior" command.
<HADOOP_HOME>/bin/superior Usage: superior [COMMAND | -help] Where COMMAND is one of: resourcepool prints resource pool status queue prints queue status application prints application status policy prints policy status
Most commands print help when invoked without parameters.
- Superior resourcepool command:
This command displays resourcepool and associated policy related status and configuration information.
Superior resourcepool command can be used only by admin user and user with yarn admin rights.
Usage output:
>superior resourcepool Usage: resourcepool [-help] [-list] [-status <resourcepoolname>] -help prints resource pool usage -list prints all resource pool summary report -status <resourcepoolname> prints status and configuration of specified resource pool- resourcepool -list prints resource pool summary in a table format. Here is an example:
> superior resourcepool -list NAME NUMBER_MEMBER TOTAL_RESOURCE AVAILABLE_RESOURCE Pool1 4 vcores 30,memory 1000 vcores 21,memory 80 Pool2 100 vcores 100,memory 12800 vcores 30,memory 1000 default 2 vcores 64,memory 128 vcores 40,memory 28
- resourcepool -status <resourcepoolname> prints resource pool detail information in a list format. Here is an example:
> superior resourcepool -status default NAME: default DESCRIPTION: System generated resource pool TOTAL_RESOURCE: vcores 64,memory 128 AVAILABLE_RESOURCE: vcores 40,memory 28 NUMBER_MEMBER: 2 MEMBERS: node1,node2 CONFIGURATION: |-- RESOURCE_SELECT: |__RESOURCES:
- resourcepool -list prints resource pool summary in a table format. Here is an example:
- Superior queue command
This command displays hierarchy queue information.
Usage output:
>superior queue Usage: queue [-help] [-list] [-e] [[-name <queue_name>] [-r|-c]] [-status <queue_name>] -c only work with -name <queue_name> option. If this option is used, command will print information of specified queue and its direct children. -e only work with -list or -list -name option. If this option is used, command will print effective state of specified queue and all of its descendants. -help prints queue sub command usage -list prints queue summary report. This option can work with -name <queue_name> and -r options. -name <queue_name> print specified queue, this can work with -r option. By default, it will print queue's own information. When -r is defined, command will print all of its descendant queues. When -c is defined, it will print its direct children queues. -r only work with -name <queue_name> option. If this option is used, command will print information of specified queue and all of its descendants. -status <queue_name> prints status of specified queue- queue -list prints a table format of queue summary information. When command displays, command will display them based on queue hierarchy fashion. With SUBMIT ACL or ADMIN ACL rights for queue, user will be able to see queues. Here is an example:
> superior queue -list NAME STATE NRUN_APP NPEND_APP NRUN_CONTAINER NPEND_REQUEST RES_INUSE RES_REQUEST root OPEN|ACTIVE 10 20 100 200 vcores 100,memory 1000 vcores 200,memory 2000 root.Q1 OPEN|ACTIVE 5 10 50 100 vcores 50,memory 500 vcores 100,memory 1000 root.Q1.Q11 OPEN|ACTIVE 5 10 50 100 vcores 50,memory 500 vcores 100,memory 1000 root.Q1.Q12 CLOSE|INACTIVE 0 0 0 0 vcores 0,memory 0 vcores 0,memory 0 root.Q2 OPEN|INACTIVE 5 10 50 100 vcores 50,memory 500 vcores 100,memory 1000 root.Q2.Q21 OPEN|ACTIVE 5 10 50 100 vcores 50,memory 500 vcores 100,memory 1000
- queue -list -name root.Q1 will display root.Q1 only.
> superior queue -list -name root.Q1 NAME STATE NRUN_APP NPEND_APP NRUN_CONTAINER NPEND_REQUEST RES_INUSE RES_REQUEST root.Q1 OPEN|ACTIVE 5 10 50 100 vcores 50,memory 500 vcores 100,memory 1000
- queue -list -name root.Q1 -r will print out root.Q1 and all of its descendents.
> superior queue -list -name root.Q1 -r NAME STATE NRUN_APP NPEND_APP NRUN_CONTAINER NPEND_REQUEST RES_INUSE RES_REQUEST root.Q1 OPEN|ACTIVE 5 10 50 100 vcores 50,memory 500 vcores 100,memory 1000 root.Q1.Q11 OPEN|ACTIVE 5 10 50 100 vcores 50,memory 500 vcores 100,memory 1000 root.Q1.Q12 CLOSE|INACTIVE 0 0 0 0 vcores 0,memory 0 vcores 0,memory 0
- queue -list -name root -c will print out root and all of its direct children
> superior queue -list -name root -c NAME STATE NRUN_APP NPEND_APP NRUN_CONTAINER NPEND_REQUEST RES_INUSE RES_REQUEST root OPEN|ACTIVE 10 20 100 200 vcores 100,memory 1000 vcores 200,memory 2000 root.Q1 OPEN|ACTIVE 5 10 50 100 vcores 50,memory 500 vcores 100,memory 1000 root.Q2 OPEN|INACTIVE 5 10 50 100 vcores 50,memory 500 vcores 100,memory 1000
- queue -status <queue_name> will display detail queue status and configuration.
With SUBMIT ACL, user will be able to see details except ACLS of queue.
User with ADMIN ACL rights for queue will be able to see queue details including ACL.
> superior queue -status root.Q1 NAME: root.Q1 OPEN_STATE:CLOSED ACTIVE_STATE: INACTIVE EOPEN_STATE: CLOSED EACTIVE_STATE: INACTIVE LEAF_QUEUE: Yes NUMBER_PENDING_APPLICATION: 100 NUMBER_RUNNING_APPLICATION: 10 NUMBER_PENDING_REQUEST: 10 NUMBER_RUNNING_CONTAINER: 10 NUMBER_RESERVED_CONTAINER: 0 RESOURCE_REQUEST: vcores 3,memory 3072 RESOURCE_INUSE: vcores 2,memory 2048 RESOURCE_RESERVED: vcores 0,memory 0 CONFIGURATION: |-- DESCRIPTION: Spark session queue |-- MAX_PENDING_APPLICATION: 10000 |--MAX_RUNNING_APPLICATION: 1000 |--ALLOCATION_ORDER_POLICY: FIFO |--DEFAULT_RESOURCE_SELECT: label1 |--MAX_MASTER_SHARE: 10% |--MAX_RUNNING_APPLICATION_PER_USER : -1 |--MAX_ALLOCATION_UNIT: vcores 32,memory 12800 |--ACL_USERS: user1,user2 |--ACL_USERGROUPS: usergroup1,usergroup2 |-- ACL_ADMINS: user1 |--ACL_ADMINGROUPS: usergroup1
- queue -list prints a table format of queue summary information. When command displays, command will display them based on queue hierarchy fashion. With SUBMIT ACL or ADMIN ACL rights for queue, user will be able to see queues. Here is an example:
- Superior application command
This command displays application related information.
Usage output:
>superior application Usage: application [-help] [-list] [-status <application_id>] -help prints application sub command usage -list prints all application summary report -status <application_id> prints status of specified applicationWith the view access rights to application user can see application related information.
- application -list provides summary information of all application in a table format. Here is an example:
> superior application -list ID QUEUE USER NRUN_CONTAINER NPEND_REQUEST NRSV_CONTAINER RES_INUSE RES_REQUEST RES_RESERVED application_1482743319705_0005 root.SEQ.queueB hbase 1 100 0 vcores 1,memory 1536 vcores 2000,memory 409600 vcores 0,memory 0 application_1482743319705_0006 root.SEQ.queueB hbase 0 1 0 vcores 0,memory 0 vcores 1,memory 1536 vcores 0,memory 0
- application -status <app_id> command displays detail information of specified application. Here is an example:
> superior application -status application_1443067302606_0609 ID: application_1443067302606_0609 QUEUE: root.Q1.Q11 USER: cchen RESOURCE_REQUEST: vcores 3,memory 3072 RESOURCE_INUSE: vcores 2,memory 2048 RESOURCE_RESERVED:vcores 1, memory 1024 NUMBER_RUNNING_CONTAINER: 2 NUMBER_PENDING_REQUEST: 3 NUMBER_RESERVED_CONTAINER: 1 MASTER_CONTAINER_ID: application_1443067302606_0609_01 MASTER_CONTAINER_RESOURCE: node1.domain.com BLACKLIST: node5,node8 DEMANDS: |-- PRIORITY: 20 |-- MASTER: true |-- CAPABILITY: vcores 2, memory 2048 |-- COUNT: 1 |-- RESERVED_RES : vcores 1, memory 1024 |-- RELAXLOCALITY: true |-- LOCALITY: node1/1 |-- RESOURCESELECT: label1 |-- PENDINGREASON: "application limit reached" |-- ID: application_1443067302606_0609_03 |-- RESOURCE: node1.domain.com |-- RESERVED_RES: vcores 1, memory 1024 | |--PRIORITY: 1 |-- MASTER: false |-- CAPABILITY: vcores 1,memory 1024 |-- COUNT: 2 |-- RESERVED_RES: vcores 0, memory 0 |-- RELAXLOCLITY: true |--LOCALITY: node1/1,node2/1,rackA/2 |-- RESOURCESELECT: label1 |-- PENDINGREASON: "no available resource" CONTAINERS: |-- ID: application_1443067302606_0609_01 |-- RESOURCE: node1.domain.com |-- CAPABILITY: vcores 1,memory 1024 | |-- ID: application_1443067302606_0609_02 |-- RESOURCE: node2.domain.com |-- CAPABILITY: vcores 1,memory 1024
- application -list provides summary information of all application in a table format. Here is an example:
- Superior policy command
This command displays policy related information.
Superior policy command can be used only by admin user and user with yarn admin rights.
Usage output:
>superior policy Usage: policy [-help] [-list <resourcepoolname>] [-u] [-detail] [-status <resourcepoolname>] -detail only work with -list option to show a summary information of resource pool distribution on queues, including reserve, minimum and maximum -help prints policy sub command usage -list <resourcepoolname> prints a summary information of resource pool distribution on queue -status <resourcepoolname> prints pool distribution policy configuration and status of specified resource pool -u only work with -list option to show a summary information of resource pool distribution on queues and also user accounts- policy -list <resourcepoolname> print out a summary of queue distribution information. Here is an example:
>superior policy -list default NAME: default TOTAL_RESOURCE: vcores 16,memory 16384 AVAILABLE_RESOURCE: vcores 16,memory 16384 NAMERES_INUSERES_REQUEST root.defaultvcores 0,memory 0vcores 0,memory 0 root.productionvcores 0,memory 0vcores 0,memory 0 root.production.BU1vcores 0,memory 0vcores 0,memory 0 root.production.BU2 vcores 0,memory 0vcores 0,memory 0
- policy -list <resourcepoolname> -u prints out also user level summary information
> superior policy -list default -u NAME: default TOTAL_RESOURCE: vcores 16,memory 16384 AVAILABLE_RESOURCE: vcores 16,memory 16384 NAMERES_INUSERES_REQUEST root.defaultvcores 0,memory 0vcores 0,memory 0 root.default.[_others_]vcores 0,memory 0vcores 0,memory 0 root.productionvcores 0,memory 0vcores 0,memory 0 root.production.BU1vcores 0,memory 0vcores 0,memory 0 root.production.BU1.[_others_]vcores 0,memory 0vcores 0,memory 0 root.production.BU2vcores 0,memory 0vcores 0,memory 0 root.production.BU2.[_others_]vcores 0,memory 0vcores 0,memory 0
- policy -status <resourcepoolname> print out policy detail of specified resource pool. Here is an example:
> superior policy -status pool1 NAME: pool1 TOTAL_RESOURCE: vcores 64,memory 128 AVAILABLE_RESOURCE: vcores 40,memory 28 QUEUES: |-- NAME: root.Q1 |-- RESOURCE_USE: vcores 20, memory 1000 |-- RESOURCE_REQUEST: vcores 2,memory 100 |--RESERVE: vcores 10, memory 4096 |--MINIMUM: vcore 11, memory 4096 |--MAXIMUM: vcores 500, memory 100000 |--CONFIGURATION: |-- SHARE: 50% |-- RESERVE: vcores 10, memory 4096 |-- MINIMUM: vcores 11, memory 4096 |-- MAXIMUM: vcores 500, memory 100000 |-- QUEUES: |-- NAME: root.Q1.Q11 |-- RESOURCE_USE: vcores 15, memory, 500 |-- RESOURCE_REQUEST: vcores 1, memory 50 |-- RESERVE: vcores 0, memory 0 |-- MINIMUM: vcores 0, memory 0 |-- MAXIMUM: vcores -1, memory -1 |-- USER_ACCOUNTS: |-- NAME: user1 |-- RESOURCE_USE: vcores 1, memory 10 |-- RESOURCE_REQUEST: vcores 1, memory 50 | |-- NAME: OTHERS |--RESOURCE_USE: vcores 0, memory 0 |- RESOURCE_REQUEST: vcores 0, memory 0 |-- CONFIGURATION: |-- SHARE: 100% |-- USER_POLICY: |-- NAME: user1 |-- WEIGHT: 10 | |-- NAME: OTHERS |-- WEIGHT: 1 |-- MAXIMUM: vcores 10, memory 1000
- policy -list <resourcepoolname> print out a summary of queue distribution information. Here is an example:
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot
