Submitting a Task Using the Yarn Client

Scenario

This section guides users to use a Yarn client in an O&M or service scenario.

Prerequisites

• You have installed the client.

  For example, the installation directory is /opt/client. The client directory in the following operations is only an example. Change it based on the actual installation directory onsite.

• Service users of each component are created by the MRS cluster administrator based on service requirements. In security mode, machine-machine users need to download the keytab file. A human-machine user must change the password upon the first login. In common mode, you do not need to download the keytab file or change the password.

Using the Yarn Client

1. Install a client.
   • For versions earlier than MRS 3.x, see Installing a Client.
   • For MRS 3.x or later, see Installing a Client.

2. Log in to the node where the client is installed as the client installation user.
3. Run the following command to go to the client installation directory:

   cd /opt/client

4. Run the following command to configure environment variables:

   source bigdata_env

5. If the cluster is in security mode, run the following command to authenticate the user. In normal mode, user authentication is not required.

   kinit Component service user

6. Run the Yarn command. The following provides an example:

   yarn application -list

Client-related FAQs

• Question 1: What do I do when the HDFS client exits abnormally and error message "java.lang.OutOfMemoryError" is displayed during client command execution?

  Memory required for running the Yarn client exceeds the upper limit (128 MB by default) set on the Yarn client.

  1. You can change the maximum memory of the Yarn client by modifying parameters in <Client installation path>/HDFS/component_env.
     • For MRS 3.x and later versions, modify the CLIENT_GC_OPTS parameter. For example, to set the maximum memory to 1 GB, run the following command:

       export CLIENT_GC_OPTS="-Xmx1G"

     • For MRS 3.x and later versions, modify the GC_OPTS_YARN parameter. For example, to set the maximum memory to 1 GB, run the following command:

       export GC_OPTS_YARN="-Xmx1G"

  2. After the modification, run the following command to make the modification take effect:

     source <Client installation path>//bigdata_env

• Question 2: How do I set the run log level of the Yarn client?

  By default, the logs generated during the running of the Yarn client are printed to the console. The default log level is INFO. To enable the DEBUG log level for fault locating, run the following command to export an environment variable:

  export YARN_ROOT_LOGGER=DEBUG,console

  Then run the Yarn Shell command to print DEBUG logs.

  If you want to print INFO logs again, run the following command:

  export YARN_ROOT_LOGGER=INFO,console