Help Center/ MapReduce Service/ Developer Guide (Normal_Earlier Than 3.x)/ MapReduce Development Guide/ Preparing a MapReduce Application Development Environment/ Preparing a MapReduce Application Running Environment
Updated on 2024-08-16 GMT+08:00
View PDF
Share

Preparing a MapReduce Application Running Environment

The operating environment of MapReduce can be deployed in Linux. Perform the following operations to prepare the operating environment.

Procedure

  1. Ensure that the YARN and MapReduce components on the server have been installed and are running properly.
  2. The JDK 1.7 or 1.8 has been installed on the client.
  3. Ensure that the time difference between the client and the Hadoop cluster is less than 5 minutes.

    You can query the MRS cluster time by logging in to the active management node that corresponds to the cluster management IP address to run the date command.

  4. Download the MapReduce client program to the local computer.

    1. Log in to MRS Manager.

      Enter the address in the address box of your browser. The address format is https://Floating IP address of WebService of MRS Manager:8080/web. For example, enter https://10.10.10.172:8080/web.

    2. Choose Service > Download Client to download the client program to the local PC.

  5. Decompress the MRS_Services_Client.tar client program package. Because the installation package is in .tar format, run the following commands to decompress the package twice:

    tar -xvf /opt/MRS_Services_Client.tar

    tar -xvf /opt/MRS_Service_ClientConfig.tar

  6. Set environment variables for the operating environment. Assume that the installation package is decompressed in MRS_Services_ClientConfig/.

    Go to the decompressed folder and run the following command to install the client:

    sh install.sh {client_install_home}

  7. Go the client installation directory and run the following command to initialize the environment variables:

    source bigdata_env

  8. Copy the user.keytab and krb5.conf files downloaded in Preparing a MapReduce Application Development User to the Linux environment, for example, /opt/conf. For details, see Compiling and Running Applications.

    During the secondary development, the username used by PRINCIPAL must contain the domain name. For example, if you create a user named test and the domain name is HADOOP.COM, the username of PRINCIPAL is test@HADOOP.COM. The following is sample code.

    conf.set(PRINCIPAL, "test@HADOOP.COM");

  9. Run the kinit -kt /opt/conf/user.keytab test command.

    The path of the user.keytab file is the path for storing the configuration file on the Linux host. The test username can be changed to the name of the new user created in Preparing a MapReduce Application Development User.