Preparing a Linux Client Operating Environment

Prerequisites

1. You have installed HDFS on the server and confirmed that HDFS is running properly.

2. You have installed JDK 1.7 or 1.8 on the client operating environment.

3. You have obtained the MRS_Services_Client.tar client installation package.

Scenario

Install the client on Linux.

Procedure

1. Ensure that the time difference between the client and the Hadoop cluster is less than 5 minutes. You may need to manually modify the client or Hadoop cluster time.

   You can query the MRS cluster time by logging in to the active management node that corresponds to the cluster management IP address to run the date command.

2. Download the MapReduce client program to the local computer.
   1. Log in to .
   2. Choose Service > Download Client to download the client program to the local PC.

3. Decompress the MRS_Services_Client.tar client program package. Because the installation package is in .tar format, run the following commands to decompress the package twice:

   tar -xvf MRS_Services_Client.tar

   tar -xvf MRS_Service_ClientConfig.tar

4. Set environment variables for the operating environment. Assume that the installation package is decompressed in MRS_Services_ClientConfig/.

   Go to the decompressed folder and run the following command to install the client:

   sh install.sh {client_install_home}

5. Go the client installation directory and run the following command to initialize the environment variables:

   source bigdata_env

6. Copy the following files from the server to the conf directory in the same directory as the directory of the JAR file. For details about the JAR file exported by the sample project, see Compiling and Running an Application When a Client Is Installed.

   Table 1 Configuration files

   File Name	Function	How to Obtain
   core-site.xml	Configures HDFS parameters.	${HADOOP_HOME}/etc/hadoop/core-site.xml
   hdfs-site.xml	Configures HDFS parameters.	${HADOOP_HOME}/etc/hadoop/hdfs-site.xml
   user.keytab	Provides HDFS user information for Kerberos security authentication.	If the cluster is in security mode, contact the administrator to obtain the keytab and krb5 files corresponding to the account.
   krb5.conf	Contains Kerberos server configuration information.

   

   • In Table 1, ${HADOOP_HOME} indicates the Hadoop installation directory on the server.

   • Keytab authentication is valid for 24 hours. Re-authentication is required 24 hours later.
   • In the sample code, the username of PRNCIPAL_NAME must be the same as the account name of the keytab and krb5 files.
   • The user.keytab and krb5.conf of different clusters cannot be shared.
   • In the sample code, the keytab file used in System.getProperty("user.dir") + File.separator + "conf" + File.separator + "user.keytab" must be the same as the user's keytab file.
   • The log4j.properties file in the conf directory is configured based on the customer requirements.