Installing a Client (MRS 3.x or Later)

Scenario

Install clients of all services (excluding Flume) in an MRS cluster. For details about how to install the Flume client, see Installing the Flume Client.

You can install the clients on a node in or outside the cluster.

After modifying the server configuration of a cluster component, reinstall the component client to ensure the server and the client both run the same version to provide services properly.

Video Tutorial

This tutorial uses MRS 3.1.0 as an example to describe how to install and use a client.

The UI may vary depending on the version. This tutorial is for reference only.

Prerequisites

• If the node where the client is to be installed is outside the cluster, the node must be able to communicate with the nodes in the cluster. Otherwise, client installation will fail.
• The node where the client is to be installed must have the NTP service enabled and synchronized time with the server. Otherwise, client installation will fail.
• You install the client as user root or any OS user. The user must have the operation permission on the client file storage directory and installation directory. The permission on the two directories is 755.

  This section uses the OS user user_client as an example to describe how to install the client in the /opt/hadoopclient directory.

• When you install the client as a user other than omm or root, and the /var/tmp/patch directory already exists, you have changed the permission for the directory to 777 and changed the permission for the logs in the directory to 666.

Installing a Client on a Node Inside a Cluster

1. Obtain the client software package.
   Log in to FusionInsight Manager by referring to Accessing FusionInsight Manager (MRS 3.x or Later). On the Cluster > Dashboard page, click the more sign (...) and select Download Client. In the Download Cluster Client dialog box displayed, configure parameters and click OK.

   Figure 1 Downloading a client
   

   

   • The client software package downloaded from the FusionInsight Manager homepage contains the clients of all services (excluding Flume) in the cluster. To download the client of a single service, choose Cluster > Services > Service name, click More, and select Download Client.
   • For MRS 3.3.0 or later, click Download Client on the home page.

   Table 1 Client download parameters

   Parameter	Description	Example Value
   Select Client Type	Complete Client: contains the complete client software package and configuration files, which applies to non-development task scenarios. Configuration Files Only: downloads only client configuration files in the scenario where the administrator modifies the server configuration on FusionInsight Manager after the complete client is downloaded and installed in an application development task, and developers need to update client configuration files.	Complete Client
   Select Platform Type	The client type must match the architecture of the node where the client is to be installed. Otherwise, the installation fails. For clusters of the LTS version, only the client software package whose type is the same as that of FusionInsight Manager can be downloaded. x86_64: indicates the client software package that can be deployed on a x86 platform. aarch64: indicates the client software package that can be deployed on a TaiShan server.	x86_64
   Save to Path	The path for storing the client software package on the active OMS node Select Save to Path: Customize the path for storing the client software package on the active OMS node. User omm must have the read, write, and execute permissions on the path. If the path is not changed, the client file generated is saved in the /tmp/FusionInsight-Client directory on the active OMS node in the cluster by default. Not to select Save to Path: The generated client file is automatically downloaded and saved to the local host. Before installing the client, you need to upload the file to a specified directory on the target node.	Select Save to Path

2. Copy the client software package to a specified directory on the node where the client is to be installed.

   By default, the client software package is stored on the active OMS node in the cluster. To install the client on other nodes in the cluster, log in to the active OMS node as user omm and run the following command to copy the software package to the specified node. Otherwise, skip this step.

   For example, copy the software package to the /tmp/clienttemp directory.

   scp -p /tmp/FusionInsight-Client/FusionInsight_Cluster_1_Services_Client.tar IP address of the node where the client is to be installed:/tmp/clienttemp

3. Log in to the target node as a user, for example, user_client.
   

   You can install the client as user root or any other OS user. The user must have the operation permission on the client file storage directory and installation directory. The permission on the two directories is 755.

4. Decompress the client software package.
   1. Go to the directory where the package is stored, for example, /tmp/clienttemp.

      cd /tmp/clienttemp

   2. Run the following commands to decompress the package and obtain FusionInsight_Cluster_1_Services_ClientConfig.tar:

      tar -xvf FusionInsight_Cluster_1_Services_Client.tar

   3. Run the following command to verify the decompressed file and check whether the command output is consistent with the information in the sha256 file.

      sha256sum -c FusionInsight_Cluster_1_Services_ClientConfig.tar.sha256

      FusionInsight_Cluster_1_Services_ClientConfig.tar: OK     

   4. Run the following command to decompress FusionInsight_Cluster_1_Services_ClientConfig.tar:

      tar -xvf FusionInsight_Cluster_1_Services_ClientConfig.tar

5. Go to the package directory and run the following command to install the client to a specified directory:

   cd FusionInsight_Cluster_1_Services_ClientConfig

   ./install.sh Client installation directory

   For example, run the ./install.sh /opt/hadoopclient command to install the client and wait until the installation is complete.

   ...
   The component client is installed successfully

   

   • If there is no specified client installation directory, it will be automatically created. If you specify an existing directory, it must be empty, and the path cannot contain spaces. The value can contain only uppercase letters, lowercase letters, digits, and underscores (_).
   • You need to manually delete the client installation directory when uninstalling a client.
   • To ensure that the installed client can be used only by the installation user, add the -o parameter during the installation. For example, run the ./install.sh /opt/hadoopclient -o command to install the client.

6. Use the client by referring to "Using the client of Each Component".

Installing a Client on a Node Outside a Cluster

1. Create an ECS that meets the following requirements:
   • A Linux ECS has been prepared. For details about the supported OS of the ECS, see Table 2.

     Table 2 Reference list

     CPU Architecture	OS	Supported Version
     x86 computing	EulerOS	EulerOS 2.5
     	SUSE	SUSE Linux Enterprise Server 12 SP4 (SUSE 12.4)
     	Red Hat	Red Hat-7.5-x86_64 (Red Hat 7.5)
     	CentOS	CentOS 7.6
     Kunpeng computing (Arm)	EulerOS	EulerOS 2.8
     	CentOS	CentOS 7.6

     In addition, sufficient disk space is allocated for the ECS, for example, 40 GB.

   • The ECS and the MRS cluster are in the same VPC.
   • The security group of the ECS must be the same as that of the master node in the MRS cluster.
   • The NTP service has been installed on the ECS OS and is running properly.

     If the NTP service is not installed, run the yum install ntp -y command to install it when the yum source is configured.

   • A user can log in to the Linux ECS using the password (in SSH mode).
   • All ports in the inbound direction of the MRS cluster security group are open to the client node. For details, see Adding a Security Group Rule.

2. Perform NTP time synchronization to synchronize the time of nodes outside the cluster with the time of the MRS cluster.
   1. Run the vi /etc/ntp.conf command to edit the NTP client configuration file, add the IP addresses of the master node in the MRS cluster, and comment out the IP address of other servers.

      server master1_ip prefer
      server master2_ip 

      Figure 2 Adding the master node IP addresses
      
   2. Run the service ntpd stop command to stop the NTP service.
   3. Run the following command to manually synchronize the time:

      /usr/sbin/ntpdate 192.168.10.8

      

      192.168.10.8 indicates the IP address of the active Master node.

   4. Run the service ntpd start or systemctl restart ntpd command to start the NTP service.
   5. Run the ntpstat command to check the time synchronization result.

3. Obtain the client software package.
   Log in to FusionInsight Manager by referring to Accessing FusionInsight Manager (MRS 3.x or Later). On the Cluster > Dashboard page, click the more sign (...) and select Download Client. In the Download Cluster Client dialog box displayed, configure parameters and click OK.

   Figure 3 Downloading a client
   

   

   • The client software package downloaded from the FusionInsight Manager homepage contains the clients of all services (excluding Flume) in the cluster. To download the client of a single service, choose Cluster > Services > Service name, click More, and select Download Client.
   • For MRS 3.3.0 or later, click Download Client on the home page.

   Table 3 Client download parameters

   Parameter	Description	Example Value
   Select Client Type	Complete Client: contains the complete client software package and configuration files, which applies to non-development task scenarios. Configuration Files Only: downloads only client configuration files in the scenario where the administrator modifies the server configuration on FusionInsight Manager after the complete client is downloaded and installed in an application development task, and developers need to update client configuration files.	Complete Client
   Select Platform Type	The client type must match the architecture of the node where the client is to be installed. Otherwise, the installation fails. For clusters of the LTS version, only the client software package whose type is the same as that of FusionInsight Manager can be downloaded. x86_64: indicates the client software package that can be deployed on a x86 platform. aarch64: indicates the client software package that can be deployed on a TaiShan server.	x86_64
   Save to Path	The path for storing the client software package on the active OMS node Select Save to Path: Customize the path for storing the client software package on the active OMS node. User omm must have the read, write, and execute permissions on the path. If the path is not changed, the client file generated is saved in the /tmp/FusionInsight-Client directory on the active OMS node in the cluster by default. Not to select Save to Path: The generated client file is automatically downloaded and saved to the local host. Before installing the client, you need to upload the file to a specified directory on the target node.	Select Save to Path

4. Copy the client software package to a specified directory on the node where the client is to be installed.

   The generated client software package is stored on the active OMS node of the cluster by default. You need to log in to the active OMS node as user omm and run the following command to copy the software package to a specified ECS:

   For example, copy the software package to the /tmp/clienttemp directory.

   scp -p /tmp/FusionInsight-Client/FusionInsight_Cluster_1_Services_Client.tar IP address of the node where the client is to be installed:/tmp/clienttemp

5. Log in to the target node as a user, for example, user_client.
   

   You can install the client as user root or any other OS user. The user must have the operation permission on the client file storage directory and installation directory. The permission on the two directories is 755.

6. Decompress the client software package.
   1. Go to the directory where the package is stored, for example, /tmp/clienttemp.

      cd /tmp/clienttemp

   2. Run the following commands to decompress the package and obtain FusionInsight_Cluster_1_Services_ClientConfig.tar:

      tar -xvf FusionInsight_Cluster_1_Services_Client.tar

   3. Run the following command to verify the decompressed file and check whether the command output is consistent with the information in the sha256 file.

      sha256sum -c FusionInsight_Cluster_1_Services_ClientConfig.tar.sha256

      FusionInsight_Cluster_1_Services_ClientConfig.tar: OK     

   4. Run the following command to decompress FusionInsight_Cluster_1_Services_ClientConfig.tar:

      tar -xvf FusionInsight_Cluster_1_Services_ClientConfig.tar

7. Check the network connection of the client.
   1. Ensure that the host where the client is installed can communicate with the hosts listed in the hosts file stored in the directory containing the decompressed package, for example, /tmp/FusionInsight_Cluster_1_Services_ClientConfig/hosts.
   2. If the host where the client is installed is not a host in the cluster, you need to set the mapping between the host name and the service plane IP address for each cluster node in /etc/hosts, as user root. Each host name uniquely maps an IP address. You can perform the following steps to import the domain name mapping of the cluster to the hosts file:
      1. Switch to user root or a user who has the permission to modify the hosts file.

         su - root

      2. Go to the directory where the client package is decompressed.

         cd /tmp/clienttemp/FusionInsight_Cluster_1_Services_ClientConfig

      3. Run the cat realm.ini >> /etc/hosts command to import the domain name mapping to the hosts file.
   

   • If the host where the client is installed is not a node in the cluster, configure network connections for the client to prevent errors when you run commands on the client.
   • If Spark tasks are executed in yarn-client mode, add the spark.driver.host parameter to the file Client installation directory/Spark/spark/conf/spark-defaults.conf and set the parameter to the client IP address.
   • If the yarn-client mode is used, you need to configure the mapping between the IP address and host name of the client in the hosts file on the active and standby Yarn nodes (ResourceManager nodes in the cluster) to make sure that the Spark web UI is properly displayed.

8. Log in to the node where the client is to be installed as user user_client, go to the client software package directory, and run the following command to install the client to a specified directory:

   cd /tmp/clienttemp/FusionInsight_Cluster_1_Services_ClientConfig

   ./install.sh Client installation directory

   For example, run the ./install.sh /opt/hadoopclient command to install the client and wait until the installation is complete.

   ...
   The component client is installed successfully

   

   • If there is no specified client installation directory, it will be automatically created. If you specify an existing directory, it must be empty, and the path cannot contain spaces. The value can contain only uppercase letters, lowercase letters, digits, and underscores (_).
   • You need to manually delete the client installation directory when uninstalling a client.
   • To ensure that the installed client can be used only by the installation user, add the -o parameter during the installation. For example, run the ./install.sh /opt/hadoopclient -o command to install the client.

9. Use the client by referring to "Using the client of Each Component".