Installing a Client

Scenario

Install the clients of all services, except Flume, in the MRS cluster. MRS provides shell scripts for different services so that maintenance personnel can log in to related maintenance clients and implement maintenance operations.

Reinstall the client after server configuration is modified on FusionInsight Manager or after the system is upgraded. Otherwise, the versions of the client and server will be inconsistent.

Prerequisites

A client installation directory will be automatically created if it does not exist. If the directory exists, it must be empty. The directory cannot contain any space.
If a server outside the cluster is used as the client node, the node can communicate with the cluster service plane. Otherwise, client installation will fail.
The client must have the NTP service enabled and synchronized time with the NTP server. Otherwise, client installation will fail.
If clients of all components are downloaded, HDFS and MapReduce are installed in the same directory (Client directory/HDFS/).
You can install and use the client as any user whose username and password have been obtained from the system administrator. This section uses user_client as an example. Ensure that user user_client is the owner of the server file directory (for example, /opt/Bigdata/hadoopclient) and client installation directory (for example, /opt/client). The permission for the two directories is 755.
You have obtained the component service username (a default user or new user) and password from the system administrator.
When you install the client as a user other than omm or root, and the /var/tmp/patch directory already exists, you have changed the permission for the directory to 777 and changed the permission for the logs in the directory to 666.

Procedure

Obtain the required software packages.

Log in to FusionInsight Manager.

In the upper right corner of Homepage, click Download Client. The Download Cluster Client page is displayed.

If you only need to install the client of a service in the cluster, choose Cluster > Services, click a service name, click More, and select Download Client. The Download Client page is displayed.
Set Select Client Type to Complete Client.

Configuration Files Only is to download client configuration files in the following scenario: After a complete client is downloaded and installed and the system administrator modifies server configurations on Manager, developers need to update the configuration files during application development.
- x86_64: indicates the client software package that can be deployed on the x86 servers.
- aarch64: indicates the client software package that can be deployed on the Kunpeng servers.
The cluster supports two types of clients: x86_64 and aarch64. The client type must match the architecture of the node for installing the client. Otherwise, client installation will fail.

Select the path for saving the downloaded client file.

You can directly download the client file to the node where the client is to be installed, or download the file to the active OMS node or local computer and copy it to the node where the client is to be installed.

Server: Download the file to the active OMS node of the cluster.
The generated file is stored in the /tmp/FusionInsight-Client directory on the active OMS node by default. You can also store the client file in other directories, and user omm has the read, write, and execute permissions on the directory. If the client file already exists in the path, the existing client file will be replaced.

When a cluster has many services installed, the cluster client file becomes quite large. Additionally, decompressing this file during installation can consume significant disk space. It's recommended to download the client files to a different directory that has ample space, or to promptly remove unnecessary files from the client download directory after installation. Doing so helps avoid exhausting the /tmp directory's disk space, which could interrupt the normal operation of the cluster nodes.

After the file is generated, copy the obtained package to another directory, for example, /opt/Bigdata/hadoopclient, as user omm or client installation user.
Browser: Download the file to the local computer.

Remote node: Download the file to a node other than the active OMS node. If you select this option, you need to set the following parameters:

**Table 1** Parameters
Parameter	Description	Example Value
Save to Path	Path for storing client files. If there is already a client file in the path, it will be overwritten. For a remote node, write permission for the path is required.	/tmp/FusionInsight-Client-Remote/
Host IP Address	IP address of the remote node. NOTE: The platform type of the remote node must be the same as that of the downloaded client. Otherwise, the client may fail to be installed.	x.x.x.x
Host Port	Host port of the remote node.	22
Username	Username for logging in to the remote node. For a remote node, write permission for the path is required.	xxx
Authentication Method	You can choose one of the following methods: Password: Use the password for login. SSH private keys: Use SSH private keys for login. None: To use this method, passwordless login needs to be enabled for the node.	Password
Password	This parameter is mandatory when Authentication Method is set to Password. This parameter indicates the password used for login.	xxx
SSH Private Keys	This parameter is mandatory when Authentication Method is set to SSH private keys. Click Select File and select a local file to upload.	-
Auto Deployment	Whether to enable auto deployment. This parameter is mandatory when Select Client Type is set to Complete Client. If you set this parameter to yes, the client is automatically installed and deployed on the current node. If you set this parameter to no, the client will not be automatically installed and deployed. You need to manually install the client after it is downloaded.	Yes
Deployment Path	This parameter is mandatory when Auto Deployment is set to Yes. If only the configuration file is downloaded, this parameter will not be displayed. The deployment path must be empty if it already exists on the remote node. Otherwise, it will be created automatically. The path also requires operate and write permissions.	/opt/testclient

Copy the obtained software package to the file directory (for example, /tmp/FusionInsight-Client) of the server where the client is to be installed as the user (for example, user_client) who is preparing to install the client.

The name of the client software package is in the following format: FusionInsight_Cluster_<Cluster ID>_Services_Client.tar.

The following steps and sections use FusionInsight_Cluster_1_Services_Client.tar as an example.

The host where the client is to be installed can be a node inside or outside the cluster. If the node is a server outside the cluster, it must be able to communicate with the cluster, and the NTP service must be enabled to ensure that the time is the same as that on the server.

For example, you can configure the same NTP clock source for external servers as that of the cluster. After the configuration, you can run the ntpq -np command to check whether the time is synchronized.

If there is an asterisk (*) before the IP address of the NTP clock source in the command output, the synchronization is normal. For example:

remote refid st t when poll reach delay offset jitter 
============================================================================== 
*10.10.10.162 .LOCL. 1 u 1 16 377 0.270 -1.562 0.014

If there is no asterisk (*) before the IP address of the NTP clock source and the value of refid is .INIT., or if the command output is abnormal, the synchronization is abnormal. Contact technical support.
```
remote refid st t when poll reach delay offset jitter 
============================================================================== 
10.10.10.162 .INIT. 1 u 1 16 377 0.270 -1.562 0.014
```

You can also configure the same chrony clock source for external servers as that for the cluster. After the configuration, run the chronyc sources command to check whether the time is synchronized.

In the command output, if there is an asterisk (*) before the IP address of the chrony service on the active OMS node, the synchronization is normal. For example:

MS Name/IP address         Stratum Poll Reach LastRx Last sample               
===============================================================================
^* 10.10.10.162             10  10   377   626    +16us[  +15us] +/-  308us

In the command output, if there is no asterisk (*) before the IP address of the NTP service on the active OMS node, and the value of Reach is 0, the synchronization is abnormal.

MS Name/IP address         Stratum Poll Reach LastRx Last sample               
===============================================================================
^? 10.1.1.1                      0  10     0     -     +0ns[   +0ns] +/-    0ns

Log in to the server where the client software package is located as user user_client.
Decompress the software package.

Go to the directory where the installation package is stored, such as /tmp/FusionInsight-Client. Run the following command to decompress the installation package to a local directory:

tar -xvf FusionInsight_Cluster_1_Services_Client.tar
Verify the software package.

Run the following command to verify the decompressed file and check whether the command output is consistent with the information in the sha256 file:

sha256sum -c FusionInsight_Cluster_1_Services_ClientConfig.tar.sha256
```
FusionInsight_Cluster_1_Services_ClientConfig.tar: OK     
```
Decompress the obtained installation file.

tar -xvf FusionInsight_Cluster_1_Services_ClientConfig.tar
Configure network connections for the client.
1. Ensure that the host where the client is installed can communicate with the hosts listed in the hosts file in the decompression directory (for example, /tmp/FusionInsight-Client/FusionInsight_Cluster_<Cluster ID>_Services_ClientConfig/hosts).
2. If the host where the client is installed is not a host in the cluster, you need to set the mapping between the host name and the service plane IP address for each cluster node in /etc/hosts, as user root. Each host name uniquely maps an IP address. You can perform the following steps to import the domain name mapping of the cluster to the hosts file:
  1. Switch to user root or a user who has the permission to modify the hosts file.
    su - root
  2. Go to the directory where the client package is decompressed.
    cd /tmp/FusionInsight-Client/FusionInsight_Cluster_1_Services_ClientConfig
  3. Run the cat realm.ini >> /etc/hosts command to import the domain name mapping to the hosts file.
- If the host where the client is installed is not a node in the cluster, configure network connections for the client to prevent errors when you run commands on the client.
- If Spark tasks are executed in yarn-client mode, add the spark.driver.host parameter to the file Client installation directory/Spark/spark/conf/spark-defaults.conf and set the parameter to the client IP address.
- If the yarn-client mode is used, you need to configure the mapping between the IP address and host name of the client in the hosts file on the active and standby Yarn nodes (ResourceManager nodes in the cluster) to make sure that the Spark web UI is properly displayed.
Go to the directory where the installation package is stored, and run the following command to install the client to a specified directory (an absolute path), for example, /opt/client. The client installation directory can contain only uppercase letters, lowercase letters, digits, and underscores (_).

cd /tmp/FusionInsight-Client/FusionInsight_Cluster_1_Services_ClientConfig

Run the ./install.sh /opt/client command to install the client. The client is successfully installed if information similar to the following is displayed:
```
ALL component client is installed successfully
```
- If the /opt/hadoopclient directory has been used by existing service clients, you need to use another directory in this step when installing other service clients.
- You must delete the client installation directory when uninstalling a client.
- To ensure that an installed client can only be used by the installation user (for example, user_client), add parameter -o during the installation. That is, run the ./install.sh /opt/client -o command to install the client.
- If the server where the client is to be installed is in the cluster, you do not need to specify the NTP server mode. You can run the ./install.sh /opt/client command to install the client.
- If the server where the client is to be installed is outside the cluster and the NTP server mode is chrony, run the ./install.sh /opt/client -o chrony command to install the client. If the NTP server mode is not specified, the NTP server mode in the cluster is used by default.
- If you do not need to verify clock synchronization during client installation, run the ./install.sh /opt/client -u command to install the client.
- If you do not need to verify clock synchronization during client installation and the installed client can be used by the installation user only (for example, user_client), add the -ou parameter during the installation, that is, run the ./install.sh /opt/client -ou command to install the client.
- If the client node is a server outside the cluster and cannot communicate with the service plane IP address of the active OMS node or cannot access port 20029 of the active OMS node, the client can be successfully installed but cannot be registered with the cluster or displayed on the UI.
Log in to the client to check whether the client is successfully installed.
1. Run the cd /opt/client command to go to the client installation directory.
2. Run the source bigdata_env command to configure environment variables for the client.
3. Run related commands based on the cluster mode.
  - For a cluster in normal mode, directly run commands related to the component client, for example, hdfs.
  - For a cluster in security mode, run the following command to set kinit authentication and enter the password for logging in to the client. For a cluster in normal mode, user authentication is not required.
    kinit admin
```
Password for xxx@HADOOP.COM: #Enter the login password of user admin (same as the user password for logging in to the cluster).
```
    Run the klist command to query and confirm authentication details.
```
Ticket cache: FILE:/tmp/krb5cc_0 
Default principal: xxx@HADOOP.COM   

Valid starting       Expires              Service principal 
04/09/2021 18:22:35  04/10/2021 18:22:29  krbtgt/HADOOP.COM@HADOOP.COM
```
    When kinit authentication is used, the ticket is stored in the /tmp/krb5cc_uid directory by default.
    uid indicates the ID of the user who logs in to the OS. For example, if the UID of user root is 0, the ticket generated for kinit authentication after user root logs in to the system is stored in the /tmp/krb5cc_0 directory.
    
    If the current user does not have the read/write permission for the /tmp directory, the ticket cache path is changed to Client installation directory/tmp/krb5cc_uid. For example, if the client installation directory is /opt/hadoopclient, the kinit authentication ticket is stored in /opt/hadoopclient/tmp/krb5cc_uid.
    
    If the same user is used to log in to the OS for kinit authentication, there is a risk that tickets are overwritten. You can set the -c cache_name parameter to specify the ticket cache path or set the KRB5CCNAME environment variable to avoid this problem.
After the cluster is reinstalled, the previously installed client is no longer available. Perform the following operations to deploy the client again:
1. Log in to the node where the client is deployed as user root.
2. Run the following command to view the directory where the client is located: (In the following example, /opt/hadoopclient is the directory where the client is located.)
  ll /opt
```
drwxr-x---. 6 root root       4096 Dec 11 19:00 hadoopclient 
drwxr-xr-x. 3 root root       4096 Dec  9 02:04 godi 
drwx------. 2 root root      16384 Nov  6 01:03 lost+found 
drwxr-xr-x. 2 root root       4096 Nov  7 09:49 rh 
```
3. Run the following command to delete the files in the folder (for example, /opt/client) where all client programs are located:
  mv /opt/client /tmp/clientbackup
4. Reinstall the client.