Preparing the Configuration Files for Connecting to the Cluster
Preparing User Information for Cluster Authentication
For an MRS cluster with Kerberos authentication enabled, prepare a user who has the operation permission on related components for program authentication.
The following Kafka permission configuration example is for reference only. You can modify the configuration as you need.
- Log in to FusionInsight Manager.
- Choose Cluster > Services > Kafka. On the displayed page, click More > Enable Ranger in the upper right corner. Check whether the button is grayed out.
- If it is grayed out, create a user and assign related operation rights to the user in Ranger.
- Choose System > Permission > User. On the displayed page, click Create. On the displayed page, create a machine-machine user, for example, developuser.
Add the kafkaadmin user group to User Group.
- Log in to the Ranger management page as the Ranger administrator rangeradmin.
- On the home page, click the component plug-in name in the KAFKA area, for example, Kafka.
- Click
in the Action column of the row containing the all - topic policy.
- In the Allow Conditions area, add an allow condition. Select the user created in 2.a for Select User, and select Select/Deselect All for Permissions.
- Click Save.
- Choose System > Permission > User. On the displayed page, click Create. On the displayed page, create a machine-machine user, for example, developuser.
- If the button is available, create a user and grant related operation permissions to the user on Manager.
- Choose System > Permission > User. On the displayed page, click Create on the displayed page. Create a machine-machine user, for example, developuser. Add this user to the kafkaadmin user group.
- Click OK.
- If it is grayed out, create a user and assign related operation rights to the user in Ranger.
- Log in to FusionInsight Manager as user admin and choose System > Permission > User. In the Operation column of developuser, choose More > Download Authentication Credential. Save the file and decompress it to obtain the user.keytab and krb5.conf files of the user.
Preparing the Configuration Files of the Running Environment
During the development or a test run of the program, you need to use the cluster configuration files to connect to an MRS cluster. The configuration files usually contain the cluster component information file and user files used for security authentication. You can obtain the required information from the created MRS cluster.
Nodes used for program debugging or running must be able to communicate with the nodes within the MRS cluster, and the hosts domain name must be configured.
- Scenario 1: Prepare the configuration files required for debugging in the local Windows development environment.
- Log in to the FusionInsight Manager portal and choose Cluster > Dashboard > More > Download Client (For MRS 3.3.0 or later, click Download Client in the upper right corner of the Homepage). Set Select Client Type to Configuration Files Only and click OK. After the client files are packaged and generated, download the client to the local PC as prompted and decompress it.
For example, if the client configuration file package is FusionInsight_Cluster_1_Services_Client.tar, decompress it to obtain FusionInsight_Cluster_1_Services_ClientConfig_ConfigFiles.tar. Then, continue to decompress this file.
- Go to Kafka\config in the directory where the client configuration file is decompressed and obtain the Kafka configuration files listed in Table 1.
Table 1 Configuration files File
Function
client.properties
Kafka client configuration information
consumer.properties
Kafka consumer configuration information
kafkaSecurityMode
whether to enable the security mode for Kafka.
producer.properties
Kafka producer configuration information
server.properties
Kafka server configuration information
- Copy the hosts file content from the decompression directory to the hosts file of the local PC.
- If you need to debug the application in the local Windows environment, ensure that the local PC can communicate with the hosts listed in the hosts file.
- If your PC cannot communicate with the network plane where the MRS cluster is deployed, you can bind an EIP to access the MRS cluster. For details, see Kafka Access Configuration on Windows Using EIPs.
- C:\WINDOWS\system32\drivers\etc\hosts is an example directory in a Windows environment for storing the local hosts file.
- Log in to the FusionInsight Manager portal and choose Cluster > Dashboard > More > Download Client (For MRS 3.3.0 or later, click Download Client in the upper right corner of the Homepage). Set Select Client Type to Configuration Files Only and click OK. After the client files are packaged and generated, download the client to the local PC as prompted and decompress it.
- Scenario 2: Prepare the configuration files required for running the program in a Linux environment.
- Install the MRS cluster client on the node.
For example, the client installation directory can be /opt/client.
- Obtain the configuration files.
- Log in to FusionInsight Manager and choose Cluster > Dashboard > More > Download Client (For MRS 3.3.0 or later, click Download Client in the upper right corner of the Homepage). Set Select Client Type to Configuration Files Only, select Save to Path, and click OK to download the client configuration file to the active OMS node of the cluster.
- Log in to the active OMS node as user root, go to the directory where the client configuration file is stored (/tmp/FusionInsight-Client/ by default), decompress the software package, and obtain the configuration files listed in Table 1 from the Kafka/config directory.
For example, if the client software package is FusionInsight_Cluster_1_Services_Client.tar and the download path is /tmp/FusionInsight-Client on the active OMS node, run the following commands:
cd /tmp/FusionInsight-Client
tar -xvf FusionInsight_Cluster_1_Services_Client.tar
tar -xvf FusionInsight_Cluster_1_Services_ClientConfig_ConfigFiles.tar
cd FusionInsight_Cluster_1_Services_ClientConfig_ConfigFiles
- Check the network connection of the client node.
During the client installation, the system automatically configures the hosts file on the client node. You are advised to check whether the /etc/hosts file contains the host names of the nodes in the cluster. If no, manually copy the content of the hosts file in the decompression directory to the hosts file on the node where the client is located, to ensure that the local host can communicate with each host in the cluster.
- Install the MRS cluster client on the node.
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot