Creating a Hadoop Cluster Configuration
On the Cluster Configurations page, you can create, edit, or delete Hadoop cluster configurations.
When creating a Hadoop link, the Hadoop cluster configurations can simplify the link creation. See Figure 1 for details.
CDM supports the following types of Hadoop links:
- MRS clusters: MRS HDFS, MRS HBase, and MRS Hive
- FusionInsight clusters: FusionInsight HDFS, FusionInsight HBase, and FusionInsight Hive
- Apache clusters: Apache HDFS, Apache HBase, and Apache Hive
Scenario
Before creating a Hadoop link, you are advised to create cluster configurations to simplify the link parameter configurations.
Prerequisites
- A cluster has been created.
- You have obtained the Hadoop cluster configuration file and keytab file. See Table 1 for details.
Obtaining the Cluster Configuration File and Keytab File
The methods for obtaining the Hadoop cluster configuration file and keytab file vary depending on the Hadoop cluster type. For details, see Table 1.
Hadoop Link |
Obtaining the Cluster Configuration File |
Obtaining the Keytab File |
---|---|---|
MRS cluster
|
For clusters of MRS 3.x:
For clusters of MRS 2.x or earlier:
See MRS documentation for details. |
For clusters of MRS 3.x:
For clusters of MRS 2.x or earlier:
See MRS documentation for details. |
FusionInsight clusters:
|
See the FusionInsight documentation for details. |
See the FusionInsight documentation for details. |
Apache clusters:
|
In the Apache cluster scenario, only the required configuration files and packaging rules are described. For details about how to obtain each configuration file, see the corresponding documentation.
|
In the Apache cluster scenario, only the principles for packaging authentication credential files are required. For details about how to obtain the authentication credential files, see the corresponding documentation.
|
- A cluster configuration file contains the configuration parameters of the cluster. If the cluster configuration parameters are modified, you need to obtain the configuration file again.
- The keytab file is the authentication credential file. Before obtaining the keytab file, you need to change the password of this user at least once in the cluster. Otherwise, the downloaded keytab file may be unavailable. After a user password is changed, the exported keytab file becomes invalid, and you need to export a keytab file again.
- The keytab file is used only in a cluster in security mode. In other cases, you do not need to prepare the keytab file.
Procedure
- On the CDM console, choose Cluster Management in the left navigation pane. Locate the row that contains a cluster and choose Job Management > Links > Cluster Configurations.
- On the Cluster Configurations page, click Create Cluster Configuration and set the parameters as prompt.
Figure 2 Creating cluster configurations
- Configuration Name: Enter a cluster configuration name that is easy to remember and distinguish based on the type of the data source to be connected.
- Configuration File: Click Select File to select a local cluster configuration file, and then click on the right to upload the file.
- Principal: This parameter is required only for clusters in security mode. Principal is the username in Kerberos security mode and must be the same as that in the keytab file.
- Keytab File: Upload the keytab file only for clusters in security mode. Click Select File to select a local keytab file, and then click Upload on the right to upload the file.
- Description: Add a description to identify and distinguish the cluster configuration.
- Click OK. When creating a Hadoop link, set Authentication Method as required, Use Cluster Config to Yes, and then select the corresponding cluster configuration name to quickly create a Hadoop link.
Figure 3 Use Cluster Config
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot