Help Center/ Cloud Data Migration/ User Guide/ Creating a Link in a CDM Cluster/ Configuring Link Parameters/ MRS Hudi Link Parameters
Updated on 2026-07-29 GMT+08:00
View PDF
Share

MRS Hudi Link Parameters

Table 1 describes the MRS Hudi link parameters.

Do not change the password or user when the job is running. If you do so, the password will not take effect immediately and the job will fail.

Table 1 Hudi link parameters

Parameter

Description

Example Value

Name

Link name

Hudilink

Manager IP

Enter or select the Manager IP address.

You can click Select to select a created MRS cluster. CDM automatically fills in the authentication information.

If the Hadoop type is MRS, enter the IP address of MRS Manager.

If the Hadoop type is FusionInsight HD, enter the IP address of FusionInsight HD Manager.

Enter the IP address based on the scenario and sequence.

  • If you enter one IP address, enter the management-plane floating IP address of the MRS cluster.
  • If you enter two IP addresses, enter the IP addresses of the active and standby nodes on the service plane of the MRS cluster. Use semicolons (;) to separate the IP addresses.
  • If you enter three IP addresses, enter the IP address of the active node on the service plane of the MRS cluster, IP address of the standby node on the service plane of the MRS cluster, and the floating IP address of the management plane of the MRS cluster. Use semicolons (;) to separate the IP addresses.
NOTE:

MRS clusters whose Kerberos encryption type is aes256-sha2,aes128-sha2 are not supported, and only MRS clusters whose Kerberos encryption type is aes256-sha1,aes128-sha1 are supported.
  • 127.0.0.1
  • 127.0.0.1;127.0.0.2;127.0.0.3

Link type

When connecting to an MRS cluster via JDBC or API to submit SQL statements, if it involves a non-MRS cluster, a non-Hive 3.x scenario, or a scenario where Hive serves as the source, select JDBC. API is not supported in standalone mode. By default, API is only used for submitting the SQL statements used to insert data. To submit other SQL statements, select JDBC.

JDBC

JobGateway IP

This parameter is displayed when Link type is set to API.

It indicates the JobGateway IP address, which can be obtained from the MRS JobGateway module. Use commas (,) to separate multiple IP addresses.

192.168.0.2,192.168.0.3

Authentication Method

Authentication method used for accessing MRS

  • SIMPLE: Select this for non-security mode.
  • KERBEROS: Select this for security mode.

KERBEROS

Username

Username for logging in to MRS Manager

cdm

Password

Password for logging in to MRS Manager

-

OBS storage support

Whether to support OBS storage. If the Hudi table data is stored in OBS, you need to enable this function.

Yes

AK

This parameter is available when OBS storage support is set to Yes.

AK and SK are used to log in to the OBS server.

You need to create an access key for the current account and obtain an AK/SK pair.

To obtain an access key, perform the following steps:
  1. Log in to the management console, move the cursor to the username in the upper right corner, and select My Credentials from the drop-down list.
  2. On the My Credentials page, choose Access Keys, and click Create Access Key. See Figure 1.
    Figure 1 Clicking Create Access Key

  3. Click OK and save the access key file as prompted. The access key file will be saved to your browser's configured download location. Open the credentials.csv file to view Access Key Id and Secret Access Key.
    NOTE:
    • Only two access keys can be added for each user.
    • To ensure access key security, the access key is automatically downloaded only when it is generated for the first time and cannot be obtained from the management console later. Keep them properly.

-

SK

-

OBS Test Path

This parameter is available when OBS storage support is set to Yes.

Enter a complete file path. The permission to access the path will be verified through the metadata query API.

NOTE:
  • For object storage, the path must be accurate to object, for example, obs://bucket/dir/test.txt. Otherwise, a 404 error occurs.
  • For a parallel file system, the path must be accurate to directory, for example, obs://bucket/dir.

obs://bucket/dir/test.txt

Properties

Other client attributes

  • socketTimeout: Spark JDBC connection timeout interval, in seconds
  • fs.defaultFS: NameNode URI for the HDFS metadata
  • hadoop.user.name: username for accessing HDFS
  • keepStageTableWhenFailed: whether to retain the temporary table when a task fails. In resumable data transfer, set the value to true. The default value is false.

-

Default Queue

Value of the request parameter queue, which determines the queue to which a Spark task is submitted for execution.

default