Help Center/ MapReduce Service/ Component Operation Guide (LTS) (Ankara Region)/ Using Loader/ Managing Loader Links
Updated on 2024-11-29 GMT+08:00
View PDF
Share

Managing Loader Links

Scenario

You can create, view, edit, and delete links on the Loader page.

Creating a Connection

  1. Select Loader. On the right of Loader WebUI, click the link to open the Loader web UI.
  2. On the Loader page, click New job.
  3. Click Add next to Connection and set connection parameters.

    For details about the parameters, see Loader Connection Configuration.

  4. Click OK.

    If connection configurations, for example, IP address, port, and access user information, are incorrect, the connection will fail to be verified and saved.

    You can click Test to immediately check whether the connection is available.

Viewing a Connection

  1. On the Loader page, click New job.
  2. Click the drop-down list of Connection to view the connections you have created.

Editing a Connection

  1. On the Loader page, click New job.
  2. Select the name of the connection to be edited from the Connection drop-down list.
  3. Click Edit next to Connection.
  4. On the dialog box displayed, modify the connection parameters based on service requirements.
  5. Click Test.

    • If the test is successful, go to 6.
    • If the test fails, repeat 4.

  6. Click Save.

    If a Loader job has integrated into a Loader link, editing the link parameters may affect Loader running.

Deleting a Connection

  1. On the Loader page, click New job.
  2. Select the name of the connection to be deleted from the Connection drop-down list.
  3. Click Delete.
  4. In the displayed dialog box, click OK.

    If a Loader job has integrated a Loader connection, the connection cannot be deleted.

Loader Connection Configuration

Loader supports the following connections:

  • generic-jdbc-connector: For details about parameter settings, see Table 1.
  • ftp-connector: For details about parameter settings, see Table 2.
  • sftp-connector: For details about parameter settings, see Table 3.
  • hdfs-connector: For details about parameter settings, see Table 4.
  • oracle-connector: For details about parameter settings, see Table 5.
  • mysql-fastpath-connector: For details about parameter settings, see Table 7.
  • oracle-partition-connector: For details about parameter settings, see Table 6.
  • clickhouse-connector: For details about parameter settings, see Table 8.
Table 1 generic-jdbc-connector configuration

Parameter

Description

Name

Name of a Loader connection

Connector

Select generic-jdbc-connector.

JDBC Driver Class

JDBC driver classes are as follows:

  • oracle: oracle.jdbc.driver.OracleDriver
  • SQLServer: com.microsoft.sqlserver.jdbc.SQLServerDriver
  • mysql: com.mysql.jdbc.Driver
  • postgresql: org.postgresql.Driver

JDBC Connection String

Database access address, which can be an IP address or domain name.

Enter the database connection string. The following uses the IP address 10.10.10.10 as an example to describe how to access the test database.

  • oracle: jdbc:oracle:thin:@10.10.10.10:1521:test
  • SQLServer: jdbc:sqlserver://10.10.10.10:1433;DatabaseName=test (1433 is an example port number.)
  • mysql: jdbc:mysql://10.10.10.10/test?&useUnicode=true&characterEncoding=GBK
  • postgresql: jdbc:postgresql://10.10.10.10:5432/test
  • MOTService: jdbc:opengauss://10.10.10.10:20105/test (20105 is an example port number.)

Username

Username for accessing the database

Password

Password of the user. Use the actual password.
Table 2 ftp-connector configuration

Parameter

Description

Name

Name of a Loader connection

Connector

Select ftp-connector.

FTP Mode

Select ACTIVE or PASSIVE.

FTP Protocol

Select:

  • FTP
  • SSL_EXPLICIT
  • SSL_IMPLICIT
  • TLS_EXPLICIT
  • TLS_IMPLICIT

File Name Encoding Type

Encoding type of the file name or file path.
Table 3 sftp-connector configuration

Parameter

Description

Name

Name of a Loader connection

Connector

Select sftp-connector.
Table 4 hdfs-connector configuration

Parameter

Description

Name

Name of a Loader connection

Connector

Select hdfs-connector.
Table 5 oracle-connector configuration

Parameter

Description

Name

Name of a Loader connection

Connector

Select oracle-connector.

JDBC Connection String

Connection string used for connecting to the database

Username

Username for accessing the database

Password

Password of the user. Use the actual password.
Table 6 oracle-partition-connector configuration

Parameter

Description

Name

Name of a Loader connection

Connector

Select oracle-partition-connector.

JDBC Driver Class

Enter oracle.jdbc.driver.OracleDriver.

JDBC Connection String

Connection string used for connecting to the database

Username

Username for accessing the database

Password

Password of the user. Use the actual password.
Table 7 mysql-fastpath-connector configuration

Parameter

Description

Name

Name of a Loader connection

Connector

Select mysql-fastpath-connector.

NOTICE:

When mysql-fastpath-connector is used, the mysqldump and mysqlimport commands of MySQL must be available on NodeManagers, and the MySQL client version to which the two commands belong must be compatible with the MySQL server version. If the two commands are unavailable or the versions are incompatible, see http://dev.mysql.com/doc/refman/5.7/en/linux-installation-rpm.html. Install the MySQL client applications and tools.

For example, you need to install the following RPM packages in the RHEL-x86 system (select the package version based on the site requirements):

  • mysql-community-client-5.7.23-1.el7.x86_64.rpm
  • mysql-community-common-5.7.23-1.el7.x86_64.rpm
  • mysql-community-devel-5.7.23-1.el7.x86_64.rpm
  • mysql-community-embedded-5.7.23-1.el7.x86_64.rpm
  • mysql-community-libs-5.7.23-1.el7.x86_64.rpm
  • mysql-community-libs-compat-5.7.23-1.el7.x86_64.rpm

JDBC Connection String

Connection string used for connecting to the database

Username

Username for accessing the database

Password

Password of the user. Use the actual password.
Table 8 clickhouse-connector configuration

Parameter

Description

Name

Name of a Loader connection

Connector

Select clickhouse-connector.

ClickHouse Connection String
  • Kerberos authentication has been enabled for the cluster. The format is jdbc:clickhouse://Database IP address:Database port number/Database name?ssl=true&sslmode=none.
  • Kerberos authentication is disabled for the cluster. The format is jdbc:clickhouse://Database IP address:Database port number/Database name.
NOTE:
  • Database IP address: To obtain the IP address of the ClickHouseBalancer instance, log in to FusionInsight Manager, choose Cluster > Services > ClickHouse, and click Instance.
  • Database port number:
    • To obtain the port number of a cluster with Kerberos authentication enabled, log in to FusionInsight Manager, choose Cluster > Services, click Logical Cluster, view the logical cluster, and obtain the value of Ssl Port in HTTP Balancer Port.
    • To obtain the port number of a cluster with Kerberos authentication disabled, log in to FusionInsight Manager, choose Cluster > Services, click Logical Cluster, view the logical cluster, and obtain the value of Port in HTTP Balancer Port.

Username

Username for accessing the database

Password

Password of the user. Use the actual password.
Parent topic: Using Loader