Help Center/ MapReduce Service/ Component Operation Guide (Normal)/ Using Loader/ Managing Loader Links (MRS 3.x and Later Versions)
Updated on 2023-08-03 GMT+08:00

Managing Loader Links (MRS 3.x and Later Versions)

Scenario

You can create, view, edit, and delete links on the Loader page.

This section applies to MRS 3.x or later.

Creating a Connection

  1. Log in to the service page.

    For MRS 3.x or later, log in to FusionInsight Manager (for details, see Accessing FusionInsight Manager (MRS 3.x or Later)) and choose Cluster > Services.

  2. Select Loader. On the right of Loader WebUI, click the link to open the Loader web UI.
  3. On the Loader page, click New job.
  4. Click Add next to Connection and set connection parameters.

    For details about the parameters, see Loader Connection Configuration.

  5. Click OK.

    If connection configurations, for example, IP address, port, and access user information, are incorrect, the connection will fail to be verified and saved.

    You can click Test to immediately check whether the connection is available.

Viewing a Connection

  1. On the Loader page, click New job.
  2. Click the drop-down list of Connection to view the connections you have created.

Editing a Connection

  1. On the Loader page, click New job.
  2. Select the name of the connection to be edited from the Connection drop-down list.
  3. Click Edit next to Connection.
  4. On the dialog box displayed, modify the connection parameters based on service requirements.
  5. Click Test.

    • If the test is successful, go to 6.
    • If the test fails, repeat 4.

  6. Click Save.

    If a Loader job has integrated into a Loader link, editing the link parameters may affect Loader running.

Deleting a Connection

  1. On the Loader page, click New job.
  2. Select the name of the connection to be deleted from the Connection drop-down list.
  3. Click Delete.
  4. In the displayed dialog box, click OK.

    If a Loader job has integrated a Loader connection, the connection cannot be deleted.

Loader Connection Configuration

Loader supports the following connections:

  • generic-jdbc-connector: For details about parameter settings, see Table 1.
  • ftp-connector: For details about parameter settings, see Table 2.
  • sftp-connector: For details about parameter settings, see Table 3.
  • hdfs-connector: For details about parameter settings, see Table 4.
  • oracle-connector: For details about parameter settings, see Table 5.
  • mysql-fastpath-connector: For details about parameter settings, see Table 7.
  • oracle-partition-connector: For details about parameter settings, see Table 6.
Table 1 generic-jdbc-connector configuration

Parameter

Description

Name

Name of a Loader connection

Connector

Select generic-jdbc-connector.

JDBC Driver Class

JDBC driver classes are as follows:

  • oracle: oracle.jdbc.driver.OracleDriver
  • SQLServer: com.microsoft.sqlserver.jdbc.SQLServerDriver
  • mysql: com.mysql.jdbc.Driver
  • postgresql: org.postgresql.Driver
  • gaussdb200: com.huawei.gauss200.jdbc.Driver

JDBC Connection String

Database access address, which can be an IP address or domain name.

Enter the database connection string. In the following examples, 10.10.10.10 indicates the IP address and test indicates the database name.

  • oracle: jdbc:oracle:thin:@10.10.10.10:1521:orcl
  • SQLServer: jdbc:sqlserver://10.10.10.10:1433;DatabaseName=test
  • mysql: jdbc:mysql://10.10.10.10/test?&useUnicode=true&characterEncoding=GBK
  • postgresql: jdbc:postgresql://10.10.10.10:5432/test
  • gaussdb200: jdbc:gaussdb://10.10.10.10:15400/test (15400 is an example port number.)

Username

Username for accessing the database

Password

Password of the user. Use the actual password.

Table 2 ftp-connector configuration

Parameter

Description

Name

Name of a Loader connection

Connector

Select ftp-connector.

FTP Mode

Select ACTIVE or PASSIVE.

FTP Protocol

Select:

  • FTP
  • SSL_EXPLICIT
  • SSL_IMPLICIT
  • TLS_EXPLICIT
  • TLS_IMPLICIT

File Name Encoding Type

Encoding type of the file name or file path.

Table 3 sftp-connector configuration

Parameter

Description

Name

Name of a Loader connection

Connector

Select sftp-connector.

Table 4 hdfs-connector configuration

Parameter

Description

Name

Name of a Loader connection

Connector

Select hdfs-connector.

Table 5 oracle-connector configuration

Parameter

Description

Name

Name of a Loader connection

Connector

Select oracle-connector.

JDBC Connection String

String for connecting to the database, for example, jdbc:oracle:thin:@IP: port:database.

Username

Username for accessing the database

Password

Password of the user. Use the actual password.

Table 6 oracle-partition-connector configuration

Parameter

Description

Name

Name of a Loader connection

Connector

Select oracle-partition-connector.

JDBC Driver Class

Enter oracle.jdbc.driver.OracleDriver.

JDBC Connection String

String for connecting to the database, for example, jdbc:oracle:thin:@IP: port:database.

Username

Username for accessing the database

Password

Password of the user. Use the actual password.

Table 7 mysql-fastpath-connector configuration

Parameter

Description

Name

Name of a Loader connection

Connector

Select mysql-fastpath-connector.

NOTICE:

When mysql-fastpath-connector is used, the mysqldump and mysqlimport commands of MySQL must be available on NodeManagers, and the MySQL client version to which the two commands belong must be compatible with the MySQL server version. If the two commands are unavailable or the versions are incompatible, see http://dev.mysql.com/doc/refman/5.7/en/linux-installation-rpm.html. Install the MySQL client applications and tools.

For example, you need to install the following RPM packages in the RHEL-x86 system (select the package version based on the site requirements):

  • mysql-community-client-5.7.23-1.el7.x86_64.rpm
  • mysql-community-common-5.7.23-1.el7.x86_64.rpm
  • mysql-community-devel-5.7.23-1.el7.x86_64.rpm
  • mysql-community-embedded-5.7.23-1.el7.x86_64.rpm
  • mysql-community-libs-5.7.23-1.el7.x86_64.rpm
  • mysql-community-libs-compat-5.7.23-1.el7.x86_64.rpm

JDBC Connection String

String for connecting to the database, for example, jdbc:mysql://IP/database?&useUnicode=true&characterEncoding=GBK.

Username

Username for accessing the database

Password

Password of the user. Use the actual password.