Help Center/ MapReduce Service/ User Guide (Kuala Lumpur Region)/ MRS Cluster Component Operation Gudie/ Using Loader/ Loader Link Configuration
Updated on 2022-08-12 GMT+08:00
View PDF
Share

Loader Link Configuration

This section applies to versions earlier than MRS 3.x.

Overview

Loader supports the following links. This section describes configurations of each link.

  • obs-connector
  • generic-jdbc-connector
  • ftp-connector or sftp-connector
  • hbase-connector, hdfs-connector, or hive-connector

OBS Link

An OBS link is a data exchange channel between Loader and OBS. Table 1 describes the configuration parameters.

Table 1 obs-connector configuration

Parameter

Description

Name

Name of a Loader connection

OBS Server

Enter an OBS endpoint. The common format is OBS.Region.DomainName.

Run the following command to query the endpoints of OBS:

cat /opt/Bigdata/apache-tomcat-7.0.78/webapps/web/WEB-INF/classes/cloud-obs.properties

Port

Specifies the port for accessing OBS data. The default value is 443.

Access Key

AK for a user to access OBS

Security Key

SK corresponding to AK

Relational Database Link

A relational database link is a data exchange channel between Loader and a relational database. Table 2 describes the configuration parameters.

Some parameters are hidden by default. They appear only after you click Show Senior Parameter.

Table 2 generic-jdbc-connector configuration

Parameter

Description

Name

Name of a Loader link

Database Type

Data types supported by Loader links: ORACLE, MYSQL, and MPPDB

Host

Database access address, which can be an IP address or domain name.

Port

Port for accessing the database

Database

Name of the database saving data

Username

Username for accessing the database

Password

Password of the user Use the actual password.
Table 3 Senior parameter configuration

Parameter

Description

Fetch Size

A maximum volume of data obtained during each database access

Connection Properties

Drive properties exclusive to the database link supported by databases of different types, for example, autoReconnect of MYSQL. If you want to define the drive properties, click Add.

Identifier Enclose

Delimiter for reserving keywords in the database SQL. Delimiters defined in different databases vary.

File Server Link

File server links include FTP and SFTP links and serve as a data exchange channel between Loader and a file server. Table 4 describes the configuration parameters.

Table 4 ftp-connector or sftp-connector configuration

Parameter

Description

Name

Name of a Loader link

Hostname/IP

Enter the file server access address, which can be a host name or IP address.

Port

Port for accessing the file server.

  • Use port 21 for FTP.
  • Use port 22 for SFTP.

Username

Username for logging in to the file server

Password

Password of the user

MRS Cluster Link

MRS cluster links include HBase, HDFS, and Hive links and serve as a data exchange channel between Loader and HBase, HDFS, or Hive.

When configuring an MRS cluster link, set the name, select a connector, for example, hbase-connector, hdfs-connector, or hive-connector, and save the settings.

Parent topic: Using Loader