Destination Link Configurations of Loader Jobs
Overview
When Loader jobs save data to different storage locations, a destination link needs to be selected and the link properties need to be configured.
obs-connector
Parameter |
Description |
---|---|
Bucket Name |
OBS file system for storing final data. |
Output Directory |
Directory for storing final data in the file system. A directory must be specified. |
File Format |
Loader supports the following file formats of data stored in OBS:
|
Line Separator |
Identifier of each line end of final data |
Field Separator |
Identifier of each field end of final data |
Encoding Type |
Text encoding type of final data. It takes effect on text files only. |
generic-jdbc-connector
Parameter |
Description |
---|---|
Schema Name |
Name of the database storing final data |
Table |
Name of the table saving final data |
ftp-connector or sftp-connector
Parameter |
Description |
---|---|
Output Directory |
Directory for storing final data in the file server. A directory must be specified. |
File Format |
Loader supports the following file formats of data stored in the file server:
|
Line Separator |
Identifier of each line end of final data
NOTE:
If FTP or SFTP serves as a destination link and File Format is set to BINARY_FILE, the value of Line Separator in the advanced properties is invalid. |
Field Separator |
Identifier of each field end of final data
NOTE:
If FTP or SFTP serves as a destination link and File Format is set to BINARY_FILE, the value of Field Separator in the advanced properties is invalid. |
Encoding Type |
Text encoding type of final data. It takes effect on text files only. |
hbase-connector
Parameter |
Description |
---|---|
Table Name |
Name of the HBase table saving final data. You can query and select it on the interface. |
Method |
Data can be imported to an HBase table using either BULKLOAD or PUTLIST. |
Clear Data Before Import |
Whether to clear data in the destination HBase table. Options are as follows:
|
hdfs-connector
Parameter |
Description |
---|---|
Output Directory |
Directory for storing final data in HDFS. A directory must be specified. |
File Format |
Loader supports the following file formats of data stored in HDFS:
|
Compression Codec |
Compression mode used when a file is saved to HDFS. The following modes are supported: NONE, DEFLATE, GZIP, BZIP2, LZ4, and SNAPPY. |
Overwrite |
How to process files in the output directory when files are imported to HDFS. Options are as follows:
|
Line Separator |
Identifier of each line end of final data
NOTE:
If HDFS serves as a destination link and File Format is set to BINARY_FILE, the value of Line Separator in the advanced properties is invalid. |
Field Separator |
Identifier of each field end of final data
NOTE:
If HDFS serves as a destination link and File Format is set to BINARY_FILE, the value of Field Separator in the advanced properties is invalid. |
hive-connector
Parameter |
Description |
---|---|
Database |
Name of the Hive database storing final data. You can query and select it on the interface. |
Table |
Name of the Hive table saving final data. You can query and select it on the interface. |
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.