To HDFS
Type |
Parameter |
Description |
Example Value |
---|---|---|---|
Basic parameters |
Write Directory |
HDFS directory to which data will be written. |
/user/cdm/output |
File Format |
Format used for transmitting data. The CSV and JSON formats are supported for migration to tables, and the binary format is supported for file migration. |
CSV |
|
Newline character processing mode |
Policy for processing newline characters when the data written to a text file table contains newline characters (\n|\r|\r\n)
|
Delete |
|
Newline Replacement String |
It indicates the string that will replace newline characters. |
- |
|
Advanced attributes |
Write to Temporary File |
This parameter is displayed only when File Format is set to Binary. It indicates writing the binary file to a temporary file whose name extension is .tmp. |
No |
Line Separator |
This parameter is displayed only when File Format is set to CSV. It indicates the line feed character in a file. By default, the system automatically identifies \n, \r, and \r\n. You can configure special characters. For spaces and carriage returns, encode them with URL. You can also configure them by editing the job JSON, in which case URL encoding is not required. |
\n |
|
Field Delimiter |
This parameter is displayed only when File Format is set to CSV. Field delimiter in the file. Special characters must be encoded using URLs. |
, |
|
Job Success Marker File |
Marker file name When the job is successful, a marker file is generated in the destination directory. If this parameter is left blank, no marker file will be generated. |
finish.txt |
|
Use Quote Character |
This parameter is displayed only when File Format is set to CSV. Enclose a string using quote characters. Field separators in the quote characters are regarded as a part of the string value. Only quotation marks (") can be used as quote characters. |
No |
|
Customize Hierarchical Directory |
Users can customize the directory hierarchy of files. Example: [Table name]/[Year]/[Month]/[Day]/[Data file name]. csv |
No |
|
Hierarchical Directory |
This parameter is displayed when Customize Hierarchical Directory is set to Yes. Used to specify the directory level of a file, with time macro supported (the time format is yyyy/MM/dd). If the source link is a relational database link, the directory name consists of the source table name and a custom directory name. In other scenarios, the directory is a custom directory. |
${dateformat(yyyy/MM/dd, -1, DAY)} |
|
File Name Prefix |
This parameter is displayed only when File Format is set to CSV. Prefix of the file name. File name format: prefix-jobname-timestamp-index |
data |
|
Compression Format |
This parameter is displayed only when File Format is set to CSV. Compression format of the file to be written
|
SNAPPY |
|
Encryption |
This parameter is displayed only when File Format is set to Binary. Encryption mode for the uploaded data
|
None |
|
DEK |
This parameter is displayed when File Format is set to Binary and Encryption Mode is set. It indicates the data encryption key. The AES-256-GCM key consists of 64 hexadecimal digits. |
DD0AE00DFECD78BF051BCFDA25BD4E320DB0A7AC75A1F3FC3D3C56A457DCDC1B |
|
IV |
This parameter is displayed when File Format is set to Binary and Encryption Mode is set. It indicates the initialization vector, which consists of 32 hexadecimal digits. |
5C91687BA886EDCD12ACBC3FF19A3C3F |
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot