Dumping Data to OBS
Dumping the JSON, BLOB, and CSV Data to the Text Data
Parameter |
Description |
Value |
---|---|---|
Task Name |
Name of the dump task. The names of dump tasks created for the same stream must be unique. A dump task name is 1 to 64 characters long. Only letters, digits, hyphens (-), and underscores (_) are allowed. |
- |
Dump Bucket |
Name of the OBS bucket used to store data from the DIS stream. The bucket name is created when you create a bucket in OBS. |
- |
File Directory |
Directory created in OBS to store files from the DIS stream. This directory name is 0 to 50 characters long. By default, this parameter is left unspecified. |
- |
Time Directory Format |
Data will be saved according to the time format in the file directory of the OBS bucket. For example, if the time directory is accurate to day, the save path will be in the format of bucket name/file directory/year/month/day.
Possible values are as follows:
You can only select but not enter a value in this field. |
- |
Record Delimiter |
Delimiter used to separate different dump records.
Possible values are as follows:
You can only select but not enter a value in this field. |
- |
Offset |
|
Latest |
Dump Interval (s) |
Interval at which data from the DIS stream will be imported into dump destination, such as OBS, MRS, DLI, and DWS. If no data was pushed to the DIS stream during the time specified here, the dump file will not be generated. Value range: 30s to 900s Unit: second Default value: 300s |
- |
Dumping the JSON Data to the CSV Data
Parameter |
Description |
Value |
---|---|---|
Task Name |
Name of the dump task. The names of dump tasks created for the same stream must be unique. A dump task name is 1 to 64 characters long. Only letters, digits, hyphens (-), and underscores (_) are allowed. |
- |
Dump Bucket |
Name of the OBS bucket used to store data from the DIS stream. The bucket name is created when you create a bucket in OBS. |
- |
File Directory |
Directory created in OBS to store files from the DIS stream. This directory name is 0 to 50 characters long. By default, this parameter is left unspecified. |
- |
Time Directory Format |
Data will be saved according to the time format in the file directory of the OBS bucket. For example, if the time directory is accurate to day, the save path will be in the format of bucket name/file directory/year/month/day.
Possible values are as follows:
You can only select but not enter a value in this field. |
- |
Offset |
|
Latest |
Dump Interval (s) |
User-defined interval at which data is imported from the current DIS stream into the target place. If no data is pushed to the DIS stream during the current interval, no dump file package will be generated. Value range: 30s to 900s Unit: second Default value: 300s |
- |
Dumping the JSON and CSV Data to the Parquet Data
Dumping the JSON and CSV Data to the CarbonData Data
Parameter |
Description |
Value |
---|---|---|
Source Data Schema |
JSON or CSV data sample, used to describe the JSON or CSV data format. DIS can generate an Avro schema based on the JSON or CSV data sample and convert the uploaded JSON or CSV data to the CarbonData format. |
- |
CarbonData Retrieval Attribute |
Attribute of the carbon table, used to create a carbon writer. The following keys are supported:
|
- |
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot