Adding MRS Dump Tasks

project_id

Yes

String

Project ID.

stream_name

Yes

String

Name of the stream.

Maximum: 60

X-Auth-Token

Yes

String

User token

The token can be obtained by calling the IAM API used to obtain a user token. The value of X-Subject-Token in the response header is the user token.

destination_type

Yes

String

Type of the dump task

• OBS: Data will be dumped to OBS.

• MRS: Data will be dumped to MRS

• DLI: Data will be dumped to DLI

• CLOUDTABLE: Data will be dumped to CloudTable

• DWS: Data will be dumped to DWS.

Default: NOWHERE

Enumeration values:

• MRS

mrs_destination_descriptor

No

MRSDestinationDescriptorRequest object

Parameter list of the MRS to which data in the DIS stream will be dumped.

task_name

Yes

String

Name of the dump task. The task name consists of letters, digits, hyphens (-), and underscores (_). It contains 1 to 64 characters.

agency_name

Yes

String

Name of the agency created on IAM. DIS uses an agency to access your specified resources. The parameters for creating an agency are as follows: - Agency Type: Cloud service- Cloud Service: DIS- Validity Period: unlimited- Scope: Global service- Project: OBS- Policy: Tenant Administrator If agencies are available, you can use an IAM API to obtain the available agencies. For details about the API, see https://support.huaweicloud.com/en-us/api-iam/iam_12_0001.html. This parameter cannot be left unspecified and the parameter value cannot exceed 64 characters. If there are dump tasks on the console, the system displays a message indicating that an agency will be automatically created. The name of the automatically created agency is dis_admin_agency.

Maximum: 64

deliver_time_interval

Yes

Integer

User-defined interval at which data is imported from the current DIS stream into OBS. If no data is pushed to the DIS stream during the current interval, no dump file package will be generated.

Value range: 30–900

Default value: 300

Unit: second

Minimum: 30

Maximum: 900

Default: 300

consumer_strategy

No

String

Offset

• LATEST: maximum offset, indicating that the latest data will be extracted.

• TRIM_HORIZON: minimum offset, indicating that the earliest data will be extracted.

Default value: LATEST

Default: LATEST

Enumeration values:

• LATEST

• TRIM_HORIZON

mrs_cluster_name

Yes

String

Name of the MRS cluster to which data in the DIS stream will be dumped.

Note:

Only MRS clusters with non-Kerberos authentication are supported.

mrs_cluster_id

Yes

String

ID of the MRS cluster to which data in the DIS stream will be dumped.

mrs_hdfs_path

Yes

String

Hadoop Distributed File System (HDFS) path of the MRS cluster to which data in the DIS stream will be dumped.

file_prefix

No

String

Self-defined directory created in the OBS bucket and used to temporarily store data in the DIS stream. Directory levels are separated by slashes (/) and cannot start with slashes.

The value can contain a maximum of 50 characters, including letters, digits, underscores (_), and slashes (/).

This parameter is left blank by default.

hdfs_prefix_folder

No

String

Directory to store files that will be dumped to the chosen MRS cluster. Different directory levels are separated by slash (/).

The directory name contains 0 to 50 characters.

This parameter is left blank by default.

obs_bucket_path

Yes

String

Name of the OBS bucket used to temporarily store data in the DIS stream.

retry_duration

No

String

Time duration for DIS to retry if data fails to be dumped. If the retry time exceeds the value of this parameter, the data that fails to be dumped is backed up to the OBS bucket/file_prefix/mrs_error directory.

Value range: 0–7200

Unit: second

Default value: 1800

If the value is set to 0, no retry is allowed.

201

Normal response.