Help Center/ Distributed Message Service for Kafka/ API Reference/ Out-of-Date APIs/ API V2/ Smart Connect/ Creating a Dumping Task
Updated on 2026-01-05 GMT+08:00
Online Debugging
CLI Examples
View PDF
Share

Creating a Dumping Task

Function

This API is used to create a dumping task.

This API is out-of-date and may not be maintained in the future. Please use the API described in Enabling Smart Connect (Pay-per-Use Instance).

Call Method

For details, see How to Call an API.

URI

POST /v2/{project_id}/connectors/{connector_id}/sink-tasks

Table 1 URI parameters

Parameter

Mandatory

Type

Description

project_id

Yes

String

Project ID. For details, see Obtaining a Project ID.

connector_id

Yes

String

Instance dump ID.

For details, see Querying Instance IDs.

Request Parameters

Table 2 Request body parameters

Parameter

Mandatory

Type

Description

source_type

Yes

String

Source data type. Currently, only BLOB is supported.

task_name

Yes

String

Name of a dumping task.

destination_type

Yes

String

Storage type. Currently, only OBS is supported.

obs_destination_descriptor

Yes

ObsDestinationDescriptor object

Description of the dump.
Table 3 ObsDestinationDescriptor

Parameter

Mandatory

Type

Description

topics

Yes

String

Names of the topics to be dumped. Multiple topics can be separated by commas (,). Regular expressions are supported.

topics_regex

No

String

Regular expression of the topics to be dumped. Set either the topics_regex parameter or the topics parameter. Do not set both of them. Do not set it to ".*".

consumer_strategy

Yes

String

Offset.

  • latest: Messages are consumed from the end of the topic.
  • earliest: Messages are consumed from the start of the topic.

The default value is latest.

destination_file_type

Yes

String

Dump file format. Currently, only TXT files are supported.

access_key

Yes

String

Access key ID (AK).

secret_key

Yes

String

Secret access key (SK).

obs_bucket_name

Yes

String

Name of the OBS bucket used to store the data.

obs_path

No

String

OBS path. This parameter can be left blank by default.

The value can contain a maximum of 64 characters, including letters, digits, underscores (_), hyphens (-), and slashes (/).

This parameter is left empty by default.

partition_format

Yes

String

Directory structure of the object file written into OBS. The directory structure is in the format of yyyy/MM/dd/HH/mm (time at which the dump task was created).

  • yyyy: year.
  • yyyy/MM: year and month.
  • yyyy/MM/dd: year, month, and day.
  • yyyy/MM/dd/HH: year, month, day, and hour.
  • yyyy/MM/dd/HH/mm: year, month, day, hour, and minute. For example, 2017/11/10/14/49 means that the directory structure is 2017 > 11 > 10 > 14 > 49, where 2017 is the outermost folder.
NOTE:

After the data is dumped successfully, the storage directory structure is obs_bucket_path/file_prefix/partition_format. The default time zone is GMT+08:00.

record_delimiter

No

String

Delimiter for the dump file, which is used to separate the user data that is written into the dump file.

Value range:

  • Comma (,)
  • Semicolon (;)
  • Vertical bar (|)
  • Newline (\n)
  • NULL

Default value: newline (\n).

deliver_time_interval

Yes

Integer

No package files will be generated if there is no data within a time segment.

Value range: 30 to 900

Unit: second.

NOTE:

This parameter is mandatory if streaming data is dumped to OBS.

Response Parameters

Status code: 200

Table 4 Response body parameter

Parameter

Type

Description

task_id

String

Task ID.

Example Request

Creating a dumping task to dump data in topic-test to OBS

POST https://{endpoint}/v2/{project_id}/connectors/{connector_id}/sink-tasks

{
  "source_type" : "BLOB",
  "task_name" : "obsTransfer-1122976956",
  "destination_type" : "OBS",
  "obs_destination_descriptor" : {
    "consumer_strategy" : "earliest",
    "destination_file_type" : "TEXT",
    "access_key" : "XXXXXXXXXXXXXXXXXXXX",
    "secret_key" : "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
    "obs_bucket_name" : "6666",
    "obs_path" : "obsTransfer-1122976956",
    "partition_format" : "yyyy/MM/dd/HH/mm",
    "record_delimiter" : "",
    "deliver_time_interval" : 300,
    "topics" : "topic-test"
  }
}

Example Response

Status code: 200

Successfully creating a dumping task

{
  "task_id" : "2962882a-386c-4c9d-bb59-3b4f55d82961"
}

Status Code

Status Code

Description

200

A dumping task is created successfully.

Error Codes

See Error Codes.

Parent Topic: Smart Connect