Updated on 2026-11-03 GMT+08:00

Creating a Smart Connect Task

Function

This API is used to create a Smart Connect task.

Calling Method

For details, see Calling APIs.

Authorization Information

Each account has all the permissions required to call all APIs, but IAM users must be assigned the required permissions.

  • If you are using role/policy-based authorization, see Permissions Policies and Supported Actions for details on the required permissions.
  • If you are using identity policy-based authorization, the following identity policy-based permissions are required.

    Action

    Access Level

    Resource Type (*: required)

    Condition Key

    Alias

    Dependencies

    dms:instance:createConnectorTask

    Write

    kafka

    • g:ResourceTag/<tag-key>

    • g:EnterpriseProjectId

    dms:instance:createConnectorSinkTask

    -

URI

POST /v2/{project_id}/instances/{instance_id}/connector/tasks

Table 1 Path Parameters

Parameter

Mandatory

Type

Description

project_id

Yes

String

Definition

Project ID. For details, see Obtaining a Project ID.

Constraints

N/A

Range

N/A

Default Value

N/A

instance_id

Yes

String

Definition

Instance ID. To obtain it, call the API for querying all instances.

Constraints

N/A

Range

N/A

Default Value

N/A

Request Parameters

Table 2 Request body parameters

Parameter

Mandatory

Type

Description

task_name

No

String

Definition

Smart Connect task name.

Constraints

N/A

Range

N/A

Default Value

N/A

start_later

No

Boolean

Definition

Whether to start a task later.

Constraints

N/A

Range

  • true: Manually start the task later.

  • false: Start the task immediately after it is created.

Default Value

false

topics

No

String

Definition

Topic of a Smart Connect task.

Constraints

N/A

Range

N/A

Default Value

N/A

topics_regex

No

String

Definition

Topic regular expression of a Smart Connect task.

Constraints

Max.: 64 characters.

Range

N/A

Default Value

N/A

source_type

No

String

Definition

Source type of a Smart Connect task.

Constraints

N/A

Range

  • KAFKA_REPLICATOR_SOURCE: Kafka data replication.

  • NONE

Default Value

N/A

source_task

No

SmartConnectTaskReqSourceConfig object

Definition

Source configuration of a Smart Connect task.

Constraints

N/A

sink_type

No

String

Definition

Target type of a Smart Connect task.

Constraints

N/A

Range

  • OBS_SINK: dump.

  • NONE

Default Value

N/A

sink_task

No

SmartConnectTaskReqSinkConfig object

Definition

Target configuration of a Smart Connect task.

Constraints

N/A

Table 3 SmartConnectTaskReqSourceConfig

Parameter

Mandatory

Type

Description

current_cluster_name

No

String

Definition

Current Kafka instance name. (Mandatory only when the source type is Kafka.)

Constraints

The value can contain 1 to 40 characters. Only letters, digits, hyphens (-), and underscores (_).

Range

N/A

Default Value

N/A

cluster_name

No

String

Definition

Peer Kafka instance name. (Mandatory only when the source type is Kafka.)

Constraints

The value can contain 1 to 40 characters. Only letters, digits, hyphens (-), and underscores (_).

Range

N/A

Default Value

N/A

user_name

No

String

Definition

Username set when SASL_SSL was enabled for the target Kafka instance, or when the SASL_SSL user was created.

Constraints

This parameter is mandatory only when the source type is Kafka and the target Kafka authentication mode is SASL_SSL.

Range

N/A

Default Value

N/A

password

No

String

Definition

Password set when SASL_SSL was enabled for the target Kafka instance, or when the SASL_SSL user was created.

Constraints

This parameter is mandatory only when the source type is Kafka and the target Kafka authentication mode is SASL_SSL.

Range

N/A

Default Value

N/A

sasl_mechanism

No

String

Definition

Target Kafka authentication mode.

Constraints

This parameter is mandatory only when the source type is Kafka and the authentication mode is SASL_SSL.

Range

N/A

Default Value

N/A

instance_id

No

String

Definition

Target Kafka instance ID.

Constraints

This parameter is mandatory only when the source type is Kafka. Either instance_id or bootstrap_servers needs to be specified.

Range

N/A

Default Value

N/A

bootstrap_servers

No

String

Definition

Target Kafka instance address.

Constraints

This parameter is mandatory only when the source type is Kafka. Either instance_id or bootstrap_servers needs to be specified.

Range

N/A

Default Value

N/A

security_protocol

No

String

Definition

Target Kafka authentication mode.

Constraints

This parameter is mandatory only when the source type is Kafka.

Range

  • SASL_SSL: SASL_SSL is enabled.

  • PLAINTEXT: SASL_SSL is disabled.

Default Value

N/A

direction

No

String

Definition

Synchronization direction.

Constraints

This parameter is mandatory only when the source type is Kafka.

Range

  • pull: Replicate data from the target Kafka instance to the current Kafka instance.

  • push: Replicate data from the current instance to the target Kafka instance.

  • two-way: Replicate data bidirectionally.

Default Value

N/A

sync_consumer_offsets_enabled

No

Boolean

Definition

Whether to synchronize the consumer offset.

Constraints

This parameter is mandatory only when the source type is Kafka.

Range

  • true: Synchronize the consumer offset.

  • false: Do not synchronize the consumer offset.

Default Value

false

replication_factor

No

Integer

Definition

Number of replicas specified for a topic during automatic topic creation in the peer instance.

Constraints

  • This parameter is mandatory only when the source type is Kafka.

  • The value of this parameter cannot exceed the number of brokers of the peer instance.

  • This parameter takes precedence over the default.replication.factor parameter configured in the target instance.

Range

N/A

Default Value

N/A

task_num

No

Integer

Definition

Number of data replication tasks. You are advised to retain the default value. If Sync Direction is set to Both, the actual number of tasks is twice the number of configured tasks.

Constraints

This parameter is mandatory only when the source type is Kafka.

Range

N/A

Default Value

2.

rename_topic_enabled

No

Boolean

Definition

Whether to rename a topic. If yes, add the alias of the source Kafka instance before the target topic name to form a new name.

Constraints

This parameter is mandatory only when the source type is Kafka.

Range

  • true: Rename the topic.

  • false: Do not rename the topic.

Default Value

false.

provenance_header_enabled

No

Boolean

Definition

Whether to add the source header. The target topic receives the replicated messages. The message header contains the message source.

Constraints

  • This parameter is mandatory only when the source type is Kafka.

  • If you select Both for Sync Direction, enable Add Source Header to prevent infinite replication.

Range

  • true: Add the source header.

  • false: Do not add the source header.

Default Value

false

consumer_strategy

No

String

Definition

Start offset. latest: Obtain the latest data. earliest: Obtain the earliest data.

Constraints

This parameter is mandatory only when the source type is Kafka.

Range

N/A

Default Value

N/A

compression_type

No

String

Definition

Compression algorithm. (Displayed only when the source type is Kafka.)

Constraints

N/A

Range

  • none

  • gzip

  • snappy

  • lz4

  • zstd

Default Value

N/A

topics_mapping

No

String

Definition

Topic mapping, which is used to customize the target topic name.

Constraints

  • This parameter is mandatory only when the source type is Kafka.

  • Rename Topics and Topic Mapping cannot be configured at the same time.

  • Topic mapping format: source topic:target topic. Use commas (,) to separate multiple topic mappings, for example, topic-sc-1:topic-sc-2,topic-sc-3:topic-sc-4.

Range

N/A

Default Value

N/A

Table 4 SmartConnectTaskReqSinkConfig

Parameter

Mandatory

Type

Description

consumer_strategy

No

String

Definition

Dump start offset.

Constraints

This parameter is mandatory only when the target type is OBS.

Range

  • latest: Obtain the latest data.

  • earliest: Obtain the earliest data.

Default Value

N/A

destination_file_type

No

String

Definition

Dump file format. (This parameter is displayed only when the target type is OBS.)

Constraints

Only text is supported.

Range

Text.

Default Value

N/A

deliver_time_interval

No

Integer

Definition

Dump period (s).

Constraints

This parameter is mandatory only when the target type is OBS.

Range

Text.

Default Value

300.

access_key

No

String

Definition

Access key ID (AK).

Constraints

This parameter is mandatory only when the target type is OBS.

Range

N/A

Default Value

N/A

secret_key

No

String

Definition

SK: secret key used together with the access key ID.

Constraints

This parameter is mandatory only when the target type is OBS.

Range

N/A

Default Value

N/A

obs_bucket_name

No

String

Definition

Dump address, which is the OBS bucket used to store the topic data.

Constraints

This parameter is mandatory only when the target type is OBS.

Range

N/A

Default Value

N/A

obs_path

No

String

Definition

Dump directory, which is the directory for storing topic files dumped to OBS. Use slashes (/) to separate directory levels.

Constraints

This parameter is mandatory only when the target type is OBS.

Range

N/A

Default Value

N/A

partition_format

No

String

Definition

Time directory format.

Constraints

This parameter is mandatory only when the target type is OBS.

Range

  • yyyy: year.

    • yyyy/MM: year and month.

  • yyyy/MM/dd: year, month, and day.

  • yyyy/MM/dd/HH: year, month, day, and hour.

  • yyyy/MM/dd/HH/mm: year, month, day, hour, and minute.

Default Value

N/A

record_delimiter

No

String

Definition

Line break, which is used to separate the user data that is written into the dump file.

Constraints

This parameter is mandatory only when the target type is OBS.

Range

  • ,

  • ;

  • |

  • \n

Default Value

N/A

store_keys

No

Boolean

Definition

Whether to dump the key.

Constraints

This parameter is mandatory only when the target type is OBS.

Range

  • true: Dump the key.

  • false: Do not dump the key.

Default Value

false

Response Parameters

Status code: 200

Table 5 Response body parameters

Parameter

Type

Description

task_name

String

Definition:

Smart Connect task name.

Range:

N/A

topics

String

Definition:

Topic of a Smart Connect task.

Range:

N/A

topics_regex

String

Definition:

Topic regular expression of a Smart Connect task.

Range:

N/A

source_type

String

Definition:

Source type of a Smart Connect task.

Range:

  • NONE: none

  • KAFKA_REPLICATOR_SOURCE: Kafka data replication

source_task

SmartConnectTaskRespSourceConfig object

Definition:

Source configuration of a Smart Connect task.

sink_type

String

Definition:

Target type of a Smart Connect task.

Range:

  • NONE: none

  • OBS_SINK: dump

sink_task

SmartConnectTaskRespSinkConfig object

Definition:

Target configuration of a Smart Connect task.

id

String

Definition:

ID of a Smart Connect task.

Range:

N/A

status

String

Definition

Status of a Smart Connect task.

Range

  • PAUSED

  • RUNNING

  • WAITING

  • RESTARTING

create_time

Long

Definition:

Time when a Smart Connect task was created.

Range:

N/A

Table 6 SmartConnectTaskRespSourceConfig

Parameter

Type

Description

current_cluster_name

String

Definition:

Current Kafka instance name. (Displayed only when the source type is Kafka.)

Range:

N/A

cluster_name

String

Definition:

Target Kafka instance name. (Displayed only when the source type is Kafka.)

Range:

N/A

user_name

String

Definition:

Username of a target Kafka instance. (Displayed only when the source type is Kafka.)

Range:

N/A

sasl_mechanism

String

Definition:

Target Kafka authentication mode. (Displayed only when the source type is Kafka.)

Range:

  • PLAIN

  • SCRAM-SHA-512

instance_id

String

Definition:

Target Kafka instance ID. (Displayed only when the source type is Kafka.)

Range:

N/A

bootstrap_servers

String

Definition:

Target Kafka instance address. (Displayed only when the source type is Kafka.)

Range:

N/A

security_protocol

String

Definition:

Target Kafka authentication mode. (Displayed only when the source type is Kafka.)

Range:

  • PLAINTEXT: SSL is disabled and data is transmitted in plaintext.

  • SASL_SSL: SASL is used for authentication. Data is encrypted with an SSL certificate for high-security transmission.

  • SASL_PLAINTEXT: SASL is used for authentication. Data is transmitted in plaintext for better performance.

direction

String

Definition:

Synchronization direction. (Displayed only when the source type is Kafka.)

Range:

  • pull

  • push

  • two-way

sync_consumer_offsets_enabled

Boolean

Definition

Whether to synchronize the consumer offset. (Displayed only when the source type is Kafka.)

Range

  • true: Synchronize the consumer offset.

  • false: Do not synchronize the consumer offset.

replication_factor

Integer

Definition:

Number of replicas. (Displayed only when the source type is Kafka.)

Range:

N/A

task_num

Integer

Definition:

Number of tasks. (Displayed only when the source type is Kafka.)

Range:

N/A

rename_topic_enabled

Boolean

Definition:

Whether to rename a topic. (Displayed only when the source type is Kafka.)

Range:

  • true: Yes

  • false: No

provenance_header_enabled

Boolean

Definition:

Whether to add the source header. (Displayed only when the source type is Kafka.)

Range:

  • true: Yes

  • false: No

consumer_strategy

String

Definition:

Start offset. (Displayed only when the source type is Kafka.)

Range:

  • latest: obtains the latest data.

  • earliest: obtains the earliest data.

compression_type

String

Definition:

Compression algorithm. (Displayed only when the source type is Kafka.)

Range:

  • none

  • gzip

  • snappy

  • lz4

  • zstd

topics_mapping

String

Definition:

Topic mapping. (Displayed only when the source type is Kafka.)

Range:

N/A

Table 7 SmartConnectTaskRespSinkConfig

Parameter

Type

Description

consumer_strategy

String

Definition:

Dump start offset. (Displayed only when the target type is OBS.)

Range:

  • latest: obtains the latest data.

  • earliest: obtains the earliest data.

destination_file_type

String

Definition

Dump file format. Only text is supported. (Displayed only when the target type is OBS.)

Range

TEXT

deliver_time_interval

Integer

Definition:

Dumping period (s). (Displayed only when the target type is OBS.)

Range:

N/A

obs_bucket_name

String

Definition:

Dumping address. (Displayed only when the target type is OBS.)

Range:

N/A

obs_path

String

Definition:

Dump directory. (Displayed only when the target type is OBS.)

Range:

N/A

partition_format

String

Definition:

Time directory format. (Displayed only when the target type is OBS.)

Range:

N/A

record_delimiter

String

Definition:

Line break. (Displayed only when the target type is OBS.)

Range:

N/A

store_keys

Boolean

Definition:

Storage key. (Displayed only when the target type is OBS.)

Range:

N/A

obs_part_size

Integer

Definition:

Size (in bytes) of each file to be uploaded. The default value is 5242880. (Displayed only when the target type is OBS.)

Range:

N/A

flush_size

Integer

Definition:

Size of flushed data. (Displayed only when the target type is OBS.)

Range:

N/A

timezone

String

Definition:

Time zone. (Displayed only when the target type is OBS.)

Range:

N/A

schema_generator_class

String

Definition:

schema_generator class. The default value is io.confluent.connect.storage.hive.schema.DefaultSchemaGenerator. (Displayed only when the target type is OBS.)

Range:

N/A

partitioner_class

String

Definition:

partitioner class. The default value is io.confluent.connect.storage.partitioner.TimeBasedPartitioner. (Displayed only when the target type is OBS.)

Range:

N/A

value_converter

String

Definition:

Value converter. The default value is org.apache.kafka.connect.converters.ByteArrayConverter. (Displayed only when the target type is OBS.)

Range:

N/A

key_converter

String

Definition:

Key converter. The default value is org.apache.kafka.connect.converters.ByteArrayConverter. (Displayed only when the target type is OBS.)

Range:

N/A

kv_delimiter

String

Definition:

Key-value delimiter. The default value is :. (Displayed only when the target type is OBS.)

Range:

N/A

Example Requests

  • Creating a dumping task that starts immediately.

    POST https://{endpoint}/v2/{project_id}/instances/{instance_id}/connector/tasks
    
    {
      "task_name" : "smart-connect-1",
      "start_later" : false,
      "source_type" : "NONE",
      "topics_regex" : "topic-obs*",
      "sink_type" : "OBS_SINK",
      "sink_task" : {
        "consumer_strategy" : "earliest",
        "destination_file_type" : "TEXT",
        "deliver_time_interval" : 300,
        "access_key" : "********",
        "secret_key" : "********",
        "obs_bucket_name" : "obs_bucket",
        "obs_path" : "obsTransfer-1810125534",
        "partition_format" : "yyyy/MM/dd/HH/mm",
        "record_delimiter" : ";",
        "store_keys" : false
      }
    }
  • Creating a Kafka data replication task that starts later.

    POST https://{endpoint}/v2/{project_id}/instances/{instance_id}/connector/tasks
    
    {
      "task_name" : "smart-connect-2",
      "start_later" : true,
      "source_type" : "KAFKA_REPLICATOR_SOURCE",
      "source_task" : {
        "current_cluster_name" : "A",
        "cluster_name" : "B",
        "user_name" : "user1",
        "password" : "********",
        "sasl_mechanism" : "SCRAM-SHA-512",
        "security_protocol" : "SASL_SSL",
        "instance_id" : "b54c9dd8-********-********",
        "direction" : "two-way",
        "sync_consumer_offsets_enabled" : false,
        "replication_factor" : 3,
        "task_num" : 2,
        "rename_topic_enabled" : false,
        "provenance_header_enabled" : true,
        "consumer_strategy" : "latest",
        "compression_type" : "snappy",
        "topics_mapping" : "topic-sc-1:topic-sc-3,topic-sc-2:topic-sc-4"
      }
    }

Example Responses

Status code: 200

{
  "task_name" : "smart-connect-121248117",
  "topics" : "topic-1643449744",
  "source_task" : {
    "current_cluster_name" : "A",
    "cluster_name" : "B",
    "direction" : "pull",
    "bootstrap_servers" : "192.168.45.58:9092,192.168.44.1:9092,192.168.41.230:9092,192.168.43.112:9092",
    "instance_id" : "59f6d088-****-****-****-********",
    "consumer_strategy" : "earliest",
    "sync_consumer_offsets_enabled" : false,
    "rename_topic_enabled" : false,
    "provenance_header_enabled" : false,
    "security_protocol" : "PLAINTEXT",
    "sasl_mechanism" : "PLAIN",
    "user_name" : "",
    "topics_mapping" : "",
    "compression_type" : "none",
    "task_num" : 2,
    "replication_factor" : 3
  },
  "source_type" : "KAFKA_REPLICATOR_SOURCE",
  "sink_task" : null,
  "sink_type" : "NONE",
  "id" : "194917d0-****-****-****-********",
  "status" : "RUNNING",
  "create_time" : 1708427753133
}

SDK Sample Code

The SDK sample code is as follows.

  • Creating a dumping task that starts immediately.

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    package com.huaweicloud.sdk.test;
    
    import com.huaweicloud.sdk.core.auth.ICredential;
    import com.huaweicloud.sdk.core.auth.BasicCredentials;
    import com.huaweicloud.sdk.core.exception.ConnectionException;
    import com.huaweicloud.sdk.core.exception.RequestTimeoutException;
    import com.huaweicloud.sdk.core.exception.ServiceResponseException;
    import com.huaweicloud.sdk.kafka.v2.region.KafkaRegion;
    import com.huaweicloud.sdk.kafka.v2.*;
    import com.huaweicloud.sdk.kafka.v2.model.*;
    
    
    public class CreateConnectorTaskSolution {
    
        public static void main(String[] args) {
            // The AK and SK used for authentication are hard-coded or stored in plaintext, which has great security risks. It is recommended that the AK and SK be stored in ciphertext in configuration files or environment variables and decrypted during use to ensure security.
            // In this example, AK and SK are stored in environment variables for authentication. Before running this example, set environment variables CLOUD_SDK_AK and CLOUD_SDK_SK in the local environment
            String ak = System.getenv("CLOUD_SDK_AK");
            String sk = System.getenv("CLOUD_SDK_SK");
            String projectId = "{project_id}";
    
            ICredential auth = new BasicCredentials()
                    .withProjectId(projectId)
                    .withAk(ak)
                    .withSk(sk);
    
            KafkaClient client = KafkaClient.newBuilder()
                    .withCredential(auth)
                    .withRegion(KafkaRegion.valueOf("<YOUR REGION>"))
                    .build();
            CreateConnectorTaskRequest request = new CreateConnectorTaskRequest();
            request.withInstanceId("{instance_id}");
            CreateSmartConnectTaskReq body = new CreateSmartConnectTaskReq();
            SmartConnectTaskReqSinkConfig sinkTaskbody = new SmartConnectTaskReqSinkConfig();
            sinkTaskbody.withConsumerStrategy("earliest")
                .withDestinationFileType("TEXT")
                .withDeliverTimeInterval(300)
                .withAccessKey("********")
                .withSecretKey("********")
                .withObsBucketName("obs_bucket")
                .withObsPath("obsTransfer-1810125534")
                .withPartitionFormat("yyyy/MM/dd/HH/mm")
                .withRecordDelimiter(";")
                .withStoreKeys(false);
            body.withSinkTask(sinkTaskbody);
            body.withSinkType(CreateSmartConnectTaskReq.CreateSmartConnectTaskReq.fromValue("OBS_SINK"));
            body.withSourceType(CreateSmartConnectTaskReq.CreateSmartConnectTaskReq.fromValue("NONE"));
            body.withTopicsRegex("topic-obs*");
            body.withStartLater(false);
            body.withTaskName("smart-connect-1");
            request.withBody(body);
            try {
                CreateConnectorTaskResponse response = client.createConnectorTask(request);
                System.out.println(response.toString());
            } catch (ConnectionException e) {
                e.printStackTrace();
            } catch (RequestTimeoutException e) {
                e.printStackTrace();
            } catch (ServiceResponseException e) {
                e.printStackTrace();
                System.out.println(e.getHttpStatusCode());
                System.out.println(e.getRequestId());
                System.out.println(e.getErrorCode());
                System.out.println(e.getErrorMsg());
            }
        }
    }
    
  • Creating a Kafka data replication task that starts later.

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    package com.huaweicloud.sdk.test;
    
    import com.huaweicloud.sdk.core.auth.ICredential;
    import com.huaweicloud.sdk.core.auth.BasicCredentials;
    import com.huaweicloud.sdk.core.exception.ConnectionException;
    import com.huaweicloud.sdk.core.exception.RequestTimeoutException;
    import com.huaweicloud.sdk.core.exception.ServiceResponseException;
    import com.huaweicloud.sdk.kafka.v2.region.KafkaRegion;
    import com.huaweicloud.sdk.kafka.v2.*;
    import com.huaweicloud.sdk.kafka.v2.model.*;
    
    
    public class CreateConnectorTaskSolution {
    
        public static void main(String[] args) {
            // The AK and SK used for authentication are hard-coded or stored in plaintext, which has great security risks. It is recommended that the AK and SK be stored in ciphertext in configuration files or environment variables and decrypted during use to ensure security.
            // In this example, AK and SK are stored in environment variables for authentication. Before running this example, set environment variables CLOUD_SDK_AK and CLOUD_SDK_SK in the local environment
            String ak = System.getenv("CLOUD_SDK_AK");
            String sk = System.getenv("CLOUD_SDK_SK");
            String projectId = "{project_id}";
    
            ICredential auth = new BasicCredentials()
                    .withProjectId(projectId)
                    .withAk(ak)
                    .withSk(sk);
    
            KafkaClient client = KafkaClient.newBuilder()
                    .withCredential(auth)
                    .withRegion(KafkaRegion.valueOf("<YOUR REGION>"))
                    .build();
            CreateConnectorTaskRequest request = new CreateConnectorTaskRequest();
            request.withInstanceId("{instance_id}");
            CreateSmartConnectTaskReq body = new CreateSmartConnectTaskReq();
            SmartConnectTaskReqSourceConfig sourceTaskbody = new SmartConnectTaskReqSourceConfig();
            sourceTaskbody.withCurrentClusterName("A")
                .withClusterName("B")
                .withUserName("user1")
                .withPassword("********")
                .withSaslMechanism("SCRAM-SHA-512")
                .withInstanceId("b54c9dd8-********-********")
                .withSecurityProtocol("SASL_SSL")
                .withDirection("two-way")
                .withSyncConsumerOffsetsEnabled(false)
                .withReplicationFactor(3)
                .withTaskNum(2)
                .withRenameTopicEnabled(false)
                .withProvenanceHeaderEnabled(true)
                .withConsumerStrategy("latest")
                .withCompressionType("snappy")
                .withTopicsMapping("topic-sc-1:topic-sc-3,topic-sc-2:topic-sc-4");
            body.withSourceTask(sourceTaskbody);
            body.withSourceType(CreateSmartConnectTaskReq.CreateSmartConnectTaskReq.fromValue("KAFKA_REPLICATOR_SOURCE"));
            body.withStartLater(true);
            body.withTaskName("smart-connect-2");
            request.withBody(body);
            try {
                CreateConnectorTaskResponse response = client.createConnectorTask(request);
                System.out.println(response.toString());
            } catch (ConnectionException e) {
                e.printStackTrace();
            } catch (RequestTimeoutException e) {
                e.printStackTrace();
            } catch (ServiceResponseException e) {
                e.printStackTrace();
                System.out.println(e.getHttpStatusCode());
                System.out.println(e.getRequestId());
                System.out.println(e.getErrorCode());
                System.out.println(e.getErrorMsg());
            }
        }
    }
    
  • Creating a dumping task that starts immediately.

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    # coding: utf-8
    
    import os
    from huaweicloudsdkcore.auth.credentials import BasicCredentials
    from huaweicloudsdkkafka.v2.region.kafka_region import KafkaRegion
    from huaweicloudsdkcore.exceptions import exceptions
    from huaweicloudsdkkafka.v2 import *
    
    if __name__ == "__main__":
        # The AK and SK used for authentication are hard-coded or stored in plaintext, which has great security risks. It is recommended that the AK and SK be stored in ciphertext in configuration files or environment variables and decrypted during use to ensure security.
        # In this example, AK and SK are stored in environment variables for authentication. Before running this example, set environment variables CLOUD_SDK_AK and CLOUD_SDK_SK in the local environment
        ak = os.environ["CLOUD_SDK_AK"]
        sk = os.environ["CLOUD_SDK_SK"]
        projectId = "{project_id}"
    
        credentials = BasicCredentials(ak, sk, projectId)
    
        client = KafkaClient.new_builder() \
            .with_credentials(credentials) \
            .with_region(KafkaRegion.value_of("<YOUR REGION>")) \
            .build()
    
        try:
            request = CreateConnectorTaskRequest()
            request.instance_id = "{instance_id}"
            sinkTaskbody = SmartConnectTaskReqSinkConfig(
                consumer_strategy="earliest",
                destination_file_type="TEXT",
                deliver_time_interval=300,
                access_key="********",
                secret_key="********",
                obs_bucket_name="obs_bucket",
                obs_path="obsTransfer-1810125534",
                partition_format="yyyy/MM/dd/HH/mm",
                record_delimiter=";",
                store_keys=False
            )
            request.body = CreateSmartConnectTaskReq(
                sink_task=sinkTaskbody,
                sink_type="OBS_SINK",
                source_type="NONE",
                topics_regex="topic-obs*",
                start_later=False,
                task_name="smart-connect-1"
            )
            response = client.create_connector_task(request)
            print(response)
        except exceptions.ClientRequestException as e:
            print(e.status_code)
            print(e.request_id)
            print(e.error_code)
            print(e.error_msg)
    
  • Creating a Kafka data replication task that starts later.

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    # coding: utf-8
    
    import os
    from huaweicloudsdkcore.auth.credentials import BasicCredentials
    from huaweicloudsdkkafka.v2.region.kafka_region import KafkaRegion
    from huaweicloudsdkcore.exceptions import exceptions
    from huaweicloudsdkkafka.v2 import *
    
    if __name__ == "__main__":
        # The AK and SK used for authentication are hard-coded or stored in plaintext, which has great security risks. It is recommended that the AK and SK be stored in ciphertext in configuration files or environment variables and decrypted during use to ensure security.
        # In this example, AK and SK are stored in environment variables for authentication. Before running this example, set environment variables CLOUD_SDK_AK and CLOUD_SDK_SK in the local environment
        ak = os.environ["CLOUD_SDK_AK"]
        sk = os.environ["CLOUD_SDK_SK"]
        projectId = "{project_id}"
    
        credentials = BasicCredentials(ak, sk, projectId)
    
        client = KafkaClient.new_builder() \
            .with_credentials(credentials) \
            .with_region(KafkaRegion.value_of("<YOUR REGION>")) \
            .build()
    
        try:
            request = CreateConnectorTaskRequest()
            request.instance_id = "{instance_id}"
            sourceTaskbody = SmartConnectTaskReqSourceConfig(
                current_cluster_name="A",
                cluster_name="B",
                user_name="user1",
                password="********",
                sasl_mechanism="SCRAM-SHA-512",
                instance_id="b54c9dd8-********-********",
                security_protocol="SASL_SSL",
                direction="two-way",
                sync_consumer_offsets_enabled=False,
                replication_factor=3,
                task_num=2,
                rename_topic_enabled=False,
                provenance_header_enabled=True,
                consumer_strategy="latest",
                compression_type="snappy",
                topics_mapping="topic-sc-1:topic-sc-3,topic-sc-2:topic-sc-4"
            )
            request.body = CreateSmartConnectTaskReq(
                source_task=sourceTaskbody,
                source_type="KAFKA_REPLICATOR_SOURCE",
                start_later=True,
                task_name="smart-connect-2"
            )
            response = client.create_connector_task(request)
            print(response)
        except exceptions.ClientRequestException as e:
            print(e.status_code)
            print(e.request_id)
            print(e.error_code)
            print(e.error_msg)
    
  • Creating a dumping task that starts immediately.

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    72
    73
    74
    75
    76
    77
    78
    79
    80
    81
    82
    83
    84
    85
    package main
    
    import (
    	"fmt"
    	"github.com/huaweicloud/huaweicloud-sdk-go-v3/core/auth/basic"
        kafka "github.com/huaweicloud/huaweicloud-sdk-go-v3/services/kafka/v2"
    	"github.com/huaweicloud/huaweicloud-sdk-go-v3/services/kafka/v2/model"
        region "github.com/huaweicloud/huaweicloud-sdk-go-v3/services/kafka/v2/region"
    )
    
    func main() {
        // The AK and SK used for authentication are hard-coded or stored in plaintext, which has great security risks. It is recommended that the AK and SK be stored in ciphertext in configuration files or environment variables and decrypted during use to ensure security.
        // In this example, AK and SK are stored in environment variables for authentication. Before running this example, set environment variables CLOUD_SDK_AK and CLOUD_SDK_SK in the local environment
        ak := os.Getenv("CLOUD_SDK_AK")
        sk := os.Getenv("CLOUD_SDK_SK")
        projectId := "{project_id}"
    
        auth, err := basic.NewCredentialsBuilder().
            WithAk(ak).
            WithSk(sk).
            WithProjectId(projectId).
            SafeBuild()
    
        if err != nil {
            fmt.Println(err)
            return
        }
    
        hcClient, err := kafka.KafkaClientBuilder().
             WithRegion(region.ValueOf("<YOUR REGION>")).
             WithCredential(auth).
             SafeBuild()
    
    
        if err != nil {
            fmt.Println(err)
            return
        }
    
        client := kafka.NewKafkaClient(hcClient)
    
        request := &model.CreateConnectorTaskRequest{}
    	request.InstanceId = "{instance_id}"
    	consumerStrategySinkTask:= "earliest"
    	destinationFileTypeSinkTask:= "TEXT"
    	deliverTimeIntervalSinkTask:= int32(300)
    	accessKeySinkTask:= "********"
    	secretKeySinkTask:= "********"
    	obsBucketNameSinkTask:= "obs_bucket"
    	obsPathSinkTask:= "obsTransfer-1810125534"
    	partitionFormatSinkTask:= "yyyy/MM/dd/HH/mm"
    	recordDelimiterSinkTask:= ";"
    	storeKeysSinkTask:= false
    	sinkTaskbody := &model.SmartConnectTaskReqSinkConfig{
    		ConsumerStrategy: &consumerStrategySinkTask,
    		DestinationFileType: &destinationFileTypeSinkTask,
    		DeliverTimeInterval: &deliverTimeIntervalSinkTask,
    		AccessKey: &accessKeySinkTask,
    		SecretKey: &secretKeySinkTask,
    		ObsBucketName: &obsBucketNameSinkTask,
    		ObsPath: &obsPathSinkTask,
    		PartitionFormat: &partitionFormatSinkTask,
    		RecordDelimiter: &recordDelimiterSinkTask,
    		StoreKeys: &storeKeysSinkTask,
    	}
    	sinkTypeCreateSmartConnectTaskReq:= model.GetCreateSmartConnectTaskReqSinkTypeEnum().OBS_SINK
    	sourceTypeCreateSmartConnectTaskReq:= model.GetCreateSmartConnectTaskReqSourceTypeEnum().NONE
    	topicsRegexCreateSmartConnectTaskReq:= "topic-obs*"
    	startLaterCreateSmartConnectTaskReq:= false
    	taskNameCreateSmartConnectTaskReq:= "smart-connect-1"
    	request.Body = &model.CreateSmartConnectTaskReq{
    		SinkTask: sinkTaskbody,
    		SinkType: &sinkTypeCreateSmartConnectTaskReq,
    		SourceType: &sourceTypeCreateSmartConnectTaskReq,
    		TopicsRegex: &topicsRegexCreateSmartConnectTaskReq,
    		StartLater: &startLaterCreateSmartConnectTaskReq,
    		TaskName: &taskNameCreateSmartConnectTaskReq,
    	}
    	response, err := client.CreateConnectorTask(request)
    	if err == nil {
            fmt.Printf("%+v\n", response)
        } else {
            fmt.Println(err)
        }
    }
    
  • Creating a Kafka data replication task that starts later.

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    72
    73
    74
    75
    76
    77
    78
    79
    80
    81
    82
    83
    84
    85
    86
    87
    88
    89
    90
    91
    92
    93
    package main
    
    import (
    	"fmt"
    	"github.com/huaweicloud/huaweicloud-sdk-go-v3/core/auth/basic"
        kafka "github.com/huaweicloud/huaweicloud-sdk-go-v3/services/kafka/v2"
    	"github.com/huaweicloud/huaweicloud-sdk-go-v3/services/kafka/v2/model"
        region "github.com/huaweicloud/huaweicloud-sdk-go-v3/services/kafka/v2/region"
    )
    
    func main() {
        // The AK and SK used for authentication are hard-coded or stored in plaintext, which has great security risks. It is recommended that the AK and SK be stored in ciphertext in configuration files or environment variables and decrypted during use to ensure security.
        // In this example, AK and SK are stored in environment variables for authentication. Before running this example, set environment variables CLOUD_SDK_AK and CLOUD_SDK_SK in the local environment
        ak := os.Getenv("CLOUD_SDK_AK")
        sk := os.Getenv("CLOUD_SDK_SK")
        projectId := "{project_id}"
    
        auth, err := basic.NewCredentialsBuilder().
            WithAk(ak).
            WithSk(sk).
            WithProjectId(projectId).
            SafeBuild()
    
        if err != nil {
            fmt.Println(err)
            return
        }
    
        hcClient, err := kafka.KafkaClientBuilder().
             WithRegion(region.ValueOf("<YOUR REGION>")).
             WithCredential(auth).
             SafeBuild()
    
    
        if err != nil {
            fmt.Println(err)
            return
        }
    
        client := kafka.NewKafkaClient(hcClient)
    
        request := &model.CreateConnectorTaskRequest{}
    	request.InstanceId = "{instance_id}"
    	currentClusterNameSourceTask:= "A"
    	clusterNameSourceTask:= "B"
    	userNameSourceTask:= "user1"
    	passwordSourceTask:= "********"
    	saslMechanismSourceTask:= "SCRAM-SHA-512"
    	instanceIdSourceTask:= "b54c9dd8-********-********"
    	securityProtocolSourceTask:= "SASL_SSL"
    	directionSourceTask:= "two-way"
    	syncConsumerOffsetsEnabledSourceTask:= false
    	replicationFactorSourceTask:= int32(3)
    	taskNumSourceTask:= int32(2)
    	renameTopicEnabledSourceTask:= false
    	provenanceHeaderEnabledSourceTask:= true
    	consumerStrategySourceTask:= "latest"
    	compressionTypeSourceTask:= "snappy"
    	topicsMappingSourceTask:= "topic-sc-1:topic-sc-3,topic-sc-2:topic-sc-4"
    	sourceTaskbody := &model.SmartConnectTaskReqSourceConfig{
    		CurrentClusterName: &currentClusterNameSourceTask,
    		ClusterName: &clusterNameSourceTask,
    		UserName: &userNameSourceTask,
    		Password: &passwordSourceTask,
    		SaslMechanism: &saslMechanismSourceTask,
    		InstanceId: &instanceIdSourceTask,
    		SecurityProtocol: &securityProtocolSourceTask,
    		Direction: &directionSourceTask,
    		SyncConsumerOffsetsEnabled: &syncConsumerOffsetsEnabledSourceTask,
    		ReplicationFactor: &replicationFactorSourceTask,
    		TaskNum: &taskNumSourceTask,
    		RenameTopicEnabled: &renameTopicEnabledSourceTask,
    		ProvenanceHeaderEnabled: &provenanceHeaderEnabledSourceTask,
    		ConsumerStrategy: &consumerStrategySourceTask,
    		CompressionType: &compressionTypeSourceTask,
    		TopicsMapping: &topicsMappingSourceTask,
    	}
    	sourceTypeCreateSmartConnectTaskReq:= model.GetCreateSmartConnectTaskReqSourceTypeEnum().KAFKA_REPLICATOR_SOURCE
    	startLaterCreateSmartConnectTaskReq:= true
    	taskNameCreateSmartConnectTaskReq:= "smart-connect-2"
    	request.Body = &model.CreateSmartConnectTaskReq{
    		SourceTask: sourceTaskbody,
    		SourceType: &sourceTypeCreateSmartConnectTaskReq,
    		StartLater: &startLaterCreateSmartConnectTaskReq,
    		TaskName: &taskNameCreateSmartConnectTaskReq,
    	}
    	response, err := client.CreateConnectorTask(request)
    	if err == nil {
            fmt.Printf("%+v\n", response)
        } else {
            fmt.Println(err)
        }
    }
    

For SDK sample code of more programming languages, see the Sample Code tab in API Explorer. SDK sample code can be automatically generated.

Status Codes

Status Code

Description

200

Successful.

Error Codes

See Error Codes.