Help Center/ SecMaster/ API Reference/ Security Analysis V2 APIs/ Pipeline Management/ Creating a Pipeline

Updated on 2025-10-29 GMT+08:00

Online Debugging

CLI Examples

View PDF

Creating a Pipeline

Function

This API is used to create a pipeline.

Calling Method

For details, see Calling APIs.

URI

POST /v2/{project_id}/workspaces/{workspace_id}/siem/pipes

**Table 1** Path Parameters
Parameter	Mandatory	Type	Description
project_id	Yes	String	Definition Project ID, which is used to specify the project that a resource belongs to. You can query the resources of a project by project ID. You can obtain the project ID from the API or console. Obtaining the Project ID Constraints N/A Range N/A Default Value N/A
workspace_id	Yes	String	Workspace ID.

Request Parameters

**Table 2** Request header parameters
Parameter	Mandatory	Type	Description
X-Auth-Token	Yes	String	Definition User token. You can obtain it by calling the IAM API for obtaining a user token. The user token is the value of X-Subject-Token in the response header. Obtaining a User Token Constraints N/A Range N/A Default Value N/A

**Table 3** Request body parameters
Parameter	Mandatory	Type	Description
pipe_name	Yes	String	Pipe name
pipe_alias	Yes	String	Pipe alias
category	Yes	String	Definition Pipeline directory. STREAMING_TO_INDEX: writing data to indexes in streams STREAMING_TO_LAKE: writing data to data lakes in streams STREAMING_TO_INDEX_LAKE: writing data to indexes and data lakes in streams STREAMING: transmitting data in streams Constraints N/A Range STREAMING_TO_INDEX STREAMING_TO_LAKE STREAMING_TO_INDEX_LAKE STREAMING Default Value N/A
directory	No	String	Directory group.
description	No	String	Pipe description
schema	Yes	PipeSchema object	Pipeline mode.
storage_setting	Yes	PipeStorageSetting object	Pipeline storage settings.
display_setting	No	TableDisplaySetting object	Table display settings.

**Table 4** PipeSchema
Parameter	Mandatory	Type	Description
columns	Yes	Array of TableColumnForIsapPipe objects	Pipeline column.
watermark_column	No	String	Pipe watermark column
watermark_interval	No	Float	Pipe watermark delay interval
time_filter	Yes	String	Pipe time filter column

**Table 5** TableColumnForIsapPipe
Parameter	Mandatory	Type	Description
column_name	Yes	String	Table column name
column_data_type	Yes	String	Definition Column data type. ROW: row type MAP_STRING: string mapping type MAP_DECIMAL: decimal mapping type TINYINT: tiny integer SMALLINT: small integer INT: integer BIGINT: long integer DECIMAL: decimal data type FLOAT: single-precision floating point DOUBLE: double-precision floating point CHAR: fixed-length string VARCHAR: variable-length string STRING: string type KEYWORD: keyword type BOOLEAN: boolean type DATE: date type TIME: time type TIMESTAMP: timestamp type TIMESTAMP_LTZ: local timestamp type Constraints N/A Range ROW MAP_STRING MAP_DECIMAL TINYINT SMALLINT INT BIGINT DECIMAL FLOAT DOUBLE CHAR VARCHAR STRING KEYWORD BOOLEAN DATE TIME TIMESTAMP TIMESTAMP_LTZ Default Value N/A
column_data_type_setting	No	String	Table column data type setting
column_type	No	String	Definition Column field type. PHYSICAL: physical field METADATA: metadata field VIRTUAL_METADATA: virtual metadata field COMPUTED: computation field Constraints N/A Range PHYSICAL METADATA VIRTUAL_METADATA COMPUTED Default Value N/A
column_type_setting	No	String	Column type settings.
default_value	No	String	Table column default value
nullable	No	Boolean	Table column nullable
array	No	Boolean	Is an array
depth	No	Long	Table column depth
parent_name	No	String	Table column name
own_name	No	String	Table column name without parent name

**Table 6** PipeStorageSetting
Parameter	Mandatory	Type	Description
streaming_bandwidth	No	Integer	Bandwidth in MB/s
streaming_dataspace_id	Yes	String	UUID
index_storage_period	No	Long	Index storage period
index_storage_size	No	Long	Index storage size in GB
index_shards	No	Long	Index shards
data_transformation_cu	No	Integer	Data transformation CU

**Table 7** TableDisplaySetting
Parameter	Mandatory	Type	Description
columns	No	Array of TableColumnDisplaySetting objects	Table column display list.
format	No	String	Definition Table display settings. Table: A table will be displayed. RAW: Raw data will be displayed. Constraints N/A Range TABLE RAW Default Value N/A

**Table 8** TableColumnDisplaySetting
Parameter	Mandatory	Type	Description
column_name	No	String	Table column name.
column_alias	No	String	Table column alias.
display_by_default	No	Boolean	Whether to use default display settings.

Response Parameters

Status code: 200

**Table 9** Response body parameters
Parameter	Type	Description
project_id	String	Project ID.
workspace_id	String	UUID
pipe_id	String	UUID
pipe_name	String	Pipe name
pipe_alias	String	Pipe alias
category	String	Definition Pipeline directory. STREAMING_TO_INDEX: writing data to indexes in streams STREAMING_TO_LAKE: writing data to data lakes in streams STREAMING_TO_INDEX_LAKE: writing data to indexes and data lakes in streams STREAMING: transmitting data in streams Constraints N/A Range STREAMING_TO_INDEX STREAMING_TO_LAKE STREAMING_TO_INDEX_LAKE STREAMING Default Value N/A
directory	String	Directory group.
description	String	Pipe description
process_status	String	Definition Job processing status. COMPLETED: completed. CREATING: being created UPDATING: being updated DELETING: being deleted UPDATING_FAILED: update failed DELETING_FAILED: deletion failed Constraints N/A Range COMPLETED CREATING UPDATING DELETING UPDATING_FAILED DELETING_FAILED Default Value N/A
process_error	String	Definition Pipeline processing error. NONE Constraints N/A Range NONE Default Value N/A
owner_type	String	Definition Pipeline owner type. SYSTEM: system USER: user CLOUD_LOG: cloud logs Constraints N/A Range SYSTEM USER CLOUD_LOG Default Value N/A
resources	Array of PipeResource objects	Pipeline resource.
schema	PipeSchema object	Pipeline mode.
create_time	Integer	Timestamp, in ms.
update_time	Integer	Timestamp, in ms.
delete_time	Integer	Timestamp, in ms.

**Table 10** PipeResource
Parameter	Type	Description
pipe_resource_type	Object	Pipeline resource.

**Table 11** PipeSchema
Parameter	Type	Description
columns	Array of TableColumnForIsapPipe objects	Pipeline column.
watermark_column	String	Pipe watermark column
watermark_interval	Float	Pipe watermark delay interval
time_filter	String	Pipe time filter column

**Table 12** TableColumnForIsapPipe
Parameter	Type	Description
column_name	String	Table column name
column_data_type	String	Definition Column data type. ROW: row type MAP_STRING: string mapping type MAP_DECIMAL: decimal mapping type TINYINT: tiny integer SMALLINT: small integer INT: integer BIGINT: long integer DECIMAL: decimal data type FLOAT: single-precision floating point DOUBLE: double-precision floating point CHAR: fixed-length string VARCHAR: variable-length string STRING: string type KEYWORD: keyword type BOOLEAN: boolean type DATE: date type TIME: time type TIMESTAMP: timestamp type TIMESTAMP_LTZ: local timestamp type Constraints N/A Range ROW MAP_STRING MAP_DECIMAL TINYINT SMALLINT INT BIGINT DECIMAL FLOAT DOUBLE CHAR VARCHAR STRING KEYWORD BOOLEAN DATE TIME TIMESTAMP TIMESTAMP_LTZ Default Value N/A
column_data_type_setting	String	Table column data type setting
column_type	String	Definition Column field type. PHYSICAL: physical field METADATA: metadata field VIRTUAL_METADATA: virtual metadata field COMPUTED: computation field Constraints N/A Range PHYSICAL METADATA VIRTUAL_METADATA COMPUTED Default Value N/A
column_type_setting	String	Column type settings.
default_value	String	Table column default value
nullable	Boolean	Table column nullable
array	Boolean	Is an array
depth	Long	Table column depth
parent_name	String	Table column name
own_name	String	Table column name without parent name

Example Requests

None

Example Responses

None

SDK Sample Code

The SDK sample code is as follows.

Java

    
     
       
       
         package com.huaweicloud.sdk.test;

import com.huaweicloud.sdk.core.auth.ICredential;
import com.huaweicloud.sdk.core.auth.BasicCredentials;
import com.huaweicloud.sdk.core.exception.ConnectionException;
import com.huaweicloud.sdk.core.exception.RequestTimeoutException;
import com.huaweicloud.sdk.core.exception.ServiceResponseException;
import com.huaweicloud.sdk.secmaster.v2.region.SecMasterRegion;
import com.huaweicloud.sdk.secmaster.v2.*;
import com.huaweicloud.sdk.secmaster.v2.model.*;


public class CreatePipeSolution {

    public static void main(String[] args) {
        // The AK and SK used for authentication are hard-coded or stored in plaintext, which has great security risks. It is recommended that the AK and SK be stored in ciphertext in configuration files or environment variables and decrypted during use to ensure security.
        // In this example, AK and SK are stored in environment variables for authentication. Before running this example, set environment variables CLOUD_SDK_AK and CLOUD_SDK_SK in the local environment
        String ak = System.getenv("CLOUD_SDK_AK");
        String sk = System.getenv("CLOUD_SDK_SK");
        String projectId = "{project_id}";

        ICredential auth = new BasicCredentials()
                .withProjectId(projectId)
                .withAk(ak)
                .withSk(sk);

        SecMasterClient client = SecMasterClient.newBuilder()
                .withCredential(auth)
                .withRegion(SecMasterRegion.valueOf("<YOUR REGION>"))
                .build();
        CreatePipeRequest request = new CreatePipeRequest();
        request.withWorkspaceId("{workspace_id}");
        CreatePipeRequestBody body = new CreatePipeRequestBody();
        request.withBody(body);
        try {
            CreatePipeResponse response = client.createPipe(request);
            System.out.println(response.toString());
        } catch (ConnectionException e) {
            e.printStackTrace();
        } catch (RequestTimeoutException e) {
            e.printStackTrace();
        } catch (ServiceResponseException e) {
            e.printStackTrace();
            System.out.println(e.getHttpStatusCode());
            System.out.println(e.getRequestId());
            System.out.println(e.getErrorCode());
            System.out.println(e.getErrorMsg());
        }
    }
}

        

      

    
   

Python

    
     
       
       
         # coding: utf-8

import os
from huaweicloudsdkcore.auth.credentials import BasicCredentials
from huaweicloudsdksecmaster.v2.region.secmaster_region import SecMasterRegion
from huaweicloudsdkcore.exceptions import exceptions
from huaweicloudsdksecmaster.v2 import *

if __name__ == "__main__":
    # The AK and SK used for authentication are hard-coded or stored in plaintext, which has great security risks. It is recommended that the AK and SK be stored in ciphertext in configuration files or environment variables and decrypted during use to ensure security.
    # In this example, AK and SK are stored in environment variables for authentication. Before running this example, set environment variables CLOUD_SDK_AK and CLOUD_SDK_SK in the local environment
    ak = os.environ["CLOUD_SDK_AK"]
    sk = os.environ["CLOUD_SDK_SK"]
    projectId = "{project_id}"

    credentials = BasicCredentials(ak, sk, projectId)

    client = SecMasterClient.new_builder() \
        .with_credentials(credentials) \
        .with_region(SecMasterRegion.value_of("<YOUR REGION>")) \
        .build()

    try:
        request = CreatePipeRequest()
        request.workspace_id = "{workspace_id}"
        request.body = CreatePipeRequestBody(
        )
        response = client.create_pipe(request)
        print(response)
    except exceptions.ClientRequestException as e:
        print(e.status_code)
        print(e.request_id)
        print(e.error_code)
        print(e.error_msg)

        

      

    
   

Go

    
     
       
       
         package main

import (
	"fmt"
	"github.com/huaweicloud/huaweicloud-sdk-go-v3/core/auth/basic"
    secmaster "github.com/huaweicloud/huaweicloud-sdk-go-v3/services/secmaster/v2"
	"github.com/huaweicloud/huaweicloud-sdk-go-v3/services/secmaster/v2/model"
    region "github.com/huaweicloud/huaweicloud-sdk-go-v3/services/secmaster/v2/region"
)

func main() {
    // The AK and SK used for authentication are hard-coded or stored in plaintext, which has great security risks. It is recommended that the AK and SK be stored in ciphertext in configuration files or environment variables and decrypted during use to ensure security.
    // In this example, AK and SK are stored in environment variables for authentication. Before running this example, set environment variables CLOUD_SDK_AK and CLOUD_SDK_SK in the local environment
    ak := os.Getenv("CLOUD_SDK_AK")
    sk := os.Getenv("CLOUD_SDK_SK")
    projectId := "{project_id}"

    auth := basic.NewCredentialsBuilder().
        WithAk(ak).
        WithSk(sk).
        WithProjectId(projectId).
        Build()

    client := secmaster.NewSecMasterClient(
        secmaster.SecMasterClientBuilder().
            WithRegion(region.ValueOf("<YOUR REGION>")).
            WithCredential(auth).
            Build())

    request := &model.CreatePipeRequest{}
	request.WorkspaceId = "{workspace_id}"
	request.Body = &model.CreatePipeRequestBody{
	}
	response, err := client.CreatePipe(request)
	if err == nil {
        fmt.Printf("%+v\n", response)
    } else {
        fmt.Println(err)
    }
}

        

      

    
   

For SDK sample code of more programming languages, see the Sample Code tab in API Explorer. SDK sample code can be automatically generated.

Status Codes

Status Code	Description
200	Request succeeded.

Error Codes

See Error Codes.

Parent topic: Pipeline Management

Previous topic: Querying the Pipeline List

Next topic: Analysis Script Management

Feedback

Was this page helpful?

Helpful Not helpful

Provide feedback

Thank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.

The system is busy. Please try again later.

Which of the following issues have you encountered?

Content is inconsistent with the product UI

Unclear descriptions

Lack of examples or code

Incorrect steps

Can't find what I need

Lack of best practices

Feedback (optional)

0/500

Select at least one type of issue, and enter your comments or suggestions.

Enter a maximum of 500 characters.

Submit Cancel

For any further questions, feel free to contact us through the chatbot.

Chatbot