Creating a Data Pipeline
Function
create pipe
Calling Method
For details, see Calling APIs.
URI
POST /v1/{project_id}/workspaces/{workspace_id}/siem/pipes
Parameter |
Mandatory |
Type |
Description |
---|---|---|---|
project_id |
Yes |
String |
Project ID. Minimum: 32 Maximum: 36 |
workspace_id |
Yes |
String |
Workspace ID Minimum: 32 Maximum: 36 |
Request Parameters
Parameter |
Mandatory |
Type |
Description |
---|---|---|---|
X-Auth-Token |
Yes |
String |
User token. It can be obtained by calling the IAM API used to obtain a user token. The value of X-Subject-Token in the response header is a token. Minimum: 0 Maximum: 2097152 |
Parameter |
Mandatory |
Type |
Description |
---|---|---|---|
dataspace_id |
Yes |
String |
Workspace ID Minimum: 36 Maximum: 36 |
pipe_name |
Yes |
String |
Data pipeline name. Minimum: 5 Maximum: 63 |
description |
No |
String |
Description. Minimum: 1 Maximum: 255 |
storage_period |
Yes |
Integer |
Data storage duration, in days. The default value is 30. The value ranges from 1 to 3600. Minimum: 1 Maximum: 3600 |
shards |
Yes |
Integer |
Number of pipeline partitions. One partition is created by default. A maximum of 64 partitions can be created. Minimum: 1 Maximum: 64 |
timestamp_field |
No |
String |
Timestamp field. Default: __time Minimum: 1 Maximum: 256 |
mapping |
No |
Map<String,KeyIndex> |
Index field mapping. Each key object carries information about a field. There are multiple key objects. The key is variable and indicates the field name. Nesting is supported. |
Parameter |
Mandatory |
Type |
Description |
---|---|---|---|
type |
No |
String |
Field type. The options are text (full-text index field), keyword (structured field), Long, Integer, Double, Float (time field), and Date (time field). Enumeration values:
|
is_chinese_exist |
No |
Boolean |
Whether Chinese characters are contained. |
properties |
No |
Map<String,KeyIndex> |
Nested structure. |
Response Parameters
Status code: 200
Parameter |
Type |
Description |
---|---|---|
domain_id |
String |
Account ID. Minimum: 32 Maximum: 36 |
project_id |
String |
Project ID. Minimum: 32 Maximum: 36 |
dataspace_id |
String |
Data space ID Minimum: 32 Maximum: 36 |
dataspace_name |
String |
Data space name Minimum: 32 Maximum: 36 |
pipe_id |
String |
Indicates the pipe ID. Minimum: 32 Maximum: 36 |
pipe_name |
String |
Pipeline Name Minimum: 32 Maximum: 36 |
pipe_type |
String |
Pipeline type. System-defined Preset types. User-defined Custom types. Minimum: 5 Maximum: 128 |
description |
String |
Description. Minimum: 5 Maximum: 128 |
storage_period |
Integer |
Index storage period by the day. Minimum: 1 Maximum: 100000 |
shards |
Integer |
Index shard quantity. Minimum: 1 Maximum: 128 |
create_by |
String |
Created By Minimum: 5 Maximum: 128 |
create_time |
Integer |
Creation time Minimum: 0 Maximum: 1010000000 |
update_by |
String |
Updated by Minimum: 5 Maximum: 128 |
update_time |
Integer |
Update time. Minimum: 0 Maximum: 10000000000 |
Status code: 400
Parameter |
Type |
Description |
---|---|---|
error_msg |
String |
Invalid request message. Minimum: 1 Maximum: 128 |
error_code |
String |
Error code Minimum: 1 Maximum: 128 |
Status code: 401
Parameter |
Type |
Description |
---|---|---|
error_msg |
String |
Permissions error. Minimum: 1 Maximum: 128 |
error_code |
String |
Error code Minimum: 1 Maximum: 128 |
Status code: 500
Parameter |
Type |
Description |
---|---|---|
error_msg |
String |
Internal system error. Minimum: 1 Maximum: 128 |
error_code |
String |
Error code Minimum: 1 Maximum: 128 |
Example Requests
{ "dataspace_id" : "a00106ba-bede-453c-8488-b60c70bd6aed", "pipe_name" : "pipe-01", "description" : "test pipe", "storage_period" : 30, "shards" : 3, "mapping" : { "name" : { "type" : "text" }, "id" : { "type" : "text" }, "publish_time" : { "type" : "data" } } }
Example Responses
Status code: 200
Created pipeline returned.
{ "domain_id" : "0531ed520xxxxxxebedb6e57xxxxxxxx", "project_id" : "2b31ed520xxxxxxebedb6e57xxxxxxxx", "dataspace_id" : "a00106ba-bede-453c-8488-b60c70bd6aed", "dataspace_name" : "dataspace-01", "pipe_id" : "b22106ba-bede-453c-8488-b60c70bd6aed", "pipe_name" : "pipe-01", "pipe_type" : "system-defined", "description" : "test pipe", "storage_period" : 30, "shards" : 3, "create_by" : "0642ed520xxxxxxebedb6e57xxxxxxxx", "create_time" : 1584883694354, "update_by" : "0642ed520xxxxxxebedb6e57xxxxxxxx", "update_time" : 1584883694354 }
SDK Sample Code
The SDK sample code is as follows.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 |
package com.huaweicloud.sdk.test; import com.huaweicloud.sdk.core.auth.ICredential; import com.huaweicloud.sdk.core.auth.BasicCredentials; import com.huaweicloud.sdk.core.exception.ConnectionException; import com.huaweicloud.sdk.core.exception.RequestTimeoutException; import com.huaweicloud.sdk.core.exception.ServiceResponseException; import com.huaweicloud.sdk.secmaster.v2.region.SecMasterRegion; import com.huaweicloud.sdk.secmaster.v2.*; import com.huaweicloud.sdk.secmaster.v2.model.*; public class CreatePipeSolution { public static void main(String[] args) { // The AK and SK used for authentication are hard-coded or stored in plaintext, which has great security risks. It is recommended that the AK and SK be stored in ciphertext in configuration files or environment variables and decrypted during use to ensure security. // In this example, AK and SK are stored in environment variables for authentication. Before running this example, set environment variables CLOUD_SDK_AK and CLOUD_SDK_SK in the local environment String ak = System.getenv("CLOUD_SDK_AK"); String sk = System.getenv("CLOUD_SDK_SK"); ICredential auth = new BasicCredentials() .withAk(ak) .withSk(sk); SecMasterClient client = SecMasterClient.newBuilder() .withCredential(auth) .withRegion(SecMasterRegion.valueOf("<YOUR REGION>")) .build(); CreatePipeRequest request = new CreatePipeRequest(); CreatePipeRequestBody body = new CreatePipeRequestBody(); request.withBody(body); try { CreatePipeResponse response = client.createPipe(request); System.out.println(response.toString()); } catch (ConnectionException e) { e.printStackTrace(); } catch (RequestTimeoutException e) { e.printStackTrace(); } catch (ServiceResponseException e) { e.printStackTrace(); System.out.println(e.getHttpStatusCode()); System.out.println(e.getRequestId()); System.out.println(e.getErrorCode()); System.out.println(e.getErrorMsg()); } } } |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
# coding: utf-8 import os from huaweicloudsdkcore.auth.credentials import BasicCredentials from huaweicloudsdksecmaster.v2.region.secmaster_region import SecMasterRegion from huaweicloudsdkcore.exceptions import exceptions from huaweicloudsdksecmaster.v2 import * if __name__ == "__main__": # The AK and SK used for authentication are hard-coded or stored in plaintext, which has great security risks. It is recommended that the AK and SK be stored in ciphertext in configuration files or environment variables and decrypted during use to ensure security. # In this example, AK and SK are stored in environment variables for authentication. Before running this example, set environment variables CLOUD_SDK_AK and CLOUD_SDK_SK in the local environment ak = os.environ["CLOUD_SDK_AK"] sk = os.environ["CLOUD_SDK_SK"] credentials = BasicCredentials(ak, sk) client = SecMasterClient.new_builder() \ .with_credentials(credentials) \ .with_region(SecMasterRegion.value_of("<YOUR REGION>")) \ .build() try: request = CreatePipeRequest() request.body = CreatePipeRequestBody( ) response = client.create_pipe(request) print(response) except exceptions.ClientRequestException as e: print(e.status_code) print(e.request_id) print(e.error_code) print(e.error_msg) |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
package main import ( "fmt" "github.com/huaweicloud/huaweicloud-sdk-go-v3/core/auth/basic" secmaster "github.com/huaweicloud/huaweicloud-sdk-go-v3/services/secmaster/v2" "github.com/huaweicloud/huaweicloud-sdk-go-v3/services/secmaster/v2/model" region "github.com/huaweicloud/huaweicloud-sdk-go-v3/services/secmaster/v2/region" ) func main() { // The AK and SK used for authentication are hard-coded or stored in plaintext, which has great security risks. It is recommended that the AK and SK be stored in ciphertext in configuration files or environment variables and decrypted during use to ensure security. // In this example, AK and SK are stored in environment variables for authentication. Before running this example, set environment variables CLOUD_SDK_AK and CLOUD_SDK_SK in the local environment ak := os.Getenv("CLOUD_SDK_AK") sk := os.Getenv("CLOUD_SDK_SK") auth := basic.NewCredentialsBuilder(). WithAk(ak). WithSk(sk). Build() client := secmaster.NewSecMasterClient( secmaster.SecMasterClientBuilder(). WithRegion(region.ValueOf("<YOUR REGION>")). WithCredential(auth). Build()) request := &model.CreatePipeRequest{} request.Body = &model.CreatePipeRequestBody{ } response, err := client.CreatePipe(request) if err == nil { fmt.Printf("%+v\n", response) } else { fmt.Println(err) } } |
For SDK sample code of more programming languages, see the Sample Code tab in API Explorer. SDK sample code can be automatically generated.
Status Codes
Status Code |
Description |
---|---|
200 |
Created pipeline returned. |
400 |
Request error. |
401 |
Authentication failed. |
403 |
Access denied. |
500 |
Internal system error. |
Error Codes
See Error Codes.
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot