Creating a Data Pipeline
Function
This API is used to create a data pipeline.
Calling Method
For details, see Calling APIs.
URI
POST /v1/{project_id}/workspaces/{workspace_id}/siem/pipes
|
Parameter |
Mandatory |
Type |
Description |
|---|---|---|---|
|
project_id |
Yes |
String |
Definition Project ID, which is used to specify the project that a resource belongs to. You can query the resources of a project by project ID. You can obtain the project ID from the API or console. Obtaining the Project ID Constraints N/A Range N/A Default Value N/A |
|
workspace_id |
Yes |
String |
Definition Workspace ID. Constraints N/A Range N/A Default Value N/A |
Request Parameters
|
Parameter |
Mandatory |
Type |
Description |
|---|---|---|---|
|
X-Auth-Token |
Yes |
String |
Definition User token. You can obtain it by calling the IAM API for obtaining a user token. The user token is the value of X-Subject-Token in the response header. Obtaining a User Token Constraints N/A Range N/A Default Value N/A |
|
Parameter |
Mandatory |
Type |
Description |
|---|---|---|---|
|
dataspace_id |
Yes |
String |
Workspace ID. |
|
pipe_name |
Yes |
String |
Data pipeline name. |
|
description |
No |
String |
Description. |
|
storage_period |
Yes |
Integer |
Data storage duration, in days. The default value is 30. The value ranges from 1 to 3,600. |
|
shards |
Yes |
Integer |
Number of pipeline partitions. One partition is created by default. A maximum of 64 partitions can be created. |
|
timestamp_field |
No |
String |
Timestamp. |
|
mapping |
No |
Map<String,KeyIndex> |
Index field mapping. Each key object carries information about a field. There are multiple key objects. The key is variable and indicates the field name. Nesting is supported. |
|
Parameter |
Mandatory |
Type |
Description |
|---|---|---|---|
|
type |
No |
String |
Definition Field type.
Constraints N/A Range
Default Value N/A |
|
is_chinese_exist |
No |
Boolean |
Whether non-English characters are contained. |
|
properties |
No |
Map<String,KeyIndex> |
Nested structure. |
Response Parameters
Status code: 200
|
Parameter |
Type |
Description |
|---|---|---|
|
domain_id |
String |
User domain ID. |
|
project_id |
String |
Project ID. |
|
dataspace_id |
String |
Data space ID. |
|
dataspace_name |
String |
Data space name. |
|
pipe_id |
String |
Pipeline ID. |
|
pipe_name |
String |
Pipeline name. |
|
pipe_type |
String |
Pipeline type. system-defined: predefined by the system; user-defined: custom pipeline. |
|
description |
String |
Description. |
|
storage_period |
Integer |
Index storage duration, measured in days. |
|
shards |
Integer |
Number of index shards. |
|
create_by |
String |
Creator. |
|
create_time |
Integer |
Creation time. |
|
update_by |
String |
Updater. |
|
update_time |
Integer |
Update time. |
Status code: 400
|
Parameter |
Type |
Description |
|---|---|---|
|
error_msg |
String |
Invalid request message. |
|
error_code |
String |
Error code. |
Status code: 401
|
Parameter |
Type |
Description |
|---|---|---|
|
error_msg |
String |
Insufficient permission. |
|
error_code |
String |
Error code. |
Status code: 403
|
Parameter |
Type |
Description |
|---|---|---|
|
error_msg |
String |
Invalid request message. |
|
error_code |
String |
Error code. |
Status code: 500
|
Parameter |
Type |
Description |
|---|---|---|
|
error_msg |
String |
Internal system error. |
|
error_code |
String |
Error code. |
Example Requests
{
"dataspace_id" : "a00106ba-bede-453c-8488-b60c70bd6aed",
"pipe_name" : "pipe-01",
"description" : "test pipe",
"storage_period" : 30,
"shards" : 3,
"mapping" : {
"name" : {
"type" : "text"
},
"id" : {
"type" : "text"
},
"publish_time" : {
"type" : "data"
}
}
}
Example Responses
Status code: 200
Response value to a successful creation.
{
"domain_id" : "0531ed520xxxxxxebedb6e57xxxxxxxx",
"project_id" : "2b31ed520xxxxxxebedb6e57xxxxxxxx",
"dataspace_id" : "a00106ba-bede-453c-8488-b60c70bd6aed",
"dataspace_name" : "dataspace-01",
"pipe_id" : "b22106ba-bede-453c-8488-b60c70bd6aed",
"pipe_name" : "pipe-01",
"pipe_type" : "system-defined",
"description" : "test pipe",
"storage_period" : 30,
"shards" : 3,
"create_by" : "0642ed520xxxxxxebedb6e57xxxxxxxx",
"create_time" : 1584883694354,
"update_by" : "0642ed520xxxxxxebedb6e57xxxxxxxx",
"update_time" : 1584883694354
}
SDK Sample Code
The SDK sample code is as follows.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 |
package com.huaweicloud.sdk.test; import com.huaweicloud.sdk.core.auth.ICredential; import com.huaweicloud.sdk.core.auth.BasicCredentials; import com.huaweicloud.sdk.core.exception.ConnectionException; import com.huaweicloud.sdk.core.exception.RequestTimeoutException; import com.huaweicloud.sdk.core.exception.ServiceResponseException; import com.huaweicloud.sdk.secmaster.v2.region.SecMasterRegion; import com.huaweicloud.sdk.secmaster.v2.*; import com.huaweicloud.sdk.secmaster.v2.model.*; public class CreatePipeSolution { public static void main(String[] args) { // The AK and SK used for authentication are hard-coded or stored in plaintext, which has great security risks. It is recommended that the AK and SK be stored in ciphertext in configuration files or environment variables and decrypted during use to ensure security. // In this example, AK and SK are stored in environment variables for authentication. Before running this example, set environment variables CLOUD_SDK_AK and CLOUD_SDK_SK in the local environment String ak = System.getenv("CLOUD_SDK_AK"); String sk = System.getenv("CLOUD_SDK_SK"); String projectId = "{project_id}"; ICredential auth = new BasicCredentials() .withProjectId(projectId) .withAk(ak) .withSk(sk); SecMasterClient client = SecMasterClient.newBuilder() .withCredential(auth) .withRegion(SecMasterRegion.valueOf("<YOUR REGION>")) .build(); CreatePipeRequest request = new CreatePipeRequest(); request.withWorkspaceId("{workspace_id}"); CreatePipeRequestBody body = new CreatePipeRequestBody(); request.withBody(body); try { CreatePipeResponse response = client.createPipe(request); System.out.println(response.toString()); } catch (ConnectionException e) { e.printStackTrace(); } catch (RequestTimeoutException e) { e.printStackTrace(); } catch (ServiceResponseException e) { e.printStackTrace(); System.out.println(e.getHttpStatusCode()); System.out.println(e.getRequestId()); System.out.println(e.getErrorCode()); System.out.println(e.getErrorMsg()); } } } |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 |
# coding: utf-8 import os from huaweicloudsdkcore.auth.credentials import BasicCredentials from huaweicloudsdksecmaster.v2.region.secmaster_region import SecMasterRegion from huaweicloudsdkcore.exceptions import exceptions from huaweicloudsdksecmaster.v2 import * if __name__ == "__main__": # The AK and SK used for authentication are hard-coded or stored in plaintext, which has great security risks. It is recommended that the AK and SK be stored in ciphertext in configuration files or environment variables and decrypted during use to ensure security. # In this example, AK and SK are stored in environment variables for authentication. Before running this example, set environment variables CLOUD_SDK_AK and CLOUD_SDK_SK in the local environment ak = os.environ["CLOUD_SDK_AK"] sk = os.environ["CLOUD_SDK_SK"] projectId = "{project_id}" credentials = BasicCredentials(ak, sk, projectId) client = SecMasterClient.new_builder() \ .with_credentials(credentials) \ .with_region(SecMasterRegion.value_of("<YOUR REGION>")) \ .build() try: request = CreatePipeRequest() request.workspace_id = "{workspace_id}" request.body = CreatePipeRequestBody( ) response = client.create_pipe(request) print(response) except exceptions.ClientRequestException as e: print(e.status_code) print(e.request_id) print(e.error_code) print(e.error_msg) |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 |
package main import ( "fmt" "github.com/huaweicloud/huaweicloud-sdk-go-v3/core/auth/basic" secmaster "github.com/huaweicloud/huaweicloud-sdk-go-v3/services/secmaster/v2" "github.com/huaweicloud/huaweicloud-sdk-go-v3/services/secmaster/v2/model" region "github.com/huaweicloud/huaweicloud-sdk-go-v3/services/secmaster/v2/region" ) func main() { // The AK and SK used for authentication are hard-coded or stored in plaintext, which has great security risks. It is recommended that the AK and SK be stored in ciphertext in configuration files or environment variables and decrypted during use to ensure security. // In this example, AK and SK are stored in environment variables for authentication. Before running this example, set environment variables CLOUD_SDK_AK and CLOUD_SDK_SK in the local environment ak := os.Getenv("CLOUD_SDK_AK") sk := os.Getenv("CLOUD_SDK_SK") projectId := "{project_id}" auth := basic.NewCredentialsBuilder(). WithAk(ak). WithSk(sk). WithProjectId(projectId). Build() client := secmaster.NewSecMasterClient( secmaster.SecMasterClientBuilder(). WithRegion(region.ValueOf("<YOUR REGION>")). WithCredential(auth). Build()) request := &model.CreatePipeRequest{} request.WorkspaceId = "{workspace_id}" request.Body = &model.CreatePipeRequestBody{ } response, err := client.CreatePipe(request) if err == nil { fmt.Printf("%+v\n", response) } else { fmt.Println(err) } } |
For SDK sample code of more programming languages, see the Sample Code tab in API Explorer. SDK sample code can be automatically generated.
Status Codes
|
Status Code |
Description |
|---|---|
|
200 |
Response value to a successful creation. |
|
400 |
Request error. |
|
401 |
Authentication failed. |
|
403 |
Access denied. |
|
500 |
Internal system error. |
Error Codes
See Error Codes.
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot