Managing Pipelines

Limitations and Constraints

• A maximum of 20 pipelines can be created in a data space in a region for an account.
• Pipelines created by the system cannot be edited.
• Pipelines created by the system cannot be deleted.
• The throughput of a pipeline cannot exceed 4 MB/s.

Creating a Pipeline

1. Log in to the SecMaster console.
2. Click in the upper left corner of the management console and select a region or project.
3. Click in the upper left corner of the page and choose Security & Compliance > SecMaster.
4. In the navigation pane on the left, choose Workspaces > Management. In the workspace list, click the name of the target workspace.
   Figure 1 Workspace management page
   

5. In the navigation pane on the left, choose Log Audit > Security Data.
   Figure 2 Accessing the Security Analysis tab
   

6. (Optional) Add a data space. For details, see Adding a Data Space.
7. In the data space navigation tree on the left, click on the right of the data space name and select Create Pipeline.
   Figure 3 Create Pipeline
   

8. On the Create Pipeline page, configure pipeline parameters. For details about the parameters, see Table 1.

   Table 1 Parameters for creating a pipeline

   Parameter	Description
   Data Space	Data space the pipeline belongs to. This parameter is generated by the system by default.
   Pipeline Name	Name of the pipeline. It must meet the following requirements: The name can contain 5 to 64 characters. The name must be unique in the workspace and cannot be the same as the name of any other pipeline in the workspace. The name must start with a lowercase letter and can contain only lowercase letters, digits, and underscores (_). The name cannot start with an underscore (_) or contain consecutive underscores (-). The name cannot start with the reserved prefix isap_, csb_, secmaster_, sec_, s_sec_, i_sec_, l_sec_, or security_.
   Shards	The number of shards of the pipeline. The value ranges from 1 to 64. An index can store a large amount of data that exceeds the hardware limits of a node. To solve this problem, Elasticsearch subdivides your index into multiple pieces called shards. When creating an index, you can specify the number of shards as required. Each shard is in itself a fully-functional and independent "index" that can be hosted on any node in the cluster.
   Lifecycle	Lifecycle of data in the pipeline. Value range: 7 to 180
   Description	Remarks on the pipeline. This parameter is optional.

9. Click OK.

   You can click the data space name or next to the data space to view the created pipeline.

Viewing Pipeline Details

1. Log in to the SecMaster console.
2. Click in the upper left corner of the management console and select a region or project.
3. Click in the upper left corner of the page and choose Security & Compliance > SecMaster.
4. In the navigation pane on the left, choose Workspaces > Management. In the workspace list, click the name of the target workspace.
   Figure 4 Workspace management page
   

5. In the navigation pane on the left, choose Log Audit > Security Data.
   Figure 5 Accessing the Security Analysis tab
   

6. In the data space navigation tree on the left, click a data space name to show the pipeline list.
   Figure 6 Viewing pipeline details
   

7. Click next to a pipeline name you want to view. The Pipeline Details pane is displayed on the right of the page.

   Table 2 Pipeline parameters

   Parameter	Description
   Workspace Name	Name of the workspace to which the pipeline belongs.
   Workspace ID	ID of the workspace to which the pipeline belongs.
   Data Space Name	Name of the data space to which the pipeline belongs.
   Data Space ID	ID of the data space to which the pipeline belongs.
   Pipeline Name	Name of the pipeline.
   Pipeline ID	ID of the pipeline.
   Shards	Number of shards of the pipeline.
   Lifecycle	Retention period of the data in the pipeline.
   Created	Time when the pipeline was created.
   Description	Description of the pipeline.

Editing a Pipeline

1. Log in to the SecMaster console.
2. Click in the upper left corner of the management console and select a region or project.
3. Click in the upper left corner of the page and choose Security & Compliance > SecMaster.
4. In the navigation pane on the left, choose Workspaces > Management. In the workspace list, click the name of the target workspace.
   Figure 7 Workspace management page
   

5. In the navigation pane on the left, choose Log Audit > Security Data.
   Figure 8 Accessing the Security Analysis tab
   

6. In the data space navigation tree on the left, click a data space name to show the pipeline list.
   Figure 9 Viewing pipeline details
   

7. Click More > Edit next to the pipeline name.
   Figure 10 Entry for editing a pipeline
   

8. On the Edit Pipeline page, configure pipeline parameters. For details about the parameters, see Table 3.

   Table 3 Parameters for editing a pipeline

   Parameter	Description
   Data Space	Data space to which the pipeline belongs. This parameter cannot be modified.
   Pipeline Name	Name you specified for the pipeline. The name cannot be changed after the pipeline is created.
   Shards	The number of shards of the pipeline. The value ranges from 1 to 64.
   Lifecycle	Lifecycle of data in the pipeline. Value range: 7 to 180
   Description	Remarks on the pipeline. This parameter is optional.

9. Click OK.

Deleting a Pipeline

1. Log in to the SecMaster console.
2. Click in the upper left corner of the management console and select a region or project.
3. Click in the upper left corner of the page and choose Security & Compliance > SecMaster.
4. In the navigation pane on the left, choose Workspaces > Management. In the workspace list, click the name of the target workspace.
   Figure 11 Workspace management page
   

5. In the navigation pane on the left, choose Log Audit > Security Data.
   Figure 12 Accessing the Security Analysis tab
   

6. In the data space navigation tree on the left, click a data space name to show the pipeline list.
   Figure 13 Viewing pipeline details
   

7. Click More > Delete next to the pipeline name.
   Figure 14 Deleting a Pipeline
   

8. In the dialog box displayed, click OK.