Managing Pipelines
A data transfer message topic and a storage index form a pipeline. This topic describes how to manage data pipelines. You can:
- Creating a Pipeline: If you need to use security analysis, data analysis, and intelligent modeling features in SecMaster, you need to create pipelines.
- Viewing Pipeline Details: You can view the pipeline details, including the pipeline name, data space, and creation time.
- Editing a Pipeline: You can modify the pipeline information, such as the number of shards, description, and lifecycle.
- Deleting a Pipeline: You can delete a pipeline. Data in the pipeline will also be deleted and cannot be restored. Exercise caution when performing this operation.
Limitations and Constraints
- A maximum of 20 pipelines can be created in a data space in a region for an account.
- Pipelines created by the system cannot be edited.
- Pipelines created by the system cannot be deleted.
Creating a Pipeline
- Log in to the management console.
- Click in the upper left corner of the management console and select a region or project.
- Click in the upper left corner of the page and choose Security & Compliance > SecMaster.
- In the navigation pane on the left, choose Workspaces > Management. In the workspace list, click the name of the target workspace.
Figure 1 Workspace management page
- In the navigation pane on the left, choose
. The security analysis page is displayed.Figure 2 Accessing the Security Analysis tab page
- (Optional) Add a data space. For details, see Adding a Data Space.
- In the data space navigation tree on the left, click on the right of the data space name and select Create Pipeline.
Figure 3 Create Pipeline
- On the Create Pipeline page, configure pipeline parameters. For details about the parameters, see Table 1.
Table 1 Parameters for creating a pipeline Parameter
Description
Data Space
Data space the pipeline belongs to. This parameter is generated by the system by default.
Pipeline Name
Name of the pipeline. The naming rules are as follows:
- The name can contain 5 to 63 characters.
- The name can contain letters, numbers, and hyphens (-). The name cannot start or end with a hyphen (-) or contain consecutive hyphens (-).
- The name must be unique in the data space.
Shards
The number of shards of the pipeline. The value ranges from 1 to 64.
An index can store a large amount of data that exceeds the hardware limits of a node. To solve this problem, Elasticsearch subdivides your index into multiple pieces called shards. When creating an index, you can specify the number of shards as required. Each shard is in itself a fully-functional and independent "index" that can be hosted on any node in the cluster.
Lifecycle
Lifecycle of data in the pipeline. Value range: 7 to 180
Description
Remarks on the pipeline. This parameter is optional.
- Click OK.
You can click the data space name or next to the data space to view the created pipeline.
Viewing Pipeline Details
- Log in to the management console.
- Click in the upper left corner of the management console and select a region or project.
- Click in the upper left corner of the page and choose Security & Compliance > SecMaster.
- In the navigation pane on the left, choose Workspaces > Management. In the workspace list, click the name of the target workspace.
Figure 4 Workspace management page
- In the navigation pane on the left, choose
. The security analysis page is displayed.Figure 5 Accessing the Security Analysis tab page
- In the data space navigation tree on the left, click a data space name to show the pipeline list.
Figure 6 Viewing pipeline details
- Click next to a pipeline name you want to view. The Pipeline Details pane is displayed on the right of the page.
Table 2 Pipeline parameters Parameter
Description
Workspace Name
Name of the workspace to which the pipeline belongs.
Workspace ID
ID of the workspace to which the pipeline belongs.
Data Space Name
Name of the data space to which the pipeline belongs.
Data Space ID
ID of the data space to which the pipeline belongs.
Pipeline Name
Name of the pipeline.
Pipeline ID
ID of the pipeline.
Shards
Number of shards of the pipeline.
Lifecycle
Retention period of the data in the pipeline.
Created
Time when the pipeline was created.
Description
Description of the pipeline.
Editing a Pipeline
- Log in to the management console.
- Click in the upper left corner of the management console and select a region or project.
- Click in the upper left corner of the page and choose Security & Compliance > SecMaster.
- In the navigation pane on the left, choose Workspaces > Management. In the workspace list, click the name of the target workspace.
Figure 7 Workspace management page
- In the navigation pane on the left, choose
. The security analysis page is displayed.Figure 8 Accessing the Security Analysis tab page
- In the data space navigation tree on the left, click a data space name to show the pipeline list.
Figure 9 Viewing pipeline details
- Click More > Edit next to the pipeline name.
Figure 10 Entry for editing a pipeline
- On the Edit Pipeline page, configure pipeline parameters. For details about the parameters, see Table 3.
Table 3 Parameters for editing a pipeline Parameter
Description
Data Space
Data space to which the pipeline belongs. This parameter cannot be modified.
Pipeline Name
Name you specified for the pipeline. The name cannot be changed after the pipeline is created.
Shards
The number of shards of the pipeline. The value ranges from 1 to 64.
Lifecycle
Lifecycle of data in the pipeline. Value range: 7 to 180
Description
Remarks on the pipeline. This parameter is optional.
- Click OK.
Deleting a Pipeline
- Log in to the management console.
- Click in the upper left corner of the management console and select a region or project.
- Click in the upper left corner of the page and choose Security & Compliance > SecMaster.
- In the navigation pane on the left, choose Workspaces > Management. In the workspace list, click the name of the target workspace.
Figure 11 Workspace management page
- In the navigation pane on the left, choose
. The security analysis page is displayed.Figure 12 Accessing the Security Analysis tab page
- In the data space navigation tree on the left, click a data space name to show the pipeline list.
Figure 13 Viewing pipeline details
- Click More > Delete next to the pipeline name.
Figure 14 Deleting a pipeline
- In the dialog box displayed, click OK.
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot