Connecting CBH Logs to SecMaster

Notes and Constraints

• The component controller (isap-agent) of the log collector can only run on Linux x86 or Arm ECSs.
• Only a non-administrator IAM user can log in to the console and check information for installing the component controller.
• The CBH instance and SecMaster must belong to the same account, so that they can communicate with each other through a VPC.

Prerequisites

• You have enabled the paid SecMaster and created a workspace.

  For details, see Buying SecMaster and Creating a Workspace.

Integrating CBH Logs into SecMaster

1. Prepare an ECS and install the log collector on the ECS for log collection. Make sure the system disk capacity for the ECS is not less than 50 GB.

   If you already have an ECS that meets the requirements, skip this step.

   If you need to buy an ECS, see Buying an ECS.

2. Buy data disks to ensure that the log collector has sufficient running space.

   An idle data disk with capacity not less than 100 GB is required for the ECS you plan to install the log collector. This data disk is used for collection management. The data disk must be in the same AZ as the ECS.

   If you have purchased an ECS and configured data disks by referring to Buying an ECS, skip this step. Otherwise, refer to Buying a Data Disk to buy for a data disk.

3. Attach the data disk to the ECS that meets the requirements.

   Attach the data disk to the ECS that meets the requirements to ensure that the log collector has sufficient running space. For details, see Attaching a Data Disk.

   Skip this step in any of the following scenarios:

   • Scenario 1: You have purchased an ECS and a data disk that meet the requirements by referring to (Optional) Step 1: Buy an ECS and the disk has been attached to the ECS.
   • Scenario 2: You already have an ECS that meets the requirements (not purchased by referring to (Optional) Step 1: Buy an ECS), and a data disk that meets the requirements and is purchased based on (Optional) Step 2: Buy a Data Disk. The data disk has been attached to the ECS during the purchase.

4. Create a non-administrator IAM account.

   IAM authentication is used for tenant log collection. So you need to create an IAM user (machine-machine account) with the minimum permission to access SecMaster APIs. MFA must be disabled for the IAM user. This account is used by the tenant-side log collector to log in to and access SecMaster. For details, see Creating a Non-Administrator IAM Account.

5. Before collecting data, you need to establish the network connection between the tenant VPC and SecMaster. For details, see Step 5: Configure Network Connection.
6. Install the component controller (isap-agent) and manage log collector nodes (ECSs) to SecMaster. For details, see Installing the Component Controller (isap-agent).
7. Install the log collection component (Logstash) and configure the log collection process. For details, see Installing the Log Collection Component (Logstash).
8. Create a log storage location (pipeline) in SecMaster for log storage and analysis.

   (Optional) Add a data space for importing data. If a third-party data storage space is available, you do not need to add a data space.

   (Optional) Add a storage pipeline in the new data space for importing data. If a storage pipeline is available, you do not need to add a storage pipeline.

   1. Go to the target workspace management page. In the navigation pane on the left, choose Log Audit > Security Data.
      Figure 2 Accessing the Security Analysis tab
      

   2. Add a data space.
      1. In the upper left corner of the data space list, click Add. The Add Data Space panel is displayed on the right.
         Figure 3 Add Data Space
         
      2. On the Add Data Space panel, set the parameters for the new data space.

         Table 2 Add Data Space

         Parameter	Description
         Data Space	Enter a data space name. The name must meet the following requirements: The name can contain 5 to 63 characters. Only letters, numbers, and hyphens (-) are allowed. The name cannot start or end with a hyphen (-) or contain consecutive hyphens (-). The name cannot be the same as any other data space name on Huawei Cloud. Example: DataTransfer
         Description	(Optional) Remarks of the data space.

      3. Click OK.
   3. In the data space navigation tree on the left, click on the right of the data space name created in 8.b and select Create Pipeline.
      Figure 4 Creating a pipeline
      
   4. On the Create Pipeline page, configure pipeline parameters. For details about the parameters, see Table 2.

      Table 3 Creating a pipeline

      Parameter	Description
      Data Space	Data space to which the pipeline belongs, which is generated by the system by default.
      Pipeline Name	Name of the pipeline. The name must meet the following requirements: The name can contain 5 to 63 characters. Only letters, numbers, and hyphens (-) are allowed. The name cannot start or end with a hyphen (-) or contain consecutive hyphens (-). The name must be unique in the data space. Example: cbh_log
      Shards	The number of shards of the pipeline. The value ranges from 1 to 64. Retain the default value. An index can potentially store a large amount of data that exceeds the hardware limits of a single node. To solve this problem, Elasticsearch subdivides your index into multiple pieces called shards. When creating an index, you can specify the number of shards as required. Each shard is hosted on any node in the cluster, and each shard is an independent and fully functional index.
      Lifecycle	Lifecycle of data in the pipeline. Value range: 7 to 180 Retain the default value.
      Description	Remarks on the pipeline. This parameter is optional.

   5. Click OK. After the pipeline is created, you can click the data space name to check the created pipeline.
   

   You are advised to use different pipelines to store data from different sources so that you can query and analyze the data.

9. Configure the connector. You need to configure log source and log destination parameters.
   1. Go to the workspace management page. In the navigation pane on the left, choose Log Audit > Collections.
      Figure 5 Accessing the Connections page
      
   2. Add a source for the data connection.
      1. On the Connections tab, click Add.
      2. Configure the data connection source details.

         The following uses UDP as an example to describe how to add a log data source. For details about other connection types, see Connector Rules.

         Table 4 Log source settings

         Parameter	Description
         Connection Method	Select Source.
         Connection Type	Select UDP.
         Title	Name of the data connection source. In this example, in-cbh-log is used.
         Description	Enter a brief description of the data source. In this example, "CBH logs" is used.
         Port	Set the port for the data connection. In this example, 9876 is used. The port must be the same as the CBH forwarding port.
         Codec	Set the encoding format. In this example, select Plain
         Advanced Settings	No configuration is required.

      3. After the setting is complete, click Confirm in the lower right corner of the page.
   3. Add a destination for the data connection.
      1. On the Connections tab, click Add.
      2. Configure the data connection destination details.

         Table 5 Log transfer destination

         Parameter	Description
         Connection Method	Select Destination.
         Connection Type	Select SecMaster.
         Title	Enter a custom name for the data connection destination. In this example, enter "out-pipe-cbh-log."
         Description	Enter a custom description of the data connection destination. In this example, "CBH logs" is used.
         Type	User-defined log destination type. Select Tenant.
         Pipe	Select the pipe created in 8. In this example, select cbh_log.
         Domain_name	Enter the domain account information of the IAM user used to log in to the console.
         User_name	Enter the user information of the IAM user used to log in to the console.
         User_password	Enter the password of the current login IAM user. (If the password is changed, the service cannot be accessed.)
         Advanced Settings	No configuration is required.

      3. After the setting is complete, click Confirm in the lower right corner of the page.

10. Add a parser created from the parser template CBH system log parsing.
    1. Go to the target workspace management page. In the navigation pane on the left, choose Log Audit > Connections. On the displayed page, click the Parsers tab.
    2. On the Parsers tab, click the Templates tab. In the Operation column of the parser template named Log analysis of cloud fortress machine, click Create by Template. The Create Parser page is displayed.
    3. On the Create Parser page, configure parser parameters.
       • Name: Retain the default value Log analysis of cloud fortress machine.
       • Description: Retain the default value. The default value is Log analysis of cloud fortress machine.
       • Parsing rule: Retain the default value.
    4. Click OK. Return to the Parsers > Parsers page to check the created parser.

11. Configure log collection channels, connect each functional component, and ensure that SecMaster and log collectors work properly.
    1. Go to the workspace management page. In the navigation pane, choose Log Audit > Collections. On the displayed page, click the Collection Channels tab.
       Figure 6 Accessing the Collection Channels tab
       
    2. Add a log collection channel group.
       1. On the Collection Channels tab, click on the right of Groups.
       2. Enter a group name, for example, CBH logs, and click .
    3. Create a log collection channel.
       1. On the right of the group list, click Add.
       2. In the Configure Basic Configuration step, configure basic information.

          Table 6 Basic information parameters

          Parameter		Description
          Basic Information	Title	Enter a custom collection channel name. In this example, "CBH logs" is used.
          	Channel grouping	Select the group created in 11.b. In this example, "CBH log" is used.
          	(Optional) Description	Enter the description of the collection channel.
          Configure Source	Source Name	Select the log source name added in 9. In this example, select in-cbh-log. After you select a source, the system automatically generates the information about the selected source.
          Destination Configuration	Destination Name	Select the log destination name added in 9. In this example, select out-pipe-cbh-log. After you select a destination, the system automatically generates the information about the selected destination.

       3. Click Next in the lower right corner of the page.
       4. On the Configure Parser page, select Log analysis of cloud fortress machine and add raw logs to the collection channel list. After the parser is configured, click Next in the lower right corner.
       5. On the Select Node page, click Create. In the displayed dialog box, select the ECS node managed in 6 and click OK.
          Figure 7 Selecting a node
          
    4. Click Next in the lower right corner of the page.
    5. On the Preview Channel Details page, confirm the configuration and click Save and Execute.
    6. On the Collection Channels tab, if the health status of a collection channel is Normal, the collection channel is successfully delivered.
       Figure 8 Collection channels configured
       

12. Go to the CBH system and configure data forwarding.
    1. Log in to the CBH system as user admin. Make sure that you have the permission for the System module.
    2. Choose System > Data Maintain > Log Backup.
    3. In the Backup to the syslog server area, click Edit. In the displayed dialog box, complete required parameters.

    Table 7 Parameters for configuring the Syslog server

    Parameter	Description
    Status	Enable or disable backup to the syslog server. The default value is . You need to enable this function manually. : Backing up logs to the syslog server is enabled. The system automatically starts backup at 00:00 every day. : Backing up logs to the syslog server is disabled.
    Sender Identifier	Identifier for connecting your bastion host to the Syslog server. The identifier is used to identify the bastion host from which the logs are received on the Syslog server. In this example, the value is cbh.
    Server IP	Enter the IP address of the ECS node you select when configuring the collection channel in 11.c.v.
    Port	The port must be the same as that configured for SecMaster log collection management. In this example, port 9876 is configured for SecMaster and CBH.
    Protocol	Protocol of the Syslog server. In this example, UDP is selected.
    Backup Content	Select at least one type of logs to be backed up. In this example, all types are selected. System login logs: record all logins to the instance. Resource login logs: record all operations for resources managed by the current instance. Command operation logs: record all commands executed in the current instance. File operation logs: record all file operations, including uploading and downloading files, in the instance. Two-person authorization logs: record all two-person authorization operations in the instance.

    4. Click OK. You can then view the backup information on the Log Backup tab.

13. After logs are integrated into the SecMaster pipeline, query logs on SecMaster.
    1. Go to the target workspace management page. In the navigation pane on the left, choose Log Audit > Security Data. Select the pipeline (cbh_log in this example) created or used in 8 to view the data after log parsing.
    2. For details about how to query and analyze logs, see .