Help Center/ Cloud Search Service/ User Guide/ Using Logstash for Data Migration/ Configuring Networking for a Logstash Cluster/ Configuring Public Network Access for a Logstash Cluster
Updated on 2025-09-05 GMT+08:00
View PDF
Share

Configuring Public Network Access for a Logstash Cluster

CSS Logstash is a data ingestion service that ingests data from designated data sources on the public network and writes it to designated storage systems. The data is subsequently used for purposes such as big data analytics and log management. An NAT gateway and return routes need to be configured to ensure proper data ingestion, storage, and processing across networks.

  • NAT gateway: Configure an NAT gateway for the Logstash cluster to enable the cluster to initiate connection requests and connect to Internet data sources.
  • Return routes: They direct Logstash nodes to connect to the IP addresses of data sources on the public network and back, ensuring proper data ingestion and transmission.

Constraints

  • An NAT gateway and EIP are required to enable public network access for a Logstash cluster. They must be purchased additionally and be in the same region as the Logstash cluster.
  • The SNAT rule added to the NAT gateway applies only to resources in the specified subnet. Make sure you use the subnet of the Logstash cluster.

Configuring Public Network Access for a Logstash Cluster

  1. Obtain the VPC and subnet of the Logstash cluster for configuring an NAT gateway later.

    1. Log in to the CSS management console.
    2. In the navigation pane on the left, choose Clusters > Logstash.
    3. In the cluster list, click the name of the target cluster. The cluster information page is displayed.
    4. Click the Overview tab. In the Configuration area, record the cluster's VPC and subnet information.

  2. Create a public NAT gateway for the Logstash cluster. For details, see Creating a Public NAT Gateway.

    Table 1 describes the key parameters. Set other parameters based on service requirements.
    Table 1 Configuring a public NAT gateway

    Parameter

    Description

    Region

    Use the region of the Logstash cluster.

    Specifications

    Select a value based on the needed bandwidth. The maximum number of connections supported by SNAT varies depending on the specifications you choose.

    VPC

    Select the VPC recorded in 1.

    Subnet

    Select the subnet recorded in 1.

  3. Add SNAT rules on the NAT gateway to allow the Logstash cluster to access the public network through a shared EIP on the gateway. For details, see Adding an SNAT Rule.

    Table 2 describes the key parameters. Set other parameters based on service requirements.

    Table 2 Adding an SNAT rule

    Parameter

    Description

    Scenario

    Select VPC.

    CIDR Block

    Select Existing, and then select the VPC recorded in 1.

    Public IP Address Type

    Select EIP, and then select an IP address that meets your requirements.

    If no EIP is available, click View EIP to go to the EIP console and buy an EIP.

    You are advised to configure the EIP as follows:

    • Billing Mode: Pay-per-use
    • Billed By: Traffic

Configuring Routes for a Logstash Cluster

Configure a return route for the Logstash cluster to make sure the Logstash nodes can connect to the IP addresses of data sources on the public network. The return route must cover all these IP addresses.

  1. Log in to the CSS management console.
  2. In the navigation pane on the left, choose Clusters > Logstash.
  3. In the cluster list, click the name of the target cluster. The cluster information page is displayed.
  4. Click the Overview tab. In the Configuration area, click Add Route next to Cluster Route.
    Figure 1 Adding a route
  5. In the displayed dialog box, configure the route information.
    Table 3 Adding a route

    Parameter

    Description

    IP Address

    Enter the IP address (or a CIDR block) of a data source, for example, 10.10.1.0.

    If the IP addresses of data sources are scattered, configure multiple routes for them.

    Subnet Mask

    Enter the subnet mask used with the IP address.

    The subnet mask must align with the network part of the IP address. That is:
    • The network bits (the 1s) of the subnet mask must encompass those of the IP address.
    • The host bits (the 0s) of the subnet mask must accommodate the IP address's host bits.

    The subnet mask should not cover too wide a range, avoiding unnecessary IP address exposure or waste of the address space. 255.255.255.0 (that is, /24) is recommended, which applies to most internal networking scenarios.
  6. Click OK to add the route.

    After the route is added, click View Route to check the route information. You can click Add Route to add another route; or click Delete Route to delete one—In the Delete Route dialog box, select a route, enter DELETE manually, and then click OK.

Testing Connectivity

Test the connectivity between the Logstash cluster and the data sources.

  1. Log in to the CSS management console.
  2. In the navigation pane on the left, choose Clusters > Logstash.
  3. In the cluster list, click the name of the target cluster. The cluster information page is displayed.
  4. Select the Configuration Center tab, and click Test Connectivity.
  5. In the Test Connectivity dialog box, enter the IP address and port number of a data source, and click Test.
    Figure 2 Testing connectivity

    If Available is displayed, the network between the Logstash cluster and the data source is connected.

    You can test a maximum of 10 IP addresses at a time. Click Add to add multiple IP addresses and click Test at the bottom to test connectivity to all these IP addresses at the same time.