Collecting Kubernetes Logs from Third-Party Clouds, IDCs, and Other Huawei Cloud Regions to LTS

Solution Overview

Cloud users often need to collect Kubernetes logs across clouds or regions. There are two typical scenarios:

• Scenario 1: collecting logs from IDCs or third-party clouds to Huawei Cloud LTS
  Figure 1 Third-party cloud log collection
  
• Scenario 2: collecting logs from one Huawei Cloud region to LTS in another Huawei Cloud region
  Figure 2 Cross-region log collection
  

In both scenarios, you need to establish a network connection, install ICAgent, and follow the log ingestion wizard.

• ICAgent: the log collector of Huawei Cloud LTS. After being installed on a host, it collects logs from the host to LTS.
  

  Ensure that the time and time zone of your local browser are consistent with those of the host to install ICAgent.

• Networking
  • Scenario 1: Direct Connect is a typical method for connecting a customer-built IDC or third-party cloud to Huawei Cloud. If Direct Connect is unavailable, you can use a VPN or public IP address.
  • Scenario 2: Cloud Connect or Direct Connect is a typical method for interconnecting Huawei Cloud regions. You can also use a VPN or public IP address.
• Jump server
  • ICAgent installed in customer-built IDCs, third-party clouds, or other Huawei Cloud regions cannot directly access the network segment used by the Huawei Cloud management plane for log reporting, necessitating a jump server for data forwarding. Use the jump server solution for Proof of Concept (PoC) tests or when log traffic is light. If you do not want to use jump servers for heavy traffic scenarios in production environments, submit a service ticket to Huawei Cloud technical support to design a network passthrough solution.
  • A typical jump server configuration is 2 vCPUs and 4 GB memory, allowing it to forward traffic at approximately 30 MB/s. Configure a proper number of jump servers based on your log traffic and use a load balancer to distribute traffic among them.

This practice describes how to collect Alibaba Cloud host logs to Huawei Cloud LTS. The method is similar to that of collecting logs from customer-built IDCs or across Huawei Cloud regions.

Below are the steps to collect the logs from a Linux host in Alibaba Cloud's China (Beijing) region to LTS in Huawei Cloud's CN East-Shanghai1 region.

Planning Resources

Purchasing a Load Balancer and an ECS as a Jump Server in Huawei Cloud CN East-Shanghai1

1. Log in to the ECS console and buy an ECS.

   Before installing ICAgent on a non-Huawei Cloud host, buy an ECS as a jump server from Huawei Cloud.

2. Buy a load balancer, add TCP listeners, and associate a backend server group with it.
   1. Add listeners for TCP ports 30200, 30201, 8149, 8923, and 8102. For details, see Adding a TCP Listener.
   2. Add the jump server to a backend server group. For details, see Backend Server.

3. Configure a security group rule for the jump server and open forwarding ports.
   1. Modify the security group rule used by the jump server.
      1. On the ECS console, click the name of the ECS used as the jump server to go to the details page.
      2. On the Security Groups tab page, click a security group name to go to the details page.
      3. Click the Inbound Rules tab and click Add Rule. Open the inbound ports 8149, 8102, 8923, 30200, 30201, and 80 to ensure that data can be transmitted from the non-Huawei Cloud host to the jump server.
      Figure 3 Modifying a security group rule
      
   2. On the LTS console, choose Host Management > Hosts in the navigation pane, and click Install ICAgent in the upper right corner. Set parameters as shown in the following figure. Set Private IP to the private IP address of the ECS to generate an installation password.
      Figure 4 Installing ICAgent
      
   3. Copy the command, log in to the jump server as user root, run the SSH tunneling command, and enter the password of user root as prompted.
   4. Run the netstat -lnp | grep ssh command to check whether the corresponding TCP ports are being listened to. If the command output similar to the following is returned, the ports are open.
      Figure 5 Viewing ports
      
      

      • Enter http://Jump server IP address in the address bar of a browser. If the access is successful, the security group rule has taken effect.
      • If the jump server is powered off and then restarted, run the installation command generated on the ICAgent installation page again. If you use the jump server in a production environment, configure the SSH tunneling command to run upon system startup.

Configuring Log Ingestion

For a Kubernetes cluster, simply install ICAgent on one node, not all nodes.

Obtain an AK/SK in advance. For details, see How Do I Obtain an Access Key (AK/SK)?

1. Configure the jump server.
   1. On the ECS console, locate the jump server and obtain its private IP address.
      Figure 6 Obtaining the private IP address
      
   2. On the LTS console, choose Host Management > Hosts in the navigation pane and click Install ICAgent. On the page displayed, set parameters as follows, set Private IP to the private IP address of the ECS to generate an installation command, and copy the command.
      Figure 7 Installing ICAgent
      
   3. Log in to the ECS, run the command copied in the previous step, and enter the node password as prompted. If no error is reported, the installation is successful.
      Figure 8 Running the generated installation command
      
   4. On the Install ICAgent page, set Connection IP to the public IP address of the jump server. Ensure that the checkbox next to Turn off command history to prevent the AK/SK from being stored is selected.
   5. Copy the ICAgent installation command and run it on the jump server. Enter the AK and SK of the current account as prompted. If the message ICAgent install success is displayed, ICAgent is successfully installed.

2. Configure a log ingestion rule. For details, see Ingesting Self-Built Kubernetes Application Logs to LTS.

Viewing a Log Stream

On the Log Management page of LTS, click the target log stream to go to its details page. If there are logs, the Alibaba Cloud Kubernetes logs have been reported to LTS.