Collecting Syslog Aggregation Server Logs to LTS

Introduction

System Logging Protocol (Syslog) is a data protocol used by network devices to collect logs to a logging server. It can record log messages of multiple event types and is supported by almost all network devices, such as routers, switches, and printers. Even Unix-like servers can generate syslog messages to record user logins, firewall events, and Apache or Nginx access.

Syslog defines related format specifications based on RFC5424 and RFC3164. RFC3164 was released in 2001, and RFC5424 was an upgraded version released in 2009. The new version is compatible with the old version and solves many problems of the old version. Therefore, RFC 5424 is recommended.

This section describes how to use the syslog protocol to upload logs to LTS. You need to buy an ECS as a syslog aggregation server. Rsyslog comes preinstalled by default on Linux servers. However, Huawei Cloud ECSs do not receive remote syslog writes by default. You need to enable this function.

Solution Overview

Figure 1 Solution flowchart

• You can buy a Linux ECS and configure it as a syslog aggregation server to receive log data from other devices. If the size of a log received by a syslog server exceeds 1,024 bytes, the log will be truncated.
• The log processing rate of a single syslog server is 10 MB/s. To process a large number of logs or ensure high reliability, you can buy multiple ECSs as syslog servers and configure load balancers for distributing traffic.
• You need to install ICAgent on syslog servers and configure log collection rules to collect logs to LTS.

Planning Resources

Buy two ECSs. One serves as a syslog aggregation server, and the other serves as a service ECS to simulate user systems or devices to send logs.

Buying an ECS

1. Log in to the management console and choose Compute > Elastic Cloud Server.
2. Buy an ECS as a syslog aggregation server.
   

   You are advised to use CentOS 6.5 64bit or later images. The recommended specifications are 2 vCPUs and 4 GB of memory.

3. Log in to the syslog server as user root to install ICAgent.
   1. Allow TCP ports 30200, 30201, 8149, 8923, and 8102 in the outbound rules of the syslog server, and then allow UDP port 514 in the inbound rules as the default listening port.

   2. Go to the LTS console and choose Host Management > Hosts in the navigation pane.
   3. On the page displayed, click Install ICAgent. Set OS to Linux and Host to Intra-region hosts. Then, select Obtain AK/SK for Installation Mode. Click Copy Command to copy the ICAgent installation command and manually replace the AK/SK.

   4. Log in to the syslog server as user root and run the ICAgent installation command. If the message ICAgent install success is displayed, ICAgent is successfully installed. You can then view the ICAgent status by choosing Host Management in the navigation pane of the LTS console and then clicking Hosts.

4. Enable the rsyslog listening and receiving functions.
   By default, rsyslog of Huawei Cloud ECSs does not receive remote syslog writes. You need to manually enable this function.

   1. Log in to the ECS.
   2. Modify the rsyslog configuration file.

      vi /etc/rsyslog.conf

   3. Add the following content to the configuration file to enable TCP and UDP remote receiving:

      Provides UDP syslog reception 
      $ModLoad imudp 
      $UDPServerRun 514
      
      Provides TCP syslog reception 
      $ModLoad imtcp 
      $InputTCPServerRun 514

   4. Save the settings. Click More > Restart in the Operation column to restart the ECS.

   5. Run any of the following commands. If the command output is normal, the service is working.

      Run service rsyslog status to check whether the rsyslog running status is running.

      Figure 2 Checking the rsyslog status
      

      Run systemctl status rsyslog to check whether the rsyslog running status is running.

      Figure 3 Rsyslog status
      

      Run netstat -anp | grep 514 to check whether the listening function is enabled.

      Figure 4 Checking whether listening is enabled
      

5. Configure syslog collection.
   1. Choose Host Management > Host Groups in the LTS navigation pane and click Create Host Group. On the displayed page, enter a host group name and select hosts.

   2. Choose Log Ingestion in the navigation pane. On the displayed page, click ECS (Elastic Cloud Server).
   3. Select a log stream.
   4. Select host groups.
   5. Set the collection path to /var/log/messages.

6. Log in to the service ECS for verification.

   After your service system or device generates logs, you can view the logs on the LTS console. Log in to the service ECS and run the logger -n x.x.x.x -P 514 testremotelog command to send syslog messages to the aggregation server. x.x.x.x indicates the IP address (public or private) of the syslog server. testremotelog indicates the log content, which can be customized.

   After the command is executed, you can view the log in the configured log group and log stream.

   Alternatively, log in to the syslog aggregation server and check whether the testremotelog log exists in /var/log/messages.

   tail -f /var/log/messages

   Figure 5 Checking whether the testremotelog log exists
   

7. Use multiple syslog servers and load balancers to implement load balancing.

   The log processing rate of a single syslog server is 10 MB/s. To process a large number of logs, you can use multiple syslog servers and load balancers.

   1. Create syslog aggregation servers and install ICAgent.
   2. Create a load balancer. For details, see Using ELB to Distribute Traffic to a Web Application Across ECSs.
   3. Add listeners for TCP/UDP ports and port 514. For details, see Adding a Listener.
   4. Add backend servers to the backend server group. For details, see Backend Server Group.