Analyzing Huawei Cloud ELB Access Logs for O&M Insights

Introduction

When distributing external traffic, Elastic Load Balance (ELB) logs details of HTTP and HTTPS requests, such as URIs, client IP addresses and ports, and status codes.

You can use ELB access logs for auditing or search for logs by time and keyword. You can also obtain external access statistics by running SQL aggregation queries. For example, you can check the number of requests with 404 responses within a certain day, or analyze the unique visitors (UVs) or page views (PVs) within a week.

Prerequisites

You have and used a Huawei Cloud load balancer.

Restrictions

ELB access logs only record layer 7 requests sent to the dedicated and shared load balancers. Layer 4 shared load balancing is not logged.

Procedure

1. Report ELB access logs to LTS. For details, see Access Logging.
   1. Log in to the management console.
   2. Click in the upper left corner to select the desired region and project.
   3. Click in the upper left corner and choose Networking > Elastic Load Balance.
   4. On the Load Balancers page, click the name of a load balancer.
   5. On the Access Logs tab, click Configure Access Log. Enable access logging, and select an LTS log group and log stream. If necessary, create a log group and a log stream first.
      Figure 1 Reporting ELB access logs to LTS
      
   6. Click OK.

2. Click in the upper left corner and choose Management & Governance > Log Tank Service.
3. On the Log Management page, click the target log stream name. On the page that is displayed, click in the upper right corner. On the Cloud Structuring Parsing tab page, select Structuring Template and select the ELB system template for log structuring. You can enable Quick Analysis if needed. For details about quick analysis, see Setting Quick Analysis.
   Figure 2 Selecting the ELB structuring template
   

4. On the log stream details page, click Visualization and run SQL queries. For details about how to visualize query results, see Log Structuring.
   • To count the PVs within a week, run the following SQL statement:

     select count(*) as pv

   • To count the UVs within a week, run the following SQL statement:

     select count(distinct remote_port) as uv

   • Statistics on 2xx/3xx/4xx/5xx (return codes) returned by all URIs in one day are collected to show the service execution result. The SQL query and analysis statements are as follows:

     select host, router_request_uri as url, count(*) as pv, 
     sum(case when status >= 200 and status < 300 then 1 else 0 end )  as "2xx times", 
     sum(case when status >= 300 and status < 400 then 1 else 0 end )  as "3xx times", 
     sum(case when status >= 400 and status < 500 then 1 else 0 end )  as "4xx times", 
     sum(case when status >= 500 and status < 600 then 1 else 0 end )  as "5xx times" 
     group by host, router_request_uri
     order by pv desc 
     limit 100

     The query results can be displayed in a table, bar chart, line chart, pie chart, or number chart.