Analyzing Huawei Cloud ELB Access Logs for O&M Insights

Introduction

When distributing external traffic, Elastic Load Balance (ELB) logs details of HTTP and HTTPS requests, such as URIs, client IP addresses and ports, and status codes.

You can use ELB access logs for auditing or search for logs by time and keyword. You can also obtain external access statistics by running SQL aggregation queries. For example, you can check the number of requests with 404 responses within a certain day, or analyze the unique visitors (UVs) or page views (PVs) within a week.

Prerequisites

You have purchased and used a load balancer.

Restrictions

• ELB access logs only record layer 7 requests sent to the dedicated and shared load balancers. Layer 4 shared load balancing is not logged.

Procedure

1. Report ELB access logs to LTS.
   1. Log in to the management console.
   2. Click in the upper left corner to select the desired region and project.
   3. Click in the upper left corner and choose Networking > Elastic Load Balance.
   4. On the Load Balancers page, click the name of a load balancer.

      

   5. On the Access Logs tab, click Configure Access Log. Enable access logging, and select an LTS log group and log stream. If necessary, create a log group and a log stream first.
      Figure 1 Reporting ELB access logs to LTS
      
   6. Click OK.
      Figure 2 Access logs reported
      

2. Go to the log stream details page on the LTS console, choose Log Configuration in the navigation pane on the left, and click the Log Structuring tab. Select Structuring Template and select the ELB system template for log structuring. You can enable Quick Analysis if needed.
   Figure 3 Selecting the ELB structuring template
   

3. On the log stream details page, click Visualization and run SQL queries. For details about how to visualize query results, see "Log Structuring".
   • To count the PVs within a week, run the following SQL statement:

     select count(*) as pv
     Figure 4 PVs
     

   • To count the UVs within a week, run the following SQL statement:

     select count(distinct remote_port) as uv
     Figure 5 UVs
     

   • Statistics on 2xx/3xx/4xx/5xx (return codes) returned by all URIs in one day are collected to show the service execution result. The SQL query and analysis statements are as follows:

     select host, router_request_uri as url, count(*) as pv, 
     sum(case when status >= 200 and status < 300 then 1 else 0 end )  as "2xx times", 
     sum(case when status >= 300 and status < 400 then 1 else 0 end )  as "3xx times", 
     sum(case when status >= 400 and status < 500 then 1 else 0 end )  as "4xx times", 
     sum(case when status >= 500 and status < 600 then 1 else 0 end )  as "5xx times" 
     group by host, router_request_uri
     order by pv desc 
     limit 100

     You can visualize the results in a table, bar chart, line chart, pie chart, or number chart. Figure 6 presents the results in a bar chart.

     Figure 6 Response codes