Using LTS to Record WAF Logs
The WAF console can store attack logs for a maximum of 30 days. You can connect Log Tank Service (LTS) to WAF to store attack logs and access logs for a longer period. You can also use LTS to transfer logs to Object Storage Service (OBS) or Data Ingestion Service (DIS) for long-term storage.
- WAF can store attack logs for up to 30 days. For more details, see Querying a Protection Event.
- LTS: LTS can store attack logs and access logs. By default, logs are stored for 30 days. You can set the storage duration to 1 to 365 days. Logs that exceed the storage duration will be automatically deleted. For more details, see Log Tank Service (LTS).
To store logs for a long term, transfer them to Object Storage Service (OBS) or Data Ingestion Service (DIS).
If LTS is connected to WAF logs, LTS will be billed separately by traffic. For details about LTS pricing, see LTS Pricing Details.
Prerequisites
Impact on the System
Connecting WAF logs to LTS only records WAF logs to LTS. This does not affect WAF performance.
Managing WAF Logs
- Log in to the WAF console.
- Click
in the upper left corner and select a region or project.
- (Optional) If you have enabled the enterprise project function, in the upper part of the navigation pane on the left, select your enterprise project from the Filter by enterprise project drop-down list. Then, WAF will display the related security data in the enterprise project on the page.
- In the navigation pane on the left, click Events.
- Click Connect to LTS on the Log Settings tab if needed.
Table 1 LTS configuration parameters Parameter
Description
Example Value
Log Types
Select the log types you want to transfer to LTS. You can transfer WAF access logs and WAF attack logs.
Attack logs and access logs are in different formats. If you select both log types, you need to configure two different log streams.
WAF access logs and WAF attack logs
Log Group
Select the log group for log transfer. You can also click Create Log Group to create a log group.
A log group is the basic unit for LTS to manage logs. It comprises log streams and categorizes them. A log group does not store any log data. It only helps with log stream management. You can create up to 100 log groups for each account. For more details, see Managing Log Groups.
lts-group-waf
WAF Access Log Stream
If you select WAF access logs for Log Types, you need to configure a WAF access log stream. You can also click Create Log Stream to create a WAF access log stream. This stream logs key information about each HTTP access, including the access time, client IP address, and requested resource URL.
lts-topic-waf-attack
WAF Attack Log Stream
If you select WAF attack logs for Log Types, you need to configure a WAF access log stream. You can also click Create Log Stream to create a WAF attack log stream. This stream logs key information about each attack, including the attack type, protective action, and attack source IP address.
lts-topic-waf-access
The configuration takes about 10 minutes to take effect. After the configuration takes effect, LTS is billed by traffic. For details about LTS pricing, see LTS Pricing Details.
- Check or analyze logs.
After WAF is connected to LTS, created log groups (① in Figure 1) and log streams (② in Figure 1) for attack and access logs will be automatically displayed on the Log Settings tab. You can click WAF access log stream or WAF attack log stream to check, search, or analyze WAF logs. For more details, see Searching and Analyzing Logs.
- After selecting a log stream, on the Log Search tab (③ in Figure 1), choose (④ in Figure 1) to download the reported logs in the log stream.
- Frontend download: You can directly save log query results to a local PC. Download records will not appear in your log download history. Each time you can download up to 5,000 log records. You can download logs in .csv or .txt format.
- Offline backend download: You can download log files to a temporary OBS bucket via a backend task. Your browser must have public network access to download these files from your log download history. Each time you can download up to 20 million log records. You can download logs in .csv, .txt, or .json format.
You can also download log files through an OBS transfer task. For details, see Transferring Logs to OBS.
WAF access_log Field Description
Field |
Type |
Field Description |
Description |
---|---|---|---|
access_log.requestid |
String |
Random ID |
The value is the same as the last eight characters of the req_id field in the attack log. |
access_log.time |
String |
Access time |
GMT time a log is generated. |
access_log.connection_requests |
String |
Sequence number of the request over the connection |
- |
access_log.eng_ip |
String |
IP address of the WAF engine |
- |
access_log.pid |
String |
The engine that processes the request |
Engine (worker PID). |
access_log.hostid |
String |
Domain name identifier of the access request. |
Protected domain name ID (upstream_id). |
access_log.tenantid |
String |
Account ID |
Each Huawei Cloud account corresponds to a tenant ID. |
access_log.projectid |
String |
ID of the project the protected domain name belongs to |
Project ID of a user in a specific region. |
access_log.remote_ip |
String |
Remote IP address of the request at layer 4 |
IP address from which a client request originates.
NOTE:
If a layer-7 proxy is deployed in front of WAF, this field indicates the IP address of the proxy node closest to WAF. The real IP address of the visitor is specified by the x-forwarded-for and x_real_ip fields. |
access_log.remote_port |
String |
Remote port of the request at layer 4 |
Port used by the IP address from which a client request originates |
access_log.sip |
String |
IP address of the client that sends the request |
For example, XFF. |
access_log.scheme |
String |
Request protocol |
Protocols that can be used in the request:
|
access_log.response_code |
String |
Response code |
Response status code returned by the origin server to WAF. |
access_log.method |
String |
Request method. |
Request type in a request line. Generally, the value is GET or POST. |
access_log.http_host |
String |
Domain name of the requested server. |
Address, domain name, or IP address entered in the address bar of a browser. |
access_log.url |
String |
Request URL. |
Path in a URL (excluding the domain name). |
access_log.request_length |
String |
Request length. |
The request length includes the access request address, HTTP request header, and number of bytes in the request body. |
access_log.bytes_send |
String |
Total number of bytes sent to the client. |
Number of bytes sent by WAF to the client. |
access_log.body_bytes_sent |
String |
Total number of bytes of the response body sent to the client |
Number of bytes of the response body sent by WAF to the client |
access_log.upstream_addr |
String |
Address of the backend server. |
IP address of the origin server for which a request is destined. For example, if WAF forwards requests to an ECS, the IP address of the ECS is returned to this parameter. |
access_log.request_time |
String |
Request processing time |
Processing time starts when the first byte of the client is read (unit: s). |
access_log.upstream_response_time |
String |
Backend server response time |
Time the backend server responds to the WAF request (unit: s). |
access_log.upstream_status |
String |
Backend server response code |
Response status code returned by the backend server to WAF. |
access_log.upstream_connect_time |
String |
Time for the origin server to establish a connection to its backend services. Unit: second. |
When SSL is used, the time for the handshake process is also recorded. Time used for establishing a connection for a request. Use commas (,) to separate the time used for each request. |
access_log.upstream_header_time |
String |
Time used by the backend server to receive the first byte of the response header. Unit: second |
Response time for multiple requests. Use commas (,) to separate the time used for each response. |
access_log.bind_ip |
String |
WAF engine back-to-source IP address. |
The IP address of the NIC used by the engine for forwarding requests to the origin server. This value is not the EIP bound to the engine even if the engine forwards requests over the EIP. |
access_log.group_id |
String |
LTS log group ID |
ID of the log group for interconnecting WAF with LTS. |
access_log.access_stream_id |
String |
Log stream ID. |
ID of access_stream of the user in the log group identified by the group_id field. |
access_log.engine_id |
String |
WAF engine ID |
Unique ID of the WAF engine. |
access_log.time_iso8601 |
String |
ISO 8601 time format of logs. |
- |
access_log.sni |
String |
Domain name requested through SNI. |
- |
access_log.tls_version |
String |
Protocol versioning an SSL connection. |
TLS version for the request. |
access_log.ssl_curves |
String |
Curve group list supported by the client. |
- |
access_log.ssl_session_reused |
String |
SSL session reuse |
Whether the SSL session can be reused r: Yes .: No |
access_log.process_time |
String |
Engine attack detection duration (unit: ms) |
- |
access_log.args |
String |
The parameter data in the URL |
- |
access_log.x_forwarded_for |
String |
IP address chain for a proxy when the proxy is deployed in front of WAF. |
The sting includes one or more IP addresses. The leftmost IP address is the originating IP address of the client. Each time the proxy server receives a request, it adds the source IP address of the request to the right of the originating IP address. |
access_log.cdn_src_ip |
String |
Client IP address identified by CDN when CDN is deployed in front of WAF |
This field specifies the real IP address of the client if CDN is deployed in front of WAF.
NOTE:
Some CDN vendors may use other fields. WAF records only the most common fields. |
access_log.x_real_ip |
String |
Real IP address of the client when a proxy is deployed in front of WAF. |
Real IP address of the client, which is identified by the proxy. |
access_log.intel_crawler |
String |
Used for intelligence anti-crawler analysis. |
- |
access_log.ssl_ciphers_md5 |
String |
MD5 value of the SSL cipher (ssl_ciphers). |
- |
access_log.ssl_cipher |
String |
SSL cipher used. |
- |
access_log.web_tag |
String |
Website name. |
- |
access_log.user_agent |
String |
User agent in the request header. |
- |
access_log.upstream_response_length |
String |
Backend server response size. |
- |
access_log.region_id |
String |
Region where the request is received. |
- |
access_log.enterprise_project_id |
String |
ID of the enterprise project that the requested domain name belongs to. |
- |
access_log.referer |
String |
Referer content in the request header. |
The value can contain a maximum of 128 characters. Characters over 128 characters will be truncated. |
access_log.rule |
String |
Protection rule that the request matched. |
If multiple rules are matched, only one rule is displayed. |
access_log.category |
String |
Log category matched by the request. |
- |
access_log.waf_time |
String |
Time an access request is received. |
- |
access_log.geo |
String |
Mark of geographical location. |
|
WAF attack_log Field Description
Field |
Type |
Field Description |
Description |
---|---|---|---|
attack_log.category |
String |
Log category |
The value is attack. |
attack_log.time |
String |
Log time |
- |
attack_log.time_iso8601 |
String |
ISO 8601 time format of logs. |
- |
attack_log.policy_id |
String |
Policy ID |
- |
attack_log.level |
String |
Protection level |
Protection level of a built-in rule in basic web protection
|
attack_log.attack |
String |
Type of attack |
Attack type. This parameter is listed in attack logs only.
|
attack_log.action |
String |
Protective action |
WAF defense action.
|
attack_log.sub_type |
String |
Crawler types |
When attack is set to robot, this parameter cannot be left blank.
|
attack_log.rule |
String |
ID of the triggered rule or the description of the custom policy type. |
- |
attack_log.rule_name |
String |
Description of a custom rule type. |
This field is empty when a basic protection rule is matched. |
attack_log.location |
String |
Location triggering the malicious load |
- |
attack_log.resp_headers |
String |
Response header |
- |
attack_log.hit_data |
String |
String triggering the malicious load |
- |
attack_log.resp_body |
String |
Response body |
- |
attack_log.backend.protocol |
String |
Backend protocol. |
- |
attack_log.backend.alive |
String |
Backend server status. |
- |
attack_log.backend.port |
String |
Backend server port. |
- |
attack_log.backend.host |
String |
Backend server host value. |
- |
attack_log.backend.type |
String |
Backend server type. |
IP address or domain name. |
attack_log.backend.weight |
number |
Backend server weight. |
- |
attack_log.status |
String |
Response status code |
- |
attack_log.upstream_status |
String |
Origin server response code. |
- |
attack_log.reqid |
String |
Random ID |
The value consists of the engine IP address suffix, request timestamp, and request ID allocated by Nginx. |
attack_log.requestid |
String |
Unique ID of the request. |
Request ID allocated by Nginx. |
attack_log.id |
String |
Attack ID |
ID of the attack |
attack_log.method |
String |
Request method |
- |
attack_log.sip |
String |
Client request IP address |
- |
attack_log.sport |
String |
Client request port |
- |
attack_log.host |
String |
Requested domain name |
- |
attack_log.http_host |
String |
Domain name of the requested server. |
- |
attack_log.hport |
String |
Port of the requested server. |
- |
attack_log.uri |
String |
Request URL. |
The domain is excluded. |
attack_log.header |
A JSON string. A JSON table is obtained after the string is decoded. |
Request header |
- |
attack_log.mutipart |
A JSON string. A JSON table is obtained after the string is decoded. |
Request multipart header |
This parameter is used to upload files. |
attack_log.cookie |
A JSON string. A JSON table is obtained after the string is decoded. |
Cookie of the request |
- |
attack_log.params |
A JSON string. A JSON table is obtained after the string is decoded. |
Params value following the request URI. |
- |
attack_log.body_bytes_sent |
String |
Total number of bytes of the response body sent to the client. |
Total number of bytes of the response body sent by WAF to the client. |
attack_log.upstream_response_time |
String |
Time elapsed since the backend server received the response content from the upstream service. Unit: second. |
Response time for multiple requests. Use commas (,) to separate the time used for each response. |
attack_log.engine_id |
String |
Unique ID of the engine |
- |
attack_log.region_id |
String |
ID of the region where the engine is located. |
- |
attack_log.engine_ip |
String |
Engine IP address. |
- |
attack_log.process_time |
String |
Detection duration |
- |
attack_log.remote_ip |
String |
Layer-4 IP address of the client that sends the request. |
- |
attack_log.x_forwarded_for |
String |
Content of X-Forwarded-For in the request header. |
- |
attack_log.cdn_src_ip |
String |
Content of Cdn-Src-Ip in the request header. |
- |
attack_log.x_real_ip |
String |
Content of X-Real-IP in the request header. |
- |
attack_log.group_id |
String |
Log group ID |
LTS log group ID |
attack_log.attack_stream_id |
String |
Log stream ID |
ID of access_stream of the user in the log group identified by the group_id field. |
attack_log.hostid |
String |
Protected domain name ID (upstream_id). |
- |
attack_log.tenantid |
String |
Account ID |
- |
attack_log.projectid |
String |
ID of the project the protected domain name belongs to |
- |
attack_log.enterprise_project_id |
String |
ID of the enterprise project that the requested domain name belongs to. |
- |
attack_log.web_tag |
String |
Website name. |
- |
attack_log.req_body |
String |
Request body. (If the request body larger than 1 KB, it will be truncated.) |
- |
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot