Ingesting Firewall Logs to LTS

Scenario

By default, the CFW log query function allows you to query logs generated in the last seven days, which may not meet the requirements of long-term security audit and complex event analysis. If you need to extend the log retention period or export a large amount of log data, you can ingest one or more types of logs to Log Tank Service (LTS). After the ingestion, the log retention period can be extended to 1 to 365 days. With CFW logs recorded by LTS, you can quickly and efficiently perform real-time decision-making analysis, device O&M management, and service trend analysis. LTS processes and analyzes large volumes of logs. It allows you to manage logs in real time, efficiently, and securely.

This section describes how to ingest CFW logs to LTS.

Billing

LTS is billed separately by traffic. For details about LTS pricing, see LTS Pricing.

Notes and Constraints

• There is a delay of approximately 10 minutes before the configuration takes effect.

Procedure

1. Create log groups and log streams.
   • A log group is the basic unit for LTS to manage logs. It comprises log streams and categorizes them. A log group does not store any log data. It only helps with log stream management.

     To facilitate subsequent query, you are advised to add the -cfw suffix to the log group name when creating the log group. For details about how to create a log group, see Creating a Log Group.

   • LTS manages logs by log stream. Each log stream belongs to exactly one log group, while a log group can contain multiple log streams. Different types of collected logs are classified and stored in different log streams for easier management.

     To facilitate subsequent query, you are advised to add the suffixes -attack, -access, and -flow to attack event logs, access control logs, and traffic logs, respectively, when creating log streams. For details about how to create a log stream, see Creating a Log Stream.

2. Log in to the CFW console.
3. Click in the upper left corner of the management console and select a region or project.
4. (Optional) Switch to another firewall instance. If there are multiple firewall instances, you can select a desired instance from the drop-down list in the upper left corner of the page.
5. In the navigation pane on the left, choose Log Audit > Log Management.
6. Click Connect to LTS.
7. In the Log Types column, select log types.
   One or more types of logs can be recorded in LTS. The formats of attack logs, access logs, and traffic logs are different. You need to configure different log streams for them.

   • Attack logs: record attack alarm information, including the attack event type, protection rule, protection action, quintuple, and attack payload.
   • Access logs: record information about the traffic that matches the ACL policy, including the matching time, quintuple, response action, and the matched access control rule.
   • Traffic logs: record information about all traffic passing through the firewall, including the start time, end time, 5-tuple data, number of bytes, and number of packets.

8. Select a created log group or log stream.
9. Click OK.

   There is a delay of approximately 10 minutes before the configuration takes effect. If a message appears indicating insufficient permissions, grant the LTS FullAccess permission.

   For details about permission granting, see Using IAM to Grant Access to CFW.

Follow-up Procedure

• Log query and analysis: After the configuration is complete, firewall logs are uploaded to LTS for management. You can use LTS to perform various operations on firewall logs, such as keyword search and visual analysis. For details, see Log Search and Analysis.
• Log visualization: Log data is displayed in charts to meet visualization requirements in different scenarios for O&M and operational analysis. For details, see Log Visualization.
• Alarm rule configuration: LTS monitors keywords in logs and collects statistics on the occurrences of keywords in logs within a specified period to monitor the service running status in real time. For details, see Log Alarms.
• Viewing log fields: For details about log fields and fields that can be indexed, see Log Field Description.

References

For more information about LTS, see What Is LTS?