Log Management Overview
Background
Unlike physical machines or VMs, cloud native containers have a short lifecycle and are easily destroyed and rebuilt. If logs are stored in the ephemeral storage of a container (such as local files or stdout), they will be permanently lost once the container is destroyed due to a restart caused by a fault, scale-in, or limited node resources. This makes it extremely difficult to troubleshoot historical faults, monitor service status, and perform security audits. Therefore, a highly reliable and centralized log collection and management solution is required for the serverless container environment.
Principles
CCI is deeply integrated with Log Tank Service (LTS) to provide users with one-stop log collection, transfer, and analysis.
- Cloud native collection mechanism: After you enable and configure a log collection policy, the underlying cloud native log collection plug-in of CCI uses mechanisms such as multi-level caching, priority queues, and resumable transfer to ensure log collection reliability.
- Supported log types
- Container standard output logs (stdout/stderr): Logs printed by service processes to the console
- Container file logs: Logs output to a specific path (for example, /var/log/app/*.log) in a container
- Log transfer: Collected logs are reported to the specified LTS log group and log stream in real time. You can search for logs in a unified manner. You can also transfer logs to OBS for long-term archiving and backup based on the storage period.
Typical Application Scenarios and Examples
- Scenario description: When an application crashes or returns a 500 error, developers need to immediately view the context logs before and after the crash to locate the root cause.
- Example: You have deployed a web service in CCI and configured LTS to collect the standard output (stdout) of the service. One day, the alarm system reports that the request failure rate increases. The O&M engineer logs in to the LTS console and enters the ERROR or a specific request ID to search for error logs of all pods for running the web service. The engineer quickly locates the exception caused by database connection timeout.
- Scenario description: Regulated industries such as finance and healthcare need to retain system operation logs and access logs for a long time for future audits.
- Example: According to the audit requirements, system access logs must be retained for at least half a year. After configuring CCI to report logs to LTS, you can use the log transfer function of LTS to automatically package and transfer logs to OBS for long-term retention. This meets security compliance requirements and reduces log storage costs.
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot