Updated on 2025-07-28 GMT+08:00

Overview

Log groups and log streams are the basic units for managing logs in LTS. Create log groups and streams before using LTS.

A log group comprises log streams and categorizes them, but does not store any log data itself. Instead, collected logs are stored in different log streams by log type, enabling further log grouping management.

Figure 1 Log management

Scenarios

Assume that an Internet company has two departments: Department F (Content Platform) and Department H (Enterprise Service). The following table lists their service scenarios and log categories (for reference only).

Table 1 Service scenario overview

Department

Log Description

Department F - Content Platform

  • Service scenarios: recommendation system (microservice), content management system (monolithic architecture), and Content Delivery Network (CDN)
  • Log types:
    • Recommendation system (microservices): user behavior logs (user_behavior_log) and model training logs (model_training_log)
    • Content management system (monolithic architecture): background operation logs (cms_operation_log)
    • CDN: CDN access logs (cdn_access_log) and edge point of presence (PoP) logs (edge_node_log)

Department H - Enterprise Service

  • Service scenarios: multi-tenant software system (microservice + containerization) for multiple users, for example, User A and User B
  • Log types:
    • Common tenant logs: application access logs (app_access_log) and database slow query logs (slow_query_log)
    • Special requirement: User A requires that tenant logs be viewed independently.
Table 2 Log grouping

Department

Log Group

Log Stream

Tag 1 (Tag Key=Tag Value)

Tag 2 (Tag Key=Tag Value)

Department F

F_content_platform_logset

recommend_user_behavior_log

department=department F

service line=recommendation system

recommend_model_training_log

department=department F

service line=recommendation system

cms_operation_log

department=department F

service line=content management

cdn_access_log

department=department F

service line=CDN

Department H

H_enterprise_services_logset

tenantA_app_access_log

department=department H

tenant=user A

tenantB_app_access_log

department=department H

tenant=user B

H_public_services_logset

slow_query_log

department=department H

type=database

When managing the preceding logs with LTS, you can add tags to log groups and log streams. Each tag consists of a tag key and a tag value, which help quickly identify, search for, and manage target objects. Create log groups and log streams for different departments and configure tags by referring to Table 2.

  • Create a log group for each department to manage logs of the department in a unified manner. In this log group, you can create multiple log streams to store logs from different service lines. After adding a new project, simply create a log stream in the log group.

    For Department H, create two log groups to separate user and database logs, simplifying maintenance.

  • You can use Tag 1 to distinguish different departments, while using Tag 2 to distinguish logs of different service lines within the same department. With LTS permissions management, you can control access to logs, ensuring that personnel from each department can only view their own department's data. In conclusion, tags enable:
    1. Multi-dimensional permission isolation: Define different tags to isolate departments, service lines, and tenants, meeting the log data management requirements of different tenants.
    2. Dynamic expansion and flexible adjustment: If User B is transferred from Department H to Department L, simply change the tag of user B's log stream (tenantB_app_access_log) to department=department L. You do not need to migrate the log data.
    3. Quick search and analysis: You can quickly search for logs (such as slow query logs, reconciliation logs, and transaction logs) of other departments by tag type (such as database and billing). This allows you to analyze database performance of the entire company or collect statistics on each department's fees.