Logging

Overview

After workloads are scheduled to CCI, you can use the Cloud Native Log Collection add-on to collect pod logs, improving workload observability. This section describes how to enable logging for the workloads scheduled to CCI.

By default, for workloads scaled to CCI, container stdout logs will be collected and reported to AOM. AOM provides 500-MB free log storage space for each account each month. If the log storage space exceeds 500 MB, you will be billed based on the actual usage. For details about pricing, see Product Pricing Details.

To disable container stdout log collection, specify log.stdoutcollection.kubernetes.io: '{"collectionContainers": []}' in the pod annotation. The following is an example:

kind: Deployment
apiVersion: apps/v1
metadata:
  name: test
  namespace: default
spec:
  replicas: 1
  template:
    metadata:
      annotations:
        log.stdoutcollection.kubernetes.io: '{"collectionContainers": []}'
    spec:
      containers:
        - name: container-1
          image: nginx:latest

Log Collection Reliability

The log system's main purpose is to record all stages of data for service components, including startup, initialization, exit, runtime details, and exceptions. It is primarily employed in O&M scenarios for tasks like checking component status and analyzing fault causes.

Standard streams (stdout and stderr) and local log files use non-persistent storage. However, data integrity may be compromised due to the following risks:

• Log rotation and compression potentially deleting old files
• Temporary storage volumes being cleared when Kubernetes pods end
• Automatic OS cleanup triggered by limited node storage space

While the Cloud Native Log Collection add-on employs techniques like multi-level buffering, priority queues, and resumable uploads to enhance log collection reliability, logs could still be lost in the following situations:

• The service log throughput surpasses the collector's processing capacity.
• The service pod is abruptly terminated and reclaimed by CCE.
• The log collector pod experiences exceptions.

The following lists some recommended best practices for cloud native log management. You can review and implement them thoughtfully.

• Use dedicated, high-reliable streams to record critical service data (for example, financial transactions) and store the data in persistent storage.
• Avoid storing sensitive information like customer details, payment credentials, and session tokens in logs.

Constraints

Procedure

1. Install two add-ons: CCE Cloud Bursting Engine for CCI and Cloud Native Log Collection.
   1. Log in to the CCE console.
   2. Click the name of the target CCE cluster to go to the cluster console.
   3. In the navigation pane, choose Add-ons.
   4. Select the CCE Cloud Bursting Engine for CCI add-on and click Install.
   5. Enable the Networking option.

      

   6. On the Add-ons page, select the Cloud Native Log Collection add-on and click Install.

      

2. Create a workload on the CCE console.
   1. In the navigation pane, choose Workloads.
   2. Click Create Workload. For details, see Creating a Workload.
   3. Configure the parameters. For details, see Scheduling Pods to CCI.

      

3. Create a log collection policy.
   1. In the navigation pane, choose Logging.
   2. Click View Log Collection Policies. On the displayed page, click Create Log Collection Policy.
   3. Configure the parameters and click OK.

      

      

      Workloads scheduled to CCI do not support hot update of log policies. After a log collection policy is updated, redeploy the pods to apply the modification.

4. View the YAML file of the pod that is scheduled to CCI.

   

   To collect CCI pod logs, the Cloud Native Log Collection add-on injects the following annotations to the pod.

   Annotation	Example Value
   coordinator.cci.io/inject-volumes	'[{"name":"log-agent-conf","configMap":{"name":"log-agent-cci-logging-config","defaultMode":384},"namespace":"monitoring"},{"name":"log-agent-cert","secret":{"secretName":"log-agent-ca-secret","defaultMode":384},"namespace":"monitoring"}]'
   logconf.k8s.io/fluent-bit-configmap-reference	monitoring-log-agent-cci-logging-config
   logconfigs.logging.openvessel.io	'{"testcci001":{"container_files":{"container-1":"/var/test/*/*.log"},"regulation":""}}''
   sandbox-volume.openvessel.io/volume-names	log-agent-conf,log-agent-cert

5. View the reported logs in Logging.

   For details about logging, see Cloud Native Log Collection.