Collecting the Logs of Pods Scheduled to CCI and Reporting Them to LTS

Scenario

When service traffic bursts, the CCE Cloud Bursting Engine for CCI add-on schedules pods in a CCE cluster to CCI. You need to collect and store the logs (including container standard output logs and container file logs) of pods running on CCI to LTS for centralized management.

This practice describes how to configure a log collection policy in a CCE cluster to automatically report such logs to LTS.

Prerequisites

• You have created a CCE standard or Turbo cluster, and nodes in the cluster are running normally.
• VPC endpoints have been configured for the VPC where the CCE cluster is running.

  Reason: Pods scheduled to CCI depend on the VPC endpoints for accessing SWR and OBS to pull images from SWR. For more information, see Purchasing VPC Endpoints.

Step 1: Install the Cloud Native Log Collection Add-on

1. Log in to the CCE console and click the name of the target cluster to access the cluster console. Choose Add-ons in the navigation pane and install the Cloud Native Log Collection add-on.

   

2. Configure add-on specifications based on the number of logs and click Install.

   

   If the add-on is in the Running state, the add-on has been installed and is running normally.

   

Step 2: Install the CCE Cloud Bursting Engine for CCI Add-on

1. Go to the Add-ons page and install the CCE Cloud Bursting Engine for CCI add-on.

   

2. Configure add-on specifications, enable Network Connectivity, and click Installation Using YAML.

   

3. Set enableFullPathLogCollect and enableLogCollection to true and click Submit.
   

   • If the add-on has been installed, click Edit to update the settings.
   • enableLogCollection: enables log collection.
   • enableFullPathLogCollect: enables full-path log collection.

   

   If the add-on is in the Running state, the add-on has been installed and is running normally.

   

Step 3: Configure a Log Collection Policy

You can collect container standard output logs or container file logs as needed.

Container standard output logs: Logs output to stdout/stderr are collected.

Container file logs: Logs are collected from a specified path (for example, /var/log/app.log).

Configuring a Log Collection Policy for Container Standard Output Logs

1. In the navigation pane, choose Settings. On the Monitoring tab, enable the collection of container standard output logs and click Confirm Settings.

   

   

2. Create a workload. The following is example container standard output:

   kind: Deployment
   apiVersion: apps/v1
   metadata:
     name: c1
     namespace: default
     labels:
       bursting.cci.io/burst-to-cci: enforce # Configure a forcible scheduling policy to schedule pods to CCI 2.0.
       version: v1
   spec:
     replicas: 1
     selector:
       matchLabels:
         app: c1
         version: v1
     template:
       metadata:
          labels:
           app: c1
           version: v1
       spec:
         containers:
           - name: container-1
             image: centos:latest  # Image address. You change the address as needed.
             command:
               - /bin/bash
               - '-c'
               - for((i=0;i<=100000;i++)); do echo "$i hello"; sleep 10; done  # Container standard output logs
             resources:
               limits:
                 cpu: '1'
                 memory: 2Gi
               requests:
                 cpu: '1'
                 memory: 2Gi
         imagePullSecrets:
           - name: default-secret

   

3. In the navigation pane, choose Logging. Query the container standard output logs of the created pod by namespace, workload type, workload name, and pod name.

   

Configuring a Log Collection Policy for Container File Logs

1. Create a workload. The following is an example of collecting container file logs to a specified path:

   kind: Deployment
   apiVersion: apps/v1
   metadata:
     name: c1
     namespace: default
     labels:
       bursting.cci.io/burst-to-cci: enforce # Configure a forcible scheduling policy to schedule pods to CCI 2.0.
       version: v1
   spec:
     replicas: 1
     selector:
       matchLabels:
         app: c1
         version: v1
     template:
       metadata:
         creationTimestamp: null
         labels:
           app: c1
           version: v1
       spec:
         containers:
           - name: container-1
             image: centos:latest  # Image address. You change the address as needed.
             command:
               - /bin/bash
               - '-c'
               - for((i=0;i<=100000;i++)); do echo "$i hello" >> /var/log/container-1.log; sleep 10; done  # Container file logs collected to the specified path
             resources:
               limits:
                 cpu: '1'
                 memory: 2Gi
               requests:
                 cpu: '1'
                 memory: 2Gi
         imagePullSecrets:
           - name: default-secret

   

2. In the navigation pane, choose Logging. Click View Log Policy and create a policy to collect container file logs.

   

   

3. Select Custom Policy, enter a policy name, and select Container files for Log Type. Set Log Source to Workload and configure Namespace, Workload Name, Container, and Collection Path.

   

4. Select a log format as needed. (The single-line text is used as an example.) Use the default log group and log stream, or create a log group and log stream. Click OK.

   

   

   The log collection policy takes effect only for new pods. To collect the logs of existing pods, you need to perform a rolling release for these pods. If the logconfigs.logging.openvessel.io annotation of a new pod contains the configured log collection policy, the log collection policy has been applied to the pod.

   

5. In the navigation pane, choose Logging. Query the container file logs in the specified path of the created pod by namespace, workload type, workload name, and pod name.