How Can I Locate the Root Cause If a Workload Is Abnormal?

Step 1: Check the Basic Workload Status (Quickly Determine the Exception Type)

1. Log in to the CCE console and click the cluster name to access the cluster console.
2. In the navigation pane, choose Workloads and select a namespace in the upper left corner of the page.
3. Locate the target workload and check its status. The following scenarios cover the core workload states:
   • Not ready: Check for pod exceptions. For details, see Step 2: Check for Pod Exceptions (Locate the Root Cause of Workload Exceptions).
   • Being processed: Wait for the operation to complete. If the fault persists for an extended period, follow the troubleshooting steps for the Not ready scenario.
   • Running: No action is required. However, if the workload is running but inaccessible, check whether intra-cluster access is functioning normally. For details, see Step 3: Check for Access Exceptions (Workload Is Normal But Cannot Be Accessed).

Step 2: Check for Pod Exceptions (Locate the Root Cause of Workload Exceptions)

View pod events to identify the cause. Refer to Viewing Pod Events for the appropriate solution based on the specific event.

The pod statuses listed in the table below are obtained from the STATUS field in the output of kubectl get pod.

STATUS is a more granular status generated by kubectl based on state.Phase, state.Conditions, and status.ContainerStatuses.

Step 3: Check for Access Exceptions (Workload Is Normal But Cannot Be Accessed)

If the workload is running but inaccessible, perform the following troubleshooting steps:

1. Obtain the pod IP address on the CCE console or by running the kubectl command.
2. Log in to a node or container in the cluster, manually call the API using curl, and check whether the container port is correctly configured.
3. If {container-IP}:{port} is inaccessible, log in to the service container and attempt to access 127.0.0.1:{port}.

   For details, see Logging In to a Container.

If the workload is running but not functioning properly, or if the access result does not meet expectations, see What Should I Do If a Workload Appears to Be Normal But Is Not Functioning Properly?.

Checking Pod Status

1. Log in to the CCE console.
2. Click the cluster name to access the cluster console. In the navigation pane, choose Workloads.
3. Click the name of the target workload and check the pod statuses.

   

Viewing Pod Configurations

1. Log in to the CCE console.
2. Click the cluster name to access the cluster console. In the navigation pane, choose Workloads.
3. Click the name of the target workload. In the workload pod list, locate the row containing the target pod and choose More > View YAML in the Operation column.

   

Viewing Pod Events

Method 1

On the CCE console, click the workload name to go to the workload details page, locate the row containing the abnormal pod, and choose More > View Events in the Operation column.

Figure 1 Viewing pod events

Method 2

Use the kubectl command:

kubectl describe pod {pod-name}

Information similar to the following is displayed:

...
Events:
  Type     Reason            Age   From               Message
  ----     ------            ----  ----               -------
  Warning  FailedScheduling  49s   default-scheduler  0/2 nodes are available: 2 Insufficient cpu.
  Warning  FailedScheduling  49s   default-scheduler  0/2 nodes are available: 2 Insufficient cpu.

Viewing Container Logs

1. Log in to the CCE console.
2. Click the cluster name to access the cluster console. In the navigation pane, choose Workloads.
3. Locate the row containing the target workload and click View Log in the Operation column.

   

4. Switch between pods and containers above the logs.

   

Logging In to a Container

You can log in to a container using kubectl. For details, see Logging In to a Container.

1. Log in to the CCE console.
2. Click the cluster name to access the cluster console. In the navigation pane, choose Workloads.
3. Click the name of the target workload. In the workload pod list, locate the row containing the target pod and choose More > Remote Login in the Operation column.