Updated on 2024-10-14 GMT+08:00

How Do I Use Events to Fix Abnormal Workloads?

If a workload is abnormal, you can check the pod events first to locate the fault and then rectify the fault.

Fault Locating

To check whether there is an abnormal pod in the workload, perform the following steps:

  1. Log in to the CCE console.
  2. Click the cluster name to access the cluster console. In the navigation pane, choose Workloads.
  3. In the upper left corner of the page, select a namespace, locate the target workload, and view its status.

    • If the workload is not ready, view pod events, and determine the cause. For details, see Viewing Pod Events.
    • If the workload is processing, wait patiently.
    • If the workload is running, no action is required. If the workload status is normal but it cannot be accessed, check whether intra-cluster access is normal.

      Log in to the CCE console or use kubectl to obtain the pod IP address. Then, log in to the node where this pod locates and run curl or use other methods to manually call the APIs. Check whether the expected result is returned.

      If {Container IP address}: {Port} cannot be accessed, log in to the service container and access 127.0.0.1: {Port} to locate the fault.

Viewing Pod Events

Method 1

On the CCE console, click the workload name to go to the workload details page, locate the row containing the abnormal pod, and click View Events in the Operation column.

Method 2

Run kubectl describe pod {Pod name} to view pod events. The following shows an example:

$ kubectl describe pod prepare-58bd7bdf9-fthrp
...
Events:
  Type     Reason            Age   From               Message
  ----     ------            ----  ----               -------
  Warning  FailedScheduling  49s   default-scheduler  0/2 nodes are available: 2 Insufficient cpu.
  Warning  FailedScheduling  49s   default-scheduler  0/2 nodes are available: 2 Insufficient cpu.
Table 1 Troubleshooting methods

Event

Pod Status

Solution

PodsScheduling failed

Pending

For details, see What Should I Do If Pod Scheduling Fails?

PodsFailed to pull image

Failed to re-pull image

FailedPullImage

ImagePullBackOff

For details, see What Should I Do If a Pod Fails to Pull the Image?

PodsCreation failed

Failed to restart container

CreateContainerError

CrashLoopBackOff

For details, see What Should I Do If Container Startup Fails?

The pod status is Evicted, and the pod keeps being evicted.

Evicted

For details, see What Should I Do If a Pod Fails to Be Evicted?

The storage volume fails to be mounted to the pod.

Pending

For details, see What Should I Do If a Storage Volume Cannot Be Mounted or the Mounting Times Out?

The pod stays Creating.

Creating

For details, see What Should I Do If a Workload Remains in the Creating State?

The pod stays Terminating.

Terminating

For details, see What Should I Do If Pods in the Terminating State Cannot Be Deleted?

The pod status is Stopped.

Stopped

For details, see What Should I Do If a Workload Is Stopped Caused by Pod Deletion?