Failed to Start a Service

Symptom

After a service is started, the system displays a message, indicating a container startup failure.

Figure 1 Service startup failure

Possible Causes

Possible causes are as follows:

• The AI application is faulty and cannot be started.
• The port configured in the image is incorrect.
• The health check is incorrectly configured.
• The model inference code customize_service.py is incorrectly edited.
• The image fails to be pulled.
• Scheduling failed due to insufficient resources.

Faulty Model

If the image used to create the model has issues itself, you need to ensure that the image can start properly and can successfully return the expected content via a local curl request before creating the model.

Incorrect Port in the Image

The port enabled in the image is not 8080, or the port enabled in the image is different from the port configured during model creation. As a result, the register-agent cannot communicate with the model during service deployment. After a certain period of time (20 minutes at most), it is considered that starting the model fails.

If this fault occurs, check the port enabled in the custom image code and the port configured during model creation. Ensure that the two ports are the same. If you do not specify a port during model creation, ModelArts will listen to port 8080 by default. In this case, the port enabled in the custom image code must be 8080.

Figure 2 Port enabled in the custom image code

Figure 3 Port configured during model creation

Incorrect Health Check Configuration

If health check is enabled in the image, perform the following operations to locate the fault:

• Check whether the health check port runs properly.

  If health checks are configured in a custom image, you need to test whether the health check API works properly at the same time you test the image.

• Check whether the health check address configured during model creation is the same as the actual one.

  If the model is created using a base image provided by ModelArts, the health check URL must be /health by default.

  Figure 4 Configuring the health check URL
  

Incorrect customize_service.py

Check service runtime logs to locate the fault and rectify it.

Pulling an Image Failed

If the service fails to be started and a message is displayed indicating that the image fails to be pulled, see What Do I Do If an Image Fails to Be Pulled When a Service Is Deployed, Started, Upgraded, or Modified?

Scheduling Failed Due To Insufficient Resources

The service fails to be started, and a message is displayed indicating that resources are insufficient and service scheduling fails. For details, see Resources Are Insufficient When a Service Is Deployed, Started, Upgraded, or Modified.

Insufficient Memory

The service fails to be started, and a message is displayed indicating that the memory is insufficient. For details, see What Can I Do if the Memory Is Insufficient?.