On this page

Show all

Help Center/ Cloud Container Engine_Autopilot/ FAQs/ Workloads/ Monitoring Logs/ Why Is the Reported Container Memory Usage Inconsistent with the Auto Scaling Action?

Why Is the Reported Container Memory Usage Inconsistent with the Auto Scaling Action?

Updated on 2024-11-08 GMT+08:00

View PDF

Symptom

The memory usage of a container being monitored does not match the auto scaling requirements. For example, the GUI shows that memory usage of a container is around 40%, but the HPA scale-in threshold is set at 70%. The memory usage displayed on the GUI is below the HPA threshold, but no scale-in action has taken place.

Possible Causes

The way the container memory usage is calculated on the GUI differs from the method used for HPA auto scaling.

The memory usage of a container displayed on the GUI: container_memory_rss/Memory limit of the container
container_memory_rss specifies the resident set size (RSS). It includes some memory parts that may not be actively or effectively used.
The memory usage of a container calculated for HPA auto scaling: container_memory_working_set_bytes/Memory request of the container
container_memory_working_set_bytes specifies the working set size (WSS) and is calculated by doing as follows:

Run cat /sys/fs/cgroup/memory/memory.stat in the pod to obtain the values of total_cache (cache memory), total_rss (memory used by the current application process), and total_inactive_file (memory used by inactive files).

WSS = The value of total_cache + The value of total_rss - The value of total_inactive_file

If an application's memory usage displayed on the GUI is below the HPA scale-in threshold, but no scale-in action has been taken, or if the memory usage is not higher than the HPA scale-out threshold, but a scale-out happened anyways, the HPA scale-out behavior may not work as expected. This can occur when:

The application cache usage is high, which can cause the WSS to be significantly greater than the RSS, resulting in the container memory usage displayed on the GUI being lower than the memory usage calculated by the HPA.
The difference between the resource limit and request is large. The request may be significantly lower than the limit, causing the container memory usage shown on the GUI to be lower than the memory usage calculated by the HPA.

Parent Topic: Monitoring Logs

Feedback

Was this page helpful?

Helpful Not helpful

Provide feedback

Thank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.

The system is busy. Please try again later.

Which of the following issues have you encountered?

Content is inconsistent with the product UI

Unclear descriptions

Lack of examples or code

Incorrect steps

Can't find what I need

Lack of best practices

Feedback (optional)

0/500

Select at least one type of issue, and enter your comments or suggestions.

Enter a maximum of 500 characters.

Submit Cancel

For any further questions, feel free to contact us through the chatbot.

Chatbot