Updated on 2024-09-14 GMT+08:00

OOM Process Control Policy

Background

Both offline and online services can be configured in an OS. When Out Of Memory (OOM) occurs, the system preferentially ends the process that consumes the most memory in the offline service control group to reclaim the memory. However, some core services are often running offline. If the memory consumed by such services is reclaimed, the OS will be greatly affected.

To solve this problem, HCE adjusts the memory reclamation policy during OOM and adds the function of configuring cgroups priority. When the memory is insufficient, the kernel traverses cgroups, ends the processes for cgroups with low priorities, and reclaims the memory so that important offline services can keep running.

Prerequisites

vm.panic_on_oom is enabled by default. When the system is out of memory, a kernel panic occurs. Before you use memcg OOM for priority configuration, you must ensure that vm.panic_on_oom is disabled by running sysctl -w vm.panic_on_oom=0.

Interface Description

Interface

Description

Example Value

memcg_qos_enable

Specifies whether to enable memcg OOM priority configuration.

  • 0: Priority configuration is disabled. When OOM occurs, the process that consumes the most memory is ended based on the original OOM operation, and the memory is reclaimed.
  • 1: Priority configuration is enabled, and priorities are configured by cgroup. When OOM occurs, all processes in the cgroup with a lower priority are ended and the memory is reclaimed.
  • 2: Priority configuration is enabled, and priorities are configured by process. When OOM occurs, the largest process in the cgroup with a lower priority is ended and the memory is reclaimed.

The value is an integer ranging from 0 to 2. The default value is 0.

memory.qos_level

Specifies how to configure the priorities of cgroups. A smaller value indicates a lower priority.

  • When OOM occurs, the current cgroup is used as the parent cgroup, the process with the highest memory usage in the child cgroup with the lowest priority is ended, and the memory is reclaimed.
  • When OOM occurs, cgroups with the same priority will be sorted based on their memory usage, and the cgroup with the largest memory usage is ended.
NOTE:
  • Before using memory.qos_level, ensure that memcg_qos_enable is set to 1 or 2.
  • By default, the value of memory.qos_level of a newly created cgroup inherits the value of memory.qos_level of the parent cgroup. The priority of a child cgroup is not restricted by the parent cgroup.
  • If the priority of the parent cgroup is changed, the priorities of the child cgroups are automatically changed to be the same as that of the parent cgroup.

The value is an integer ranging from -1024 to 1023. The default value is 0.

Interface Configuration Example

Create six cgroups A, B, C, D, E and F, configure the memcg_qos_enable interface, and set the memcg OOM priorities by specifying memory.qos_level.

Table 1 Data planning

cgroup

Value of memory.qos_level

Description

A

-8

When the OOM operation is performed in the root cgroup, the kernel traverses all cgroups in the root cgroup and finally selects cgroups A and E, both with the lowest priority. Because A and E have the same priority, the kernel continues to compare the memory used by A and E.

  • If you set memcg_qos_enable to 1, the system preferentially ends all processes in the cgroup with a large memory usage and reclaims the memory.
  • If you set memcg_qos_enable to 2, the system ends the process with the largest memory usage in two cgroups and reclaims the memory.

B

10

C

1

D

2

E

-8

F

3

  1. Disable vm.panic_on_oom.
    sysctl -w vm.panic_on_oom=0
  2. Enable memcg OOM priority configuration.
    echo 1 > /proc/sys/vm/memcg_qos_enable
  3. Create cgroups A and B and set their memcg OOM priorities to -8 and 10.
    mkdir /sys/fs/cgroup/memory/A
    mkdir /sys/fs/cgroup/memory/B
    cd /sys/fs/cgroup/memory/A
    echo -8 > memory.qos_level
    cd /sys/fs/cgroup/memory/B
    echo 10 > memory.qos_level
  4. Create child cgroups C and D under cgroup A and child cgroups E and F under cgroup B, and set the memcg OOM priorities of cgroups C, D, E, and F to 1, 2, -8, and 3.
    mkdir /sys/fs/cgroup/memory/A/C
    mkdir /sys/fs/cgroup/memory/A/D
    mkdir /sys/fs/cgroup/memory/B/E
    mkdir /sys/fs/cgroup/memory/B/F
    cd /sys/fs/cgroup/memory/A/C
    echo 1 > memory.qos_level
    cd /sys/fs/cgroup/memory/A/D
    echo 2 > memory.qos_level
    cd /sys/fs/cgroup/memory/B/E
    echo -8 > memory.qos_level
    cd /sys/fs/cgroup/memory/B/F
    echo 3 > memory.qos_level