Help Center/ Cloud Container Engine/ User Guide/ Nodes/ Node O&M/ Optimizing Node System Parameters/ Modifying Node Kernel Parameters
Updated on 2024-11-11 GMT+08:00
View PDF

Modifying Node Kernel Parameters

The default Linux kernel parameters may not satisfy all users. You can modify the /etc/sysctl.conf configuration file on the node to modify the kernel parameters.

  • The commands for modifying node system parameters are valid only when public images are used. The commands provided in this document are for reference only when private images are used.
  • After the node is restarted, run the sysctl -p command to update the parameter value.
Table 1 Kernel parameters of a node

Parameter

Parameter Location

Description

Recommended Value

file-max

/etc/sysctl.conf

Maximum number of file handles in the system, which can be adjusted as required.

Obtaining the parameter:

sysctl fs.file-max

fs.file-max=1048576

nf_conntrack_buckets

nf_conntrack_max

/etc/sysctl.conf

Capacity of the connection tracing table, which can be adjusted as required.

Bucket usage = [nf_conntrack_count]/[nf_conntrack_buckets]

If the CPU usage is greater than 0.7 for a long time, increase the value of buckets to lower the CPU usage to less than 0.7.

Obtaining the parameter:

sysctl net.netfilter.nf_conntrack_count
sysctl net.netfilter.nf_conntrack_buckets
sysctl net.netfilter.nf_conntrack_max

The default value is set based on the node memory size. To change the value, refer to the following formula:

  • net.netfilter.nf_conntrack_buckets = [nf_conntrack_count] / 0.7
  • net.netfilter.nf_conntrack_max = 4* [nf_conntrack_buckets]

net.netfilter.nf_conntrack_tcp_timeout_close

/etc/sysctl.conf

Expiration time of the entry of the connection in the close state in the connection tracking table. Shortening the expiration time can speed up the recycling.

Obtaining the parameter:

sysctl net.netfilter.nf_conntrack_tcp_timeout_close

net.netfilter.nf_conntrack_tcp_timeout_close=3

net.netfilter.nf_conntrack_tcp_be_liberal

/etc/sysctl.conf

The parameter value is 0 or 1.

  • 0: The function is disabled. All RST packets that are not in the TCP window are marked as invalid.
  • 1: The function is enabled. Only RST packets that are not in the TCP window are marked as invalid. In containers, enabling this parameter can prevent the bandwidth of TCP connections that have been translated using NAT from being limited.

Obtaining the parameter:

sysctl net.netfilter.nf_conntrack_tcp_be_liberal

net.netfilter.nf_conntrack_tcp_be_liberal=1

tcp_keepalive_time

/etc/sysctl.conf

Interval for sending keepalive detection messages through TCP. If this parameter is set to a large value, TCP connections may be suspended in the Close_wait phase for a long time, which exhausts system resources.

Obtaining the parameter:

sysctl net.ipv4.tcp_keepalive_time

net.ipv4.tcp_keepalive_time=600

tcp_max_syn_backlog

/etc/sysctl.conf

Maximum number of TCP half-connections, that is, the maximum number of connections in the SYN_RECV queue.

Obtaining the parameter:

sysctl net.ipv4.tcp_max_syn_backlog

net.ipv4.tcp_max_syn_backlog=8096

tcp_max_tw_buckets

/etc/sysctl.conf

Specifies the maximum number of sockets in the time-wait state that can exist at any time. If the parameter value is too large, node resources may be exhausted.

Obtaining the parameter:

sysctl net.ipv4.tcp_max_tw_buckets

net.ipv4.tcp_max_tw_buckets=5000

net.core.somaxconn

/etc/sysctl.conf

Maximum number of TCP connections and maximum size of the ESTABLISHED queue. If the parameter value is too small, the value may be insufficient.

Obtaining the parameter:

sysctl net.core.somaxconn

net.core.somaxconn=32768

max_user_instances

/etc/sysctl.conf

Maximum number of inotify instances allowed for each user. If the parameter value is too small, the number of inotify instances may be insufficient in containers.

Obtaining the parameter:

sysctl fs.inotify.max_user_instances

fs.inotify.max_user_instances=8192

max_user_watches

/etc/sysctl.conf

Maximum number of directories of all monitoring instances. If the parameter value is too small, the number of directories may be insufficient in containers.

Obtaining the parameter:

sysctl fs.inotify.max_user_watches

fs.inotify.max_user_watches=524288

netdev_max_backlog

/etc/sysctl.conf

Size of the packet receiving queue of the network protocol stack. If the parameter value is too small, the queue size may be insufficient.

Obtaining the parameter:

sysctl net.core.netdev_max_backlog

net.core.netdev_max_backlog=16384

net.core.wmem_max

net.core.rmem_max

/etc/sysctl.conf

Memory size (bytes) of the sending and receiving buffer. If this parameter is set to a small value, the memory size may be insufficient in large file scenarios.

Obtaining the parameter:

sysctl net.core.wmem_max
sysctl net.core.rmem_max

net.core.wmem_max=16777216

net.core.rmem_max=16777216

net.ipv4.neigh.default.gc_thresh1

net.ipv4.neigh.default.gc_thresh2

net.ipv4.neigh.default.gc_thresh3

/etc/sysctl.conf

Garbage collection optimization of ARP entries.

  • gc_thresh1: indicates the minimum number of entries that can be reserved. If the number of entries is less than the gc_thresh1 value, the garbage collector (GC) will not reclaim these entries. Do not modify the default parameter setting.
  • gc_thresh2: When the number of entries exceeds the value of this parameter, the GC will clear the entries that have been stored for more than 5 seconds. Do not modify the default parameter setting.
  • gc_thresh3: indicates the maximum number of non-permanent entries. If the system provides a large number of APIs or is directly connected to a large number of devices, increase the value of this parameter.

Obtaining the parameter:

sysctl net.ipv4.neigh.default.gc_thresh1
sysctl net.ipv4.neigh.default.gc_thresh2
sysctl net.ipv4.neigh.default.gc_thresh3

net.ipv4.neigh.default.gc_thresh1=0

net.ipv4.neigh.default.gc_thresh2=4096

net.ipv4.neigh.default.gc_thresh3=163790

vm.max_map_count

/etc/sysctl.conf

If this parameter is set to a small value, a message is displayed indicating that the space is insufficient during ELK installation.

Obtaining the parameter:

sysctl vm.max_map_count

vm.max_map_count=262144

Modifying Node Kernel Parameters

Table 1 lists the kernel parameters of nodes. The following describes how to change the value of tcp_keepalive_time, which indicates the interval for sending keepalive detection messages over TCP.

  1. Log in to the node and check the /etc/sysctl.conf file. 

    cat /etc/sysctl.conf

  2. Modify the net.ipv4.tcp_keepalive_time parameter. net.ipv4.tcp_keepalive_time=600 indicates the kernel parameter name and recommended value. For details about the recommended value, see Table 1.

    To modify other kernel parameters, replace the parameter names and values in the commands by referring to Table 1.

    • If net.ipv4.tcp_keepalive_time has been set in the sysctl.conf file, run the following command to change the value: 
      sed -i "s/net.ipv4.tcp_keepalive_time=[0-9]*$/net.ipv4.tcp_keepalive_time=600/g" /etc/sysctl.conf && sysctl -p
    • If net.ipv4.tcp_keepalive_time is not set in the sysctl.conf file, run the following command to add it: 
      echo net.ipv4.tcp_keepalive_time=600 >> /etc/sysctl.conf && sysctl -p

  3. Run the command in Table 1 to check whether the modification is successful. If the returned value is the same as the modified value, the modification is successful. 

    # sysctl net.ipv4.tcp_keepalive_time
net.ipv4.tcp_keepalive_time = 600

Automatically Configuring Kernel Parameters When Creating a Node or Node Pool

You can set the script to be executed after a node or node pool is created. When creating a node or node pool, you can use the script to configure kernel parameters.

The tcp_keepalive_time parameter is used as an example to describe how to change the interval for sending keepalive detection messages over TCP. The value is the recommended value in Table 1.

  1. Confirm the OS of the node or node pool to be created, for example, CentOS 7.6.
  2. Manually test the script commands on nodes in the same cluster and running the same OS. For details about how to manually run the script, see Modifying Node Kernel Parameters.
  3. When creating a node or node pool, choose Advanced Settings > Post-installation Command to add commands. (The following commands must be configured after the verification is successful.) To modify other kernel parameters, replace the parameter names and values in the commands by referring to Table 1.

    • Log in to the node and check the /etc/sysctl.conf file. If net.ipv4.tcp_keepalive_time has been set in the file, run the following command to change it: 
      sed -i "s/net.ipv4.tcp_keepalive_time=[0-9]*$/net.ipv4.tcp_keepalive_time=600/g" /etc/sysctl.conf && sysctl -p
    • Log in to the node and check the /etc/sysctl.conf file. If net.ipv4.tcp_keepalive_time is not set in the file, run the following command to add it: 
      echo net.ipv4.tcp_keepalive_time=600 >> /etc/sysctl.conf && sysctl -p

    The command in the following figure is used only as an example. Change it as required.

  4. After the node is created, log in to the node and run the command in Table 1 to check whether the modification is successful.
Parent Topic: Optimizing Node System Parameters