Help Center/ Cloud Container Engine/ User Guide/ Nodes/ Node O&M/ Optimizing Node System Parameters/ Changing the Maximum Number of File Handles
Updated on 2024-08-16 GMT+08:00
View PDF
Share

Changing the Maximum Number of File Handles

The maximum number of file handles is the maximum number of files that can be opened. In Linux, there are two file handle restrictions. The one is system-level restriction, where the maximum number of files that can be opened by all user processes at the same time. The other is user-level restriction, where the maximum number of files that can be opened by a single user process. Containers have the third file handle restriction, that is, the maximum number of file handles of a single process in the container.

The commands for modifying node system parameters are valid only when public images are used. The commands provided in this document are for reference only when private images are used.

Changing the Maximum Number of System-Level File Handles on a Node

  1. Log in to the node and check the /etc/sysctl.conf file. 

    cat /etc/sysctl.conf

  2. Modify the fs.file-max parameter. fs.file-max=1048576 indicates the kernel parameter name and recommended value.

    • If the value of fs.file-max has been set in the sysctl.conf file, run the following command to change the value: 
      sed -i "s/fs.file-max=[0-9]*$/fs.file-max=1048576/g" /etc/sysctl.conf && sysctl -p
    • If fs.file-max is not set in the sysctl.conf file, run the following command to add it: 
      echo fs.file-max=1048576 >> /etc/sysctl.conf && sysctl -p

  3. Run the following commands to check whether the change is successful (whether the returned value is the same as that you configure). 

    # sysctl fs.file-max
fs.file-max = 1048576

Changing the Maximum Number of File Handles for a Single Process on a Node

  1. Log in to the node and view the /etc/security/limits.conf file. 

    cat /etc/security/limits.conf

    The maximum number of file handles for a single process of a node is specified by the following parameters:

    ...
root soft nofile 65535
root hard nofile 65535
* soft nofile 65535
* hard nofile 65535

  2. Run the sed command to change the maximum number of file handles. In the command, 65535 is the recommended maximum number of file handles. The /etc/security/limits.conf file on the EulerOS 2.3 node does not contain the default configuration related to nofile. Therefore, you cannot run the sed command to modify the configuration. 

    sed -i "s/nofile.[0-9]*$/nofile 65535/g" /etc/security/limits.conf

  3. Log in to the node again and run the following command to check whether the modification is successful. If the returned value is the same as the modified value, the modification is successful. 

    # ulimit -n
65535

Changing the Maximum Number of File Handles for a Single Container Process

  1. Log in to the node and view the /usr/lib/systemd/system/docker.service file.

    • CentOS/EulerOS:
      • Docker nodes: 
        cat /usr/lib/systemd/system/docker.service
      • containerd nodes: 
        cat /usr/lib/systemd/system/containerd.service
    • Ubuntu:
      • Docker nodes: 
        cat /lib/systemd/system/docker.service
      • containerd nodes: 
        cat /lib/systemd/system/containerd.service

    If LimitNOFILE or LimitNPROC is set to infinity, the maximum number of file handles supported by a single process of a container is 1,048,576.

    The maximum number of file handles for a single process of a container is specified by the following parameters:

    ...
LimitNOFILE=1048576
LimitNPROC=1048576
...

  2. Run the following commands to modify the two parameters. In the command, 1048576 is the recommended value of the maximum number of file handles.

    Changing the maximum number of file handles of a container will restart the docker/containerd process.

    • CentOS/EulerOS:
      • Docker nodes: 
        sed -i "s/LimitNOFILE=[0-9a-Z]*$/LimitNOFILE=1048576/g" /usr/lib/systemd/system/docker.service;sed -i "s/LimitNPROC=[0-9a-Z]*$/LimitNPROC=1048576/g" /usr/lib/systemd/system/docker.service && systemctl daemon-reload && systemctl restart docker
      • containerd nodes: 
        sed -i "s/LimitNOFILE=[0-9a-Z]*$/LimitNOFILE=1048576/g" /usr/lib/systemd/system/containerd.service;sed -i "s/LimitNPROC=[0-9a-Z]*$/LimitNPROC=1048576/g" /usr/lib/systemd/system/containerd.service && systemctl daemon-reload && systemctl restart containerd
    • Ubuntu:
      • Docker nodes: 
        sed -i "s/LimitNOFILE=[0-9a-Z]*$/LimitNOFILE=1048576/g" /lib/systemd/system/docker.service;sed -i "s/LimitNPROC=[0-9a-Z]*$/LimitNPROC=1048576/g" /lib/systemd/system/docker.service && systemctl daemon-reload && systemctl restart docker
      • containerd nodes: 
        sed -i "s/LimitNOFILE=[0-9a-Z]*$/LimitNOFILE=1048576/g" /usr/lib/systemd/system/containerd.service;sed -i "s/LimitNPROC=[0-9a-Z]*$/LimitNPROC=1048576/g" /usr/lib/systemd/system/containerd.service && systemctl daemon-reload && systemctl restart containerd

  3. Check the maximum number of file handles of a single process in the container. If the returned value is the same as the modified value, the modification is successful.

    • Docker nodes: 
      # cat /proc/`pidof dockerd`/limits | grep files
Max open files            1048576              1048576              files
    • containerd nodes: 
      # cat /proc/`pidof containerd`/limits | grep files
Max open files            1048576              1048576              files

Automatically Configuring the Maximum Number of File Handles When Creating a Node or Node Pool

You can set the script to be executed after a node or node pool is created. When creating a node or node pool, you can use the script to configure the maximum number of file handles.

  1. Confirm the OS of the node or node pool to be created, for example, CentOS 7.6.
  2. Manually test the script commands on nodes in the same cluster and running the same OS.

  3. When creating a node or node pool, choose Advanced Settings > Post-installation Command to add commands. (The following commands must be configured after the verification is successful.)

    • Change the maximum number of system-level file handles on a node.
      • Log in to the node and check the /etc/sysctl.conf file. If the value of fs.file-max has been set in the file, run the following command to change it: 
        sed -i "s/fs.file-max=[0-9]*$/fs.file-max=1048576/g" /etc/sysctl.conf && sysctl -p
      • Log in to the node and check the /etc/sysctl.conf file. If the value of fs.file-max is not set in the file, run the following command to add it: 
        echo fs.file-max=1048576 >> /etc/sysctl.conf && sysctl -p

      In the preceding command, fs.file-max=1048576 indicates the kernel parameter name and recommended value.

    • Run the following command to change the maximum number of file handles for a single process on a node: 
      sed -i "s/nofile.[0-9]*$/nofile 65535/g" /etc/security/limits.conf

      In the preceding command, 65535 is the recommended maximum number of file handles.

    • Change the maximum number of file handles for a single process of a container.
      • CentOS/EulerOS: 
        sed -i "s/LimitNOFILE=[0-9a-Z]*$/LimitNOFILE=1048576/g" /usr/lib/systemd/system/docker.service;sed -i "s/LimitNPROC=[0-9a-Z]*$/LimitNPROC=1048576/g" /usr/lib/systemd/system/docker.service && systemctl daemon-reload && systemctl restart docker
      • Ubuntu: 
        sed -i "s/LimitNOFILE=[0-9a-Z]*$/LimitNOFILE=1048576/g" /lib/systemd/system/docker.service;sed -i "s/LimitNPROC=[0-9a-Z]*$/LimitNPROC=1048576/g" /lib/systemd/system/docker.service && systemctl daemon-reload && systemctl restart docker

      In the preceding command, 1048576 is the recommended maximum number of file handles.

    The command in the following figure is used only as an example. Change it as required.

  4. After the node is created, log in to the node to check whether the parameters are successfully modified.
Parent Topic: Optimizing Node System Parameters