Enabling Yarn CGroups to Limit the Container CPU Usage

Scenario

CGroups is a Linux kernel feature. It groups task sets and their subsets into hierarchical structures with customizable behavior. In Yarn, this feature allows you to easily limit the resources (such as CPU) used by containers.

Currently, CGroups is only used for limiting the CPU usage.

This section is available for clusters of MRS 3.x and later.

Configuration Description

For details about how to configure CPU isolation and secure CGroups, visit the Hadoop official website:

Versions earlier than MRS 3.2.0: http://hadoop.apache.org/docs/r3.1.1/hadoop-yarn/hadoop-yarn-site/NodeManagerCgroups.html

MRS 3.2.0 or later: https://hadoop.apache.org/docs/r3.3.1/hadoop-yarn/hadoop-yarn-site/NodeManagerCgroups.html

CGroups is a Linux kernel feature and is enabled using LinuxContainerExecutor. For details about how to configure the LinuxContainerExecutor for security, see the official website. You can learn the file system permissions assigned to users and user groups from the official documentation at:

Versions earlier than MRS 3.2.0: http://hadoop.apache.org/docs/r3.1.1/hadoop-project-dist/hadoop-common/SecureMode.html#LinuxContainerExecutor

MRS 3.2.0 or later: https://hadoop.apache.org/docs/r3.3.1/hadoop-project-dist/hadoop-common/SecureMode.html#LinuxContainerExecutor

• Do not modify users, user groups, and related permissions of various paths in the corresponding file system. Otherwise, functions of CGroups may become abnormal.
• If the parameter value of yarn.nodemanager.resource.percentage-physical-cpu-limit is too small, the number of available cores may be less than one. For example, if the parameter of a four-core node is set to 20%, the number available core is less than one. As a result, all cores will be used. The CPUset mode can be used for Linux versions, for example, Cent OS, that do not support the Quota mode.

The table below describes the parameter for configuring cpuset mode, that is, only configured CPUs can be used by Yarn. Add the following parameters on Manager.

The table below describes the parameter for configuring strictcpuset mode, that is, only configured CPUs can be used by container. Add the following parameters on Manager.

To switch from cpuset mode to quota mode, the following conditions must be met:

• Set the yarn.nodemanager.linux-container-executor.cgroups.cpu-set-usage parameter to false.
• Delete the container folder (if any) from the /sys/fs/cgroup/cpuset/hadoop-yarn/ directory.
• Delete all CPUs configured in the cpuset.cpus file in /sys/fs/cgroup/cpuset/hadoop-yarn/.

Procedure

1. Log in to Manager. Choose Cluster > Services > Yarn. Click Configurations and click All Configurations.
2. In the navigation pane on the left, choose NodeManager > Customization and find the yarn-site.xml file.
3. Add the parameters in Table 1 and Table 2 as user-defined parameters.

   

   Based on the configuration files and parameter functions, locate the row where parameter yarn-site.xml resides. Enter the parameter name in the Name column and enter the parameter value in the Value column.

   Click + to add a customized parameter.

4. Click Save. In the displayed Save Configuration dialog box, confirm the modification and click OK. Click Finish when the system displays "Operation succeeded". The configuration is successfully saved.

   After the configuration is saved, restart the Yarn service whose configuration has expired for the configuration to take effect.