Huge Pages

Background

For some frequently accessed applications with large code segments, the iTLB miss rate is high. This problem can be resolved by using huge pages. This feature is disabled by default in HCE. You can enable it by setting a parameter. Executable code of an application can be loaded into huge pages to reduce the iTLB miss rate and improve performance of databases (MySQL) and other large applications.

Enabling Huge Pages

• Check whether the kernel startup parameter exec_hugepages exists. If not, add it and restart the system.
• Enable huge pages for a single application.

  HUGEPAGE_ELF=1./app    // This environment variable will not be copied to a subprocess. HUGEPAGE_ELF=0 disables huge pages.

• Enable huge pages for applications marked by hugepageedit.

  export HUGEPAGE_PROBE=1    // Export the environment variable HUGEPAGE_PROBE. This variable can be copied to a subprocess.
  hugepageedit./app         // Use hugepageedit to mark binary files of the applications for which huge pages need to be enabled. hugepageedit is provided in the glibc-devel package.
  ./app                      // Start the applications to enable huge pages.

• Enable huge pages for all applications.

  echo 1 > /sys/kernel/mm/exec_hugepages/enabled    // echo 0 (default) indicates disabled. After this feature is enabled, all applications that meet the conditions will automatically use huge pages.

• Check whether huge pages are enabled.

  cat /proc/ Process ID/smaps | grep eh    // If a process is marked with eh, huge pages are enabled for this process.

Constraints

• If application code is not 2 MB aligned, the code that is less than 2 MB in each segment cannot use huge pages. So, alignment is recommended.
• mprotect() requires input addresses be 2 MB aligned. That is, their size must be a multiple of 2 MB. Application code needs to be modified accordingly.

  For address alignment of 2 MB, compile applications to add -Wl,-zcommon-page-size=2097152 -Wl,-zmax-page-size=2097152 to linker options.

• If reserved huge pages are insufficient for an application, the system will apply for a page of 2 MB. If the application fails due to memory fragments or other reasons, rollback to small pages will be performed. If multiple threads in an application roll back to small pages, SIGBUS will be triggered.