Help Center/ ECS Instance Performance Evaluation/ Instance Performance Evaluation Guide/ Methods of Evaluating Instance Compute Performance
Updated on 2022-02-22 GMT+08:00
View PDF
Share

Methods of Evaluating Instance Compute Performance

Scenarios

This section describes how to use the SPEC CPU®2017 benchmark suite to evaluate the compute performance of a Linux instance.

Prerequisites

  • An ECS that is created using a public image on the cloud platform is available.
  • At least 2 GB of memory is reserved for each vCPU, and at least 10 GB of idle disk space is available for installing the test tool.

Preparations

The following uses an ECS running CentOS 7.4 as an example.

  1. Remotely log in to the ECS.
  2. Install dependencies.

    yum install -y m4 numactl* automake bison bzip2

  3. Upgrade GCC.

    wget https://benchmark-packages.oss-cn-qingdao.aliyuncs.com/gcc7.zip

    unzip gcc7.zip

    cd gcc7

    bash make_gcc.sh

    The installation takes about 40 minutes. Information similar to the following is displayed if the installation is successful:

    gcc version 7.3.0 (GCC)
  4. Configure environment variables.
    1. Run the following command to configure the environment variables:

      vim /etc/profile

    2. Add the following information to the configuration file: 
      export PATH=/usr/local/gcc/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/gcc/lib64:/usr/local/gmp/lib:/usr/local/mpfr/lib:/usr/local/mpc/lib:$LD_LIBRARY_PATH
export MANPATH=/usr/local/gcc/share/man:$MANPATH
    3. Run the following command for the environment variables to take effect:

      source /etc/profile

    4. Check the GCC version.

      gcc -v

      gcc version 7.3.0 (GCC)
  5. Upgrade glibc.

    In this example, glibc 2.27 is used, which must be installed in the GCC 7.3.0 environment. Otherwise, the system will display a message indicating an early GCC version.

    1. Upload the configuration file to the /home directory and run the following commands to install glibc:

      cd /home

      tar -zxvf glibc-2.27.tar.gz

      cd glibc-2.27

      mkdir build

      cd build

      export LD_LIBRARY_PATH=/usr/local/mpc-1.0.3/lib:/usr/local/gmp-6.1.0/lib:/usr/local/mpfr-3.1.4/lib:/usr/local/gcc-7.3.0/lib:/usr/local/isl-0.18/lib:/

      ../configure --prefix=/usr/local/glibc-2.27 --disable-profile --enable-add-ons --with-headers=/usr/include --with-binutils=/usr/bin --disable-sanity-checks --disable-werror

      make -j 40

      make -j 40 install

    2. Run the following command to configure the environment variables:

      vim /etc/profile

    3. Add the following information to the configuration file: 
      export PATH=/usr/local/glibc-2.27/bin:$PATH
    4. Run the following command for the environment variables to take effect:

      source /etc/profile

    5. Check the glibc version.

      ldd --version

      ldd (GNU libc) 2.27

Downloading the Benchmark Suite

Download the SPEC CPU®2017 benchmark suite.

In this example, the 1.0.5 version is used. You are advised to use SPEC CPU®2017 of version 1.0.5 or later.

Installing the Benchmark Suite

This section describes how to install SPEC CPU®2017.

  1. Upload the SPEC CPU®2017 installation package to the specified directory on the ECS.

    In this example, the installation directory is ./spec2017.

  2. Decompress speccpu2017_config.zip to the specified directory.

    In this example, the package is decompressed to cpu2017/config/.

    unzip speccpu2017_config.zip -d spec2017/config/

  3. Go to the SPEC CPU®2017 installation directory and install the tool.

    cd ./spec2017

    ./install.sh

    When the system displays a message asking you to check the installation directory, ensure that the directory is correct and press y.

    Information similar to the following is displayed if the installation is successful:

    Installation successful.

Evaluating the Instance Compute Performance

  1. Set environment variables.
    • Check the GCC and glibc versions.

      To do so, run the gcc -v and ldd --version commands, respectively.

      Ensure that the GCC version is 7.3.0 or later, and the glibc version is 2.27 or later.

    • Run the source shrc command to associate the environment variables and library files required by SPEC CPU®2017 with the /spec2017 directory.
  2. Modify the configuration file.

    Run the following commands to download the configuration file required for the test and copy the file to the config directory in the SPEC CPU®2017 installation folder:

    wget https://benchmark-packages.oss-cn-qingdao.aliyuncs.com/speccpu2017_config.zip

    unzip speccpu2017_config.zip -d /spec2017/config/

  3. Run the following command to perform a multi-vCPU RateInt test:

    ./bin/runcpu --config=spec17-opti-gcc7.3.cfg --copies=`cat /proc/cpuinfo | grep process | wc -l` --loose intrate

  4. Run the following command to delete the files generated during the test to release disk space:

    rm -fr benchspec/CPU/*/run/*

    By default, performing an intrate test consumes 3 GB to 4 GB of storage space in the disk. Therefore, delete the generated files after the test is complete.

  5. Perform the test multiple times if you need to obtain released test data. After the preceding operations are performed, the test is performed at the performance base only once, and the data cannot be released.
    • To perform the test for multiple times, add -n in the command.

      ./bin/runcpu --config=spec17-opti-gcc7.3.cfg --copies=`cat /proc/cpuinfo | grep process | wc -l` --loose intrate -n 20

    • To test a single item:

      For example, to test only 502.gcc_r, run the following command:

      ./bin/runcpu --config=spec17-opti-gcc7.3.cfg 502.gcc_r

    Table 1 SPEC CPU®2017 test items

    Rate Int

    Rate FP

    Speed Int

    Speed FP

    500.perlben_r

    503.bwaves_r

    600.perlbench_s

    603.bwaves_s

    502.gcc_r

    507.cactuBSSN_r

    602.gcc_s

    607.cactuBSSN_s

    505.mcf_r

    508.namd_r

    605.mcf_s

    619.lbm_s

    520.omnetpp_r

    510.parest_r

    620.omnetpp_s

    621.wrf_s

    523.xalancbmk_r

    511.povray_r

    623.xalancbmk_s

    627.cam4_s

    525.x264_r

    519.lbm_r

    625.x264_s

    628.pop2_s

    531.deepsjeng_r

    521.wrf_r

    631.deepsjeng_s

    638.imagick_s

    541.leela_r

    526.blender_r

    641.leela_s

    644.nab_s

    548.exchange2_r

    527.cam4_r

    648.exchange2_s

    649.fotonik3d_s

    557.xz_r

    538.imagick_r

    657.xz_s

    654.roms_s

    N/A

    544.nab_r

    N/A

    N/A

    N/A

    549.fotonik3d_r

    N/A

    N/A

    N/A

    554.roms_r

    N/A

    N/A

Obtaining the Evaluation Result

  • After the evaluation is complete, go to the /spec2017/result/ directory to obtain evaluation files.

    CPU2017.xxx.intrate.refrate.txt, CPU2017.xxx.fprate.refrate.txt, CPU2017.xxx.intspeed.refspeed.txt, and CPU2017.xxx.fpspeed.refspeed.txt

  • During the evaluation, if you want to obtain the result of the item for which the test is already complete, run the following command (value ratio is the result):

    less CPU2017.XXX.log.debug | grep Success