Spectrum MPI

Scenarios

This section describes how to install and use IBM Spectrum MPI (for example, IBM Spectrum MPI v10.1).

IBM Spectrum MPI v10.1 supports the following OSs:

IBM Spectrum MPI 10.1.0.1 Eval for x86_64 Linux
- Red Hat Enterprise Linux version 6.6 and later
- Red Hat Enterprise Linux version 7.1 and later
- SUSE Linux Enterprise Server version 11 SP4
- SUSE Linux Enterprise Server version 12 and later
IBM Spectrum MPI 10.1.0.2 Eval for Power 8 Linux
- Red Hat Enterprise Linux version 7.3 and later

Prerequisites

You have configured the ECS password-free login.

Procedure

Obtain the software package.
1. Download the IBM Spectrum MPI software package from the following website:
  Download path: https://www-01.ibm.com/marketing/iwm/iwm/web/preLogin.do?source=swerpsysz-lsf-3
  
  The software package contains the license and software packages. The following are examples:
  
  smpi_lic_s-10.1Eval-rh7_Sep15.x86_64.rpm
  
  ibm_smpi-10.1.0.3eval_170901-rh7_Apr11.x86_64.rpm
2. Download the desired HPC-X toolkit.
  In the EDR SR-IOV scenario, the IBM MPI relies on the MXM library provided by the HPC-X toolkit. Download the desired version of the HPC-X toolkit based on the ECS OS and IB driver versions. An example of the HPC-X toolkit version is hpcx-v2.0.0-gcc-MLNX_OFED_LINUX-4.2-1.2.0.0-redhat7.3-x86_64.tbz.
  
  Download path: https://developer.nvidia.com/networking/hpc-x
Install the HPC-X toolkit.
1. Upload the HPC-X package downloaded in 1 to the ECS with an MPI.
2. Run the following command to decompress the HPC-X toolkit:
  $ tar xvf hpcx-v2.0.0-gcc-MLNX_OFED_LINUX-4.2-1.2.0.0-redhat7.3-x86_64.tbz
3. Run the following command to configure the HPC-X environment variables:
  $ cd hpcx-v2.0.0-gcc-MLNX_OFED_LINUX-4.2-1.2.0.0-redhat7.3-x86_64
  
  $ export HPCX_HOME=$PWD
Install IBM Spectrum MPI.
1. Upload the MPI package downloaded in 1 to the ECS with an MPI.
2. Run the following command to switch to user root:
  $ sudo su -
3. Run the following command to configure environment variables:
  - If you choose to automatically accept the IBM Spectrum MPI installation license agreement, run the following command:
    # export IBM_SPECTRUM_MPI_LICENSE_ACCEPT=yes
  - If you choose to manually accept the IBM Spectrum MPI installation license agreement, run the following command:
    # export IBM_SPECTRUM_MPI_LICENSE_ACCEPT=no
4. Install the license.
  - If you choose to automatically accept the IBM Spectrum MPI installation license agreement, run the following command:
    # rpm -ivh smpi_lic_s-10.1Eval-rh7_Sep15.x86_64.rpm
  - If you choose to manually accept the IBM Spectrum MPI installation license agreement, run the following command:
    # rpm -ivh ibm_smpi_lic_s-10.1Eval-rh7_Sep15.x86_64.rpm
    
    Figure 1 Manually accepting the IBM Spectrum MPI installation license agreement
    
    Run the following command as prompted:
    
    # sh /opt/ibm/spectrum_mpi/lap_se/bin/accept_spectrum_mpi_license.sh
5. Run the following command to install the software:
  # rpm -ivh ibm_smpi-10.1.0.3eval_170901-rh7_Apr11.x86_64.rpm
  
  Figure 2 Installing the software
Configure MPI environment variables.
1. By default, Spectrum MPI is installed in the /opt/ibm/spectrum_mpi directory. In this case, you need to configure the following environment variables:
  export MPI_ROOT=/opt/ibm/spectrum_mpi
  
  export LD_LIBRARY_PATH=$MPI_ROOT/lib:$LD_LIBRARY_PATH
  
  export PATH=$MPI_ROOT/bin:$PATH
  
  export MANPATH=$MPI_ROOT/share/man:$MANPATH
  
  unset MPI_REMSH
2. Run the following command to check whether the environment variables have been imported:
  # which mpirun
  
  Figure 3 Checking MPI environment variables

Run the following command on an ECS to run the executable file through Spectrum MPI:

Run the following command to edit the file:

# cd

# vi hello.c

Edit the following content:

#include<mpi.h>
#include<stdio.h>
int main(intargc, char**argv){
//Initialize the MPI environment
MPI_Init(NULL, NULL);
//Get the number of processes
int world_size;
MPI_Comm_size(MPI_COMM_WORLD, &world_size);
//Get the rank of the process
int world_rank;
MPI_Comm_rank(MPI_COMM_WORLD, &world_rank);
//Get the name of the processor
char processor_name[MPI_MAX_PROCESSOR_NAME];
int name_len;
MPI_Get_processor_name(processor_name, &name_len);     
//Print off a hello world message
printf("Hello world from processor %s, rank %d"" out of %d processors\n",      processor_name, world_rank, world_size);
//FinalizetheMPIenvironment.
MPI_Finalize();
}

Run the following command to generate an executable file (for example in the /home/linux directory).
# mpicc hello.c -o spe_hello

The hello file varies depending on the MPI version. If you update the MPI version, run the # mpicc hello.c -o spe_hello command to generate a new executable file.
Run the following command to run Spectrum MPI on an ECS:
# mpirun --allow-run-as-root -np 2 /root/spe_hello

Information shown in Figure 4 is displayed.

Figure 4 Command output