Open MPI Delivered with the IB Driver

Updated on 2022-05-09 GMT+08:00

View PDF

Scenarios

This section describes how to run Open MPI (version 3.1.0rc2 is used as an example) delivered with the IB driver in a BMS cluster.

Prerequisites

Password-free login has been configured between BMSs in the cluster.
Open MPI delivered with the IB driver has been installed on all BMSs in the cluster.

Procedure

Disable the firewall.
1. Log in to a BMS in the cluster.
2. Run the following command to disable the BMS firewall:
  # service firewalld stop
  
  # iptables -F
3. Run the following command to check whether the firewall has been disabled:
  # service firewalld status
  
  Figure 1 Disabled firewall
4. Log in to all other BMSs in the cluster and repeat 1.b to 1.c to disable firewalls on all BMSs.
Modify the configuration file.
1. Log in to a BMS in the cluster.
2. Run the following command to obtain the BMS hostname:
  $ hostname
  
  Figure 2 Viewing the BMS hostname
3. Log in to all other BMSs in the cluster and repeat 1.b to 2.b to obtain hostnames of all BMSs.
4. Log in to a BMS in the cluster.
5. Run the following command to add the hosts configuration file:
  # vi /etc/hosts
  
  Add the private network IP addresses and hostnames of all BMSs in the cluster. For example, run the following commands:
  
  192.168.0.1 bms-0004
  
  192.168.0.2 bms-0005
  
  ...
6. Run the following command to add the hostfile file:
  $vi hostfile
  
  Add hostnames of all BMSs in the cluster, for example:
  
  bms-0004
  
  bms-0005
  
  ...
7. Log in to other BMSs in the cluster and repeat 2.e to 2.f.
Run MPI benchmark.
1. Perform the following command on any BMS to check whether the hostfile file has been configured:
  $ mpirun -np 2 -pernode --hostfile hostfile -mca btl_openib_if_include "mlx5_0:1" -x MXM_IB_USE_GRH=y hostname
  
  Figure 3 Checking the configuration file
  
  If the hostnames of all BMSs in the cluster are displayed, as shown in Figure 3, the hostfile file has been configured.
2. Run the MPI benchmark on any BMS with the hostfile path specified.
  For example, there are two BMSs in the cluster. Then, run the following command:
  
  $ mpirun -np 2 -pernode --hostfile hostfile -mca btl_openib_if_include "mlx5_0:1" -x MXM_IB_USE_GRH=y /usr/mpi/gcc/openmpi-3.1.0rc2/tests/imb/IMB-MPI1 PingPong
  
  Figure 4 Running Open MPI delivered with the IB drivers in the cluster
  
  If information shown in Figure 4 is displayed, Open MPI delivered with the IB driver is running in the cluster.