Example 4: Submitting OpenMPI jobs

OpenMPI is more flexible than mvapich 2 with the nodes file and can directly use the PBS_NODEFILE given by the job scheduler. The setup of the GNU version of OpenMPI is described here. Before we compile our example program, we have to make sure that we run the right compiler:

 $ module load gnu-openmpi

We can then compile our trap.c file like:

 mpicc -o trap-openmpi trap.c 

to obtain the OpenMPI executable trap-openmpi. This can now be run with the MOAB script below:

#!/bin/bash

#MOAB -l nodes=4
#MOAB -j oe
#MOAB -l walltime=120:00

#MOAB -N TRAP-OPENMPI
module load gnu-openmpi

mpirun $PBS_O_WORKDIR/trap-openmpi

Save the moab script as trap-openmpi.sh. Submit the job to MOAB using:

$ msub trap-openmpi.sh 

Once the job has completed, you should receive an output file: TRAP-OPENMPI.o which looks like:

Number of procs 4 With n = 1024 trapezoids, 
our estimate of the integral from 0.000000 
to 1.000000 = 1.000000

This example was done using the gcc version of OpenMPI. There is also a version of Intel OpenMPI available. The setup of OpenMPI Intel is described here.