From Mpich
Revision as of 20:44, 9 May 2018 by Robl (talk | contribs)

Jump to: navigation, search

This page describes how to build mpich master from the Github repository

Cray build instructions

Building on the Cray is like other systems: you'll need fairly up-to-date Autotools (autoconf >= 2.67, automake >= 1.15, libtool >= 2.44). On Argonne's theta machine, you can find these in ~robl/soft/cray/autotools

configure options

Configure MPICH with these Cray specific options:

  • --with-device=ch4:ofi:gni for the Cray transport
  • --with-pmi=cray to pick up Cray PMI library from modules
  • --with-pm=no: Hydra won't work on the cray. Instead, aprun launches the processes.
  • --enable-ugni-static: Configures the UGNI provider in a way that might help the KNL nodes perform better

All in one place so you can cut and paste:

configure --with-device=ch4:ofi:gni --with-file-system=lustre --with-pm=no --with-pmi=cray --enable-ugni-static

Build notes

  • libfabric will use standard atomic types if detected, but the Intel compilers do not export all the types needed. You will need until we update libfabric to something newer than 1.6.0

  • In fact, there are atomic types all over the place, and libfabric's gni provider doesn't have a fallback like the above general patch. For now load PrgEnv-gnu into your environment. That should bring in a fairly recent gcc (gcc-7.3 as of this writing)

Running notes

Since we used PrgEnv-gnu to build MPICH, you will need to make sure PrgEnv-gnu is in your environment. For me, my submitted jobs kept wanting to load PrgEnv-intel by default, so I had to insert

  module swap PrgEnv-intel PrgEnv-gnu

into my job submission script.