Difference between revisions of "Cray"

Revision as of 20:44, 9 May 2018

This page describes how to build mpich master from the Github repository

Cray build instructions

Building on the Cray is like other systems: you'll need fairly up-to-date Autotools (autoconf >= 2.67, automake >= 1.15, libtool >= 2.44). On Argonne's theta machine, you can find these in ~robl/soft/cray/autotools

configure options

Configure MPICH with these Cray specific options:

• --with-device=ch4:ofi:gni for the Cray transport
• --with-pmi=cray to pick up Cray PMI library from modules
• --with-pm=no: Hydra won't work on the cray. Instead, aprun launches the processes.
• --enable-ugni-static: Configures the UGNI provider in a way that might help the KNL nodes perform better

All in one place so you can cut and paste:

configure --with-device=ch4:ofi:gni --with-file-system=lustre --with-pm=no --with-pmi=cray --enable-ugni-static

Build notes

• libfabric will use standard atomic types if detected, but the Intel compilers do not export all the types needed. You will need

https://github.com/ofiwg/libfabric/pull/3984/ until we update libfabric to something newer than 1.6.0

• In fact, there are atomic types all over the place, and libfabric's gni provider doesn't have a fallback like the above general patch. For now load PrgEnv-gnu into your environment. That should bring in a fairly recent gcc (gcc-7.3 as of this writing)

Running notes

Since we used PrgEnv-gnu to build MPICH, you will need to make sure PrgEnv-gnu is in your environment. For me, my submitted jobs kept wanting to load PrgEnv-intel by default, so I had to insert

  module swap PrgEnv-intel PrgEnv-gnu

into my job submission script.