From Mpich
Revision as of 14:16, 10 July 2013 by Robl (talk | contribs) (Blue Gene/Q build instructions)

Jump to: navigation, search

This page describes how to build mpich from the master branch of the mpich.git repository on

Blue Gene/Q build instructions

The bgq toolchain must be in the $PATH before configure, otherwise the gnu cross-compiler will not be found.

export PATH=$PATH:/bgsys/drivers/V1R2M0/ppc64/gnu-linux/bin

The alternative is to provide the location of all compiler binaries using environment variables. For example,


The configure is simpler when specifying the $PATH, however the generated mpi compile scripts such as ${prefix}/bin/mpicc will also not contain the path information to the cross compiler. This means users of mpicc must also have the cross compiler in their $PATH for the compile script to work.

If one wishes to build MPICH with the XL compilers, use the environment variable approach.


Specify the bgq cross compile and pamid device


Customize the ROMIO file system.



Customize the required bgq system software libraries

The latest installed bgq system software is used by default. The location of the bgq system software can also be specified with the configure option below or the BGQ_INSTALL_DIR environment variable.


A pami installation outside of the bgq system software directory may be specified using the --with-pami configure option(s). For example:


Customize the bgq cross compile settings

A different cross compile settings file for bgq pamid can be specified using the --with-cross-file configure option. Below is the configure option that specifies what is the default cross file for a bgq pamid configuration.


Disable rpath

When shared libraries are installed it is recommended to also disable the "wrapper rpath" configure option in order to take advantage of a shared library load optimization on the bgq io nodes.


When a million processes each individually read from the filesystem the performance of the shared library load will be poor. The io node shared library optimization is a way to "stage" shared libraries on a bgq io node ramfs directory that is, in the absence of rpath information, searched first by the bgq loader. Any rpath information will be searched before this io node ramfs location and will result in a query all the way down to the filesystem.

Shared libraries can be added to the io node ramfs directory by packaging the libraries into a *.tar.gz file and copying that file into the /bgsys/linux/bgfs directory.

Enable common "no debug" and "performance" options

The xl.ndebug and xl.legacy.ndebug mpich versions installed with the bgq system software use the following options to eliminate debug and other error checks that would cause performance degradations.


Enable fine grain locking

The gcc, xl, and xl.ndebug mpich versions installed with the bgq system software use the following options to enable fine grain locking and synchronous progress mode.